February 2020 - tbb-dev - lists.torproject.org

Re: [tbb-dev] [tor-packagers] torbrowser: problems with 9.0.1 source tarball
by Nicolas Vigier 17 Mar '20

17 Mar '20

Hi! I think the best list to discuss Tor Browser build problems is the tbb-dev(a)lists.torproject.org list, so I'm anwsering there. On Fri, 15 Nov 2019, Thomas Klausner wrote: > Hi! > > I wanted to build firefox-tor-browser-68.2.0esr-9.0-1-build2 > (torbrowser 9.0.1) on NetBSD from the source tarball, but I stumbled > over a couple problems. > > The first one, .mozconfig contains --disable-eme but configure doesn't like it: > > mozbuild.configure.options.… [View More]

5 6

How Tails is special in its use of Tor Browser
by intrigeri 18 Feb '20

18 Feb '20

Hi, we've improved our documentation about how Tails is special in its use of Tor Browser: https://tails.boum.org/contribute/design/#tor-browser I hope it helps your team keep our "special needs" in mind :) If anything is unclear, please let me know. Cheers!

3 3

Proposal process improvements
by Georg Koppen 15 Feb '20

15 Feb '20

Hello! We had been talking about potential improvements to our proposal process back at our meeting in Stockholm. We should not drop that topic (even though it looks like we did :) ) and I thought it would be smart to have a further discussion per email. To recap: Tor Browser proposals are a way to think about large/impactful browser features in a more formal way, including all stakeholders. That is, by describing those new features, what motivates them, the pros/cons, potential security/… [View More]

2 1

Very very slow tor-browser-build build; catastrophic recursion in rbm?
by David Fifield 12 Feb '20

12 Feb '20

I did a build of tor-browser-build build and it took even longer than I remember it taking -- from Friday until Tuesday, roughly. This is on a VPS with six 2 GHz CPUs and 16 GB RAM. Some projects I know take a long time to build, like rust and firefox, but it seemed like most of the time was being spent in the "rbm" process, not in building the projects. I don't know much about profiling Perl, but I tried a tool called NYTProf that produces HTML output: https://people.torproject.org/~dcf/… [View More]graphs/nytprof-rbm-20200212-94fce31e/ The above NYTProf output is not from a full build. It's from modifying projects/snowflake/config only and then building snowflake for only one target; still it took almost an hour. perl -d:NYTProf rbm/rbm build snowflake --target testbuild --target torbrowser-linux-x86_64 nytprofhtml To my untrained eye it looks like there is some kind of exponential recursion happening around the RBM::config_p function. See here, where in building one project the function was called 89 million times: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/#su… Clicking through to the per-line view, we see that 45 million of those calls were from RBM::get_target, and most of the rest were from RBM::config: https://people.torproject.org/~dcf/graphs/nytprof-rbm-20200212-94fce31e/roo… You know how, if you write a Fibonacci number function and don't memoize the recursive calls, it ends up re-computing the same subtrees over and over and doing exponential work? def fib(n): if n == 0 or n == 1: return 1 return fib(n - 1) + fib(n - 2) It feels like something like that is happening here. There's a pattern where RBM::process_template calls a chain of functions that reaches RBM::config_p, which eventually calls RBM::process_template again. It's not an infinite loop, but it seems to be doing the same work many times. I looked at it for a bit but my Perl isn't strong enough to understand specifically what's happening. Notice how there are three loops of RBM::process_template before doing an RBM::config_p that calls back into further RBM::process_template. RBM::process_template Template::process Template::Service::process Template::Context::process Template::Document::process Template::Document::__ANON__[input text:10] Template::Stash::XS::get RBM::__ANON__[rbm/lib/RBM.pm:640] RBM::project_config RBM::process_template Template::process Template::Service::process Template::Context::process Template::Document::process Template::Document::__ANON__[input text:9] Template::Stash::XS::get RBM::__ANON__[rbm/lib/RBM.pm:640] RBM::project_config RBM::process_template Template::process Template::Service::process Template::Context::process Template::Document::process Template::Document::__ANON__[input text:18] Template::Stash::XS::get RBM::__ANON__[rbm/lib/RBM.pm:640] RBM::project_config RBM::config RBM::config_p RBM::DefaultConfig::__ANON__[rbm/lib/RBM/DefaultConfig.pm:606] RBM::input_files RBM::project_step_config RBM::project_config RBM::process_template Template::process ... et cetera ... Some more evidence is that while I was waiting for rbm to do something, I was watching the modification times of the git_clones: ls -ltrhd git_clones/*/.git | tail -n 15 It was constantly touching all different projects, but re-touching the same ones over and over. [View Less]

3 5

building tb-browser 9.0.4 / how to identify if src-firefox is the right one? or a release candidate?
by clematis 12 Feb '20

12 Feb '20

Hello, As people were porting tor-browser 9.0.4 to OpenBSD ports[1], there's been some concern it was built on top of a release candidate tarball (src-firefox-tor-browser-68.4.1esr-9.0-1-build1). So two questions: 1/ How to identify a release candidate tarball? Not sure about the naming convention for the distfile. 2/ This seems to be the only one in https://dist.torproject.org/torbrowser/9.0.4/ , so if this is a RC is that still OK? And was it expected? Thank you. [1] https://marc.info/?… [View More]

2 2

Tor Browser Moving to Release Train
by Matthew Finkel 03 Feb '20

03 Feb '20

Hi everyone, Following the Mozilla All-Hands last week, we've made the decision that Tor Browser will move from being based on the Firefox ESR (extended support release) train to the Firefox (rapid) Release train. This is a large change for us, and the decision was not made easily. We finally made this decision due to the development process of Mozilla's new Android browser (Fenix) and how we plan on building Tor Browser based on it (as well as maintaining and developing the desktop browser in … [View More]

1 0

Nightly rebase experiment proposal
by Arthur D. Edelstein 03 Feb '20

03 Feb '20

Hi everyone, Here's a plan for experimenting with moving Tor Browser from ESRs to mozilla-central. Why try this? There are several reasons I can think of: * There is no mobile Firefox ESR. In order to keep mobile Tor Browser secure, we would either need to constantly monitor for security fixes and apply backports. It will be simpler to follow the Mozilla release cycle. * Tor Browser will receive all security patches and security features from Mozilla, not simply those deemed important enough … [View More]to backport to the last ESR. * Closer collaboration between Mozilla and the Tor Browser team will be possible, because everyone will be focused on the same codebase. For example, non-urgent patches can land directly on mozilla-central, bypassing tor-browser.git altogether, and those patches will be deployed to Tor Browser faster than they would have been on an ESR-only schedule. Mozilla will provide expert review of patches. Also, some non-urgent patches by Tor Browser may be written by Mozilla engineers. * The Tor Browser development process will be smoother and more gradual. Instead of rebasing and patch-fixing sprints ahead of each ESR, we can automatically rebase patches every night, and fix any patch whenever its rebase fails. (The total rebase effort should be approximately the same as before.) * Patches that frequently break because they live on "hot code" can be identified and uplifted. * We'll need to continuously monitor the mozilla codebase for new features that could harm privacy. That's potentially difficult, but by focusing on Mozilla's current work, we can hopefully stop patches that affect privacy before they land. Here's how I envision the new rebase process: * An automated script rebases all tor browser patches every night against mozilla-central, mozilla-beta, mozilla-stable and mozilla-esr. An email alert will be sent if the rebase of any patch fails. (I have a prototype script that does the basic rebasing -- it found that ~40/140 patches from our ESR-52 branch could be auto-rebased onto mozilla-central without any intervention.) * Any failed patches will need to be manually fixed up promptly so that the automated rebase can resume. * Diagnostic Tor Browser bundles will be built against the latest rebase branches. We can also run regression tests. So how do we ramp up this experiment? 1. 2018-1-1 to 2017-2-22: Rebase tor-browser.git patches manually on top of Firefox 59-beta. (We have to do this anyway.) 2. 2017-2-22 to 2018-3-13: Activate a script that rebases Tor Browser patches nightly onto mozilla-(central, beta, stable, esr). 3. 2018-03-13 to 2018-05-07: Create nightly Tor Browser bundle builds based on nightly rebases. At this point, we can evaluate whether we would like to transition to releasing Tor Browser alphas based on the mozilla-beta branch. Switching Tor Browser stable to mozilla-stable can happen sometime after that. How does this sound? Interested to hear what you think. Thanks, Arthur [View Less]

7 12