[tor-project] PieroV's Monthly Status Report, June 2024

[Pier Angelo Vendrame]

Hi everyone!
Here is my status report for June 2024.

This month, I worked almost exclusively on the ESR transition.
Once we got the tags for Firefox 128.0b1, I rebased my RR branch once 
again and opened the MR [0] to merge it to our first tor-browser-128 branch.
After that, I finalized the document with comments on the range-diff 
converted to HTML with the procedure described in my previous report.
It forced me to be more meticulous about every change, and I found some 
errors I had initially missed.
You can find it in the linked MR.

After that, I switched to the tor-browser-build part. First for desktop, 
then for Android. Linux and macOS were smooth, and the builds were 
reproducible at the first attempt.

Windows was more involved for a couple of reasons. First, Firefox now 
requires the windows-rs crate. Mozilla decided not to vendor it for some 
reason I haven't investigated.
Then, I had problems with llvm-windres. I solved this by restoring the 
wrapper the upstream project deleted some years ago [1]. Indeed, Mozilla 
is still using it, but their version would break OpenSSL, so I used a 
more recent one.
After this, I could get a build, but it wasn't reproducible. I noticed 
that only some binaries were different, and my hypothesis was that it 
was binaries that included some Rust code.
We verify reproducibility by building in different machines (I used my 
workstation and one of our build servers). We don't share any artifacts: 
every machine has to compile everything from scratch (we bootstrap 
several compilers, including Rustc).
So, I verified my conjecture using the Rustc artifact I created on my 
computer on the build server, and it solved the reproducibility problem.
The differences were only in std, so I think they were due to the 
differences in the GCC-based mingw-w64 toolchain.
We've had a ticket about building Rustc with the LLVM-based mingw 
toolchain we use for everything else [2] for a long time, but it isn't 
trivial due to Rust's hard dependency on libgcc. A series of new target 
triples [3] was created to avoid hacking around to resolve this problem, 
but Firefox doesn't recognize them (and Mozilla doesn't officially 
support this toolchain).
I wrote a small patch to solve the problem, and the build became 
reproducible. We will have to discuss more about the disadvantages of 
this approach.

The Android part was much more involved. Our builds run offline, whereas 
Mozilla's don't. So, I had to write several workarounds, including a 
patch for a custom Gradle plugin, and I wrote some Groovy for the first 
time in my life 😄️.
The monorepo migration was not troublesome at all: the commands we used 
to build in firefox-android.git still work without changes. Eventually, 
our build scripts were simplified a little bit.
The Android builds aren't reproducible yet. One reason is that only 
GeckoView was patched; our Android Components and Fenix patches haven't 
been rebased yet, and they include some changes to make the builds 
reproducible. Another reason is that the developments on nimbus-fml 
between 115 and 128 introduced a regression, and its output became 
non-deterministic. I wrote a patch and opened a PR [4]. I expect all the 
reproducibility problems to disappear once our patches are rebased.
The merge request with all the build changes [5] has been reviewed and 
approved, but I am waiting for the rebase MR to be merged first.

Apart from the ESR transition work, I rebased 13.0 and 13.5 onto Firefox 
115.12.0, did some QA on the 13.5 RCs, and investigated the 
reproducibility problem we had with the 13.5 builds.

Cheers,
Pier

[0] 
https://gitlab.torproject.org/tpo/applications/tor-browser/-/merge_requests/1043
[1] 
https://github.com/mstorsjo/llvm-mingw/commit/8915db817ce728833b8628abadedc7fbc64d5d85
[2] 
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/29320
[3] https://doc.rust-lang.org/rustc/platform-support/pc-windows-gnullvm.html
[4] https://github.com/mozilla/application-services/pull/6283
[5] 
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_requests/989