[tbb-bugs] #26475 [Applications/Tor Browser]: ESR60-based .dmg images are not built reproducibly with Stylo enabled using rustc > 1.25.0

[Tor Bug Tracker & Wiki]

#26475: ESR60-based .dmg images are not built reproducibly with Stylo enabled using
rustc > 1.25.0
-------------------------------------------------+-------------------------
 Reporter:  gk                                   |          Owner:  tbb-
                                                 |  team
     Type:  defect                               |         Status:
                                                 |  needs_information
 Priority:  Very High                            |      Milestone:
Component:  Applications/Tor Browser             |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  tbb-rbm, GeorgKoppen201808,          |  Actual Points:
  TorBrowserTeam201808                           |
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by gk):

 Replying to [comment:34 alexcrichton]:
 > Nice! Bisecting to the LLVM upgrade definitely makes sense to me. That
 was a massive LLVM upgrade though (from LLVM 4.0 to LLVM 6.0), so that
 would be quite the bisection range for a regression to be introduced in :(
 >
 > If it works, though, when you build LLVM yourself that's quite curious.
 The command we use to build LLVM though is pretty huge. Looking at one of
 our  recent builds (https://travis-ci.org/rust-lang/rust/builds/411219658)
 the command we use on OSX is:
 >
 > {{{
 > "cmake" "/Users/travis/build/rust-lang/rust/src/llvm"
 "-DLLVM_ENABLE_ASSERTIONS=OFF"
 "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;Mips;PowerPC;SystemZ;MSP430;Sparc;NVPTX;Hexagon"
 "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly;RISCV"
 "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_TESTS=OFF"
 "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_ENABLE_ZLIB=OFF" "-DWITH_POLLY=OFF"
 "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF"
 "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=4"
 "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-apple-
 darwin" "-DLLVM_OCAML_INSTALL_PATH=usr/lib/ocaml"
 "-DCMAKE_EXE_LINKER_FLAGS=-static-libstdc++" "-DCMAKE_C_COMPILER=sccache"
 "-DCMAKE_C_COMPILER_ARG1=/Users/travis/build/rust-
 lang/rust/clang+llvm-6.0.0-x86_64-apple-darwin/bin/clang"
 "-DCMAKE_CXX_COMPILER=sccache"
 "-DCMAKE_CXX_COMPILER_ARG1=/Users/travis/build/rust-
 lang/rust/clang+llvm-6.0.0-x86_64-apple-darwin/bin/clang++"
 "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC --target=x86_64
 -apple-darwin -stdlib=libc++" "-DCMAKE_CXX_FLAGS=-ffunction-sections
 -fdata-sections -fPIC --target=x86_64-apple-darwin -stdlib=libc++"
 "-DCMAKE_INSTALL_PREFIX=/Users/travis/build/rust-lang/rust/build/x86_64
 -apple-darwin/llvm" "-DCMAKE_BUILD_TYPE=Release"
 > }}}
 >
 > I wonder if perhaps the way we compile LLVM is affecting this? Maybe
 some flag or maybe our own compiler we use on automation is introducing
 bugs? Or maybe it has to do with the C++ standard library and which is
 used?

 Well, I dont't know whether the problem we have is happening when using
 the "official" binaries. This is happening when cross-compiling rust
 ourselves for macOS. That said, I think I can exclude 1) from comment:33.
 I compiled LLVM with exactly the same arguments as used during the rust
 build (using the same runc container, compiler etc.) which are:
 {{{
 cmake .. -G "Unix Makefiles" -DLLVM_ENABLE_ASSERTIONS=OFF
 -DLLVM_TARGETS_TO_BUILD="X86;ARM;AArch64;Mips;PowerPC;SystemZ;MSP430;Sparc;NVPTX;Hexagon"
 -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly
 -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF
 -DLLVM_INCLUDE_DOCS=OFF -DLLVM_ENABLE_ZLIB=OFF -DWITH_POLLY=OFF
 -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_ENABLE_LIBEDIT=OFF
 -DLLVM_PARALLEL_COMPILE_JOBS=4 -DLLVM_TARGET_ARCH=x86_64
 -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu
 -DLLVM_LINK_LLVM_DYLIB=ON -DCMAKE_C_COMPILER=cc -DCMAKE_CXX_COMPILER=c++
 -DCMAKE_C_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64"
 -DCMAKE_CXX_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64"
 -DCMAKE_INSTALL_PREFIX=$distdir -DCMAKE_BUILD_TYPE:String=Release
 -DLLVM_INSTALL_UTILS=on $LLVM_HOME
 }}}
 (`-DLLVM_INSTALL_UTILS=on` is only used when compiling LLVM outside of the
 rust compilation but I doubt this would make a difference with respect to
 this bug)

 and used it with `--llvm-root` and the test script is running fine.

 Which leaves 2). I reverted the `dlmalloc` and `libcompiler_builtins
 /compiler-rt` submodule updates but compiling LLVM during the rust build
 still exhibits the failing test script. Thus, I think those updates does
 not cause this bug. I wonder what else is different between the in-tree
 LLVM compilation and the one outside of it (I am just tarring up src/llvm,
 excluding `.git`, using that tarball for compilation)...

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/26475#comment:35>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online