GitLab

at 2025-03-05T14:48:49+01:00

Bug 41121: Use the official Go binaries for bootstrapping.

The Go toolchain is reproducible, and we checked that we obtain the
same exact Go 1.23.6 binaries when building at the previous commit.

Therefore, we can simplify our bootstrap process and just use the
official binaries.

 -Go 1.4 was the last version written in C.
 -All the later versions need a Go compiler, that we provide with this project.
 +This project downloads the official Go binaries to use them for bootstrapping
 +purposes.
 -Also, starting from Go 1.20.x, Go introduced a new policy: Go 1.y.z needs at
 -least Go 1.(y - 3) to build. E.g., Go 1.19 can build Go 1.20, 1.21 and 1.22, but
 -no Go 1.23, which will need Go 1.20 or later.
 +We used to start with Go 1.4 (the last version written in C) and build all the
 +versions needed to then build the most recent Go toolchain.
 -So, right now we build Go 1.4 with Debian's GCC, then we use it to build Go
 -1.19.9, but at a certain point we will have to add another Go compiler.
 +However, starting with Go 1.21,
 +[the official binaries are reproducible](https://go.dev/blog/rebuild).
 -Other alternatives are:
 -- use Debian's Go compiler (but we use a very old version of Debian for wider
 -  binary compatibility, so it's likely not to ship a recent enough compiler for
 -  bootstrapping purposes)
 -- use the [official binaries](https://go.dev/dl/) to bootstrap, like we do for
 -  Rust
 +So, we checked that the Go 1.23.6 binaries we produced with our old procedure
 +at 80f16f97e7c2973e9aa4458606c9afd2c63c2d60 matched the official binaries.
++
 +## How to update
++
 +1. In `projects/go/config`, update version to the version we want to be the new
 +   go-bootstrap version
 +2. Build `go` with
 +   `./rbm/rbm build --target torbrowser-linux-x86_64 --target alpha go` and
 +   compare the result with the official build
 +3. If it is matching or if we can explain the differences, update the
 +   `go-bootstrap` version of the bin that we download.
 +4. Build the same version of go again with the command from above and check the
 +   two archives have the same exact hash.

  #!/bin/bash
  [% c("var/set_default_env") -%]
 -# This project is expected to be built only on Linux.
 -# Then the actual Go compiler will be built for each platform.
 +tar -xf $rootdir/[% c('input_files_by_name/go') %]
 -distdir=/var/tmp/dist/[% project %]
 -mkdir -p /var/tmp/build /var/tmp/dist
 +pushd go
 +# Disable go autoupdates (tor-browser-build#41345)
 +# See https://go.dev/doc/toolchain
 +sed -i -e 's/^GOTOOLCHAIN=.*/GOTOOLCHAIN=local/' go.env
 +popd
 +mv go [% project %]
 -# Building go 1.4.x
 -# This is needed to bootstrap the go that we actually use
 -# https://golang.org/doc/install/source#go14
 -tar -C /var/tmp/build --transform='s,^go\>,go1.4,' -xf $rootdir/[% c('input_files_by_name/go14') %]
 -cd /var/tmp/build/go1.4/src
 -# Disable cgo to avoid conflicts with newer GCC. cgo is not needed for the bootstrap go.
 -# https://github.com/golang/go/issues/13114#issuecomment-186922245
 -# Disable CC etc. that are set up for cross builds.
 -CGO_ENABLED=0 CC= CFLAGS= LDFLAGS= ./make.bash
 -export GOROOT_BOOTSTRAP="/var/tmp/build/go1.4"
+-
 -# Building go 1.19.X
 -cd $rootdir
 -tar -C /var/tmp/build --transform='s,^go\>,go1.19,' -xf $rootdir/[% c('input_files_by_name/go119') %]
 -cd /var/tmp/build/go1.19/src
+-
 -./make.bash
 -export GOROOT_BOOTSTRAP="/var/tmp/build/go1.19"
+-
 -# Building go 1.21.X
 -cd $rootdir
 -tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/go') %]
 -export GOROOT="$distdir"
 -mv /var/tmp/dist/go $distdir
 -cd $distdir/src
+-
 -./make.bash
+-
 -# This directory is non-reproducible and doesn't exist in official Go releases,
 -# so remove it to preserve reproducibility of the output.
 -rm -rf $distdir/pkg/obj/go-build/
+-
 -cd /var/tmp/dist
  [% c('tar', {
          tar_src => [ project ],
          tar_args => '-caf ' _ dest_dir _ '/' _ c('filename'),

  # vim: filetype=yaml sw=2
 -version: 1.21.12
 +# Make sure we can reproduce the official binaries before updating.
 +# Check README.md for more information before updating this version.
 +version: 1.23.6
  filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.[% c("compress_tar") %]'
  container:
 -  use_container: 1
+-
 -var:
 -  no_crosscompile: 1
 +  use_container: 0
  input_files:
 -  - project: container-image
 -  - URL: 'https://go.dev/dl/go[% c("version") %].src.tar.gz'
 +  - URL: 'https://go.dev/dl/go[% c("version") %].linux-amd64.tar.gz'
      name: go
 -    sha256sum: 30e68af27bc1f1df231e3ab74f3d17d3b8d52a089c79bcaab573b4f1b807ed4f
 -  - URL: https://go.dev/dl/go1.19.13.src.tar.gz
 -    sha256sum: ccf36b53fb0024a017353c3ddb22c1f00bc7a8073c6aac79042da24ee34434d3
 -    name: go119
 -  - URL: https://dl.google.com/go/go1.4-bootstrap-20171003.tar.gz
 -    name: go14
 -    sha256sum: f4ff5b5eb3a3cae1c993723f3eab519c5bae18866b5e5f96fe1102f0cb5c3e52
 +    # Make sure we can reproduce the official binaries before updating.
 +    # Check README.md for more information before updating this hash.
 +    sha256sum: 9379441ea310de000f33a4dc767bd966e72ab2826270e038e78b2c53c2e7802d

Pier Angelo Vendrame pushed to branch main at The Tor Project / Applications / tor-browser-build

Commits:

3 changed files:

Changes: