GitLab

at 2023-07-13T11:20:54+02:00

Bug 40049: processes/ReleaseProcess is replaced by gitlab issue templates

Merge branch 'bug_40049' into 'main'

Bug 40049: processes/ReleaseProcess is replaced by gitlab issue templates

Closes #40049

See merge request tpo/applications/tor-browser-spec!45
 #          Magical Tor Browser Release Process Incantations

 #

-#  "May this part of our job one day be replaced by a small shell script"

+#  The Tor Browser Release Process is now documented as gitlab issue templates:

+#  - https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/main/.gitlab/issue_templates/Release%20Prep%20-%20Tor%20Browser%20Stable.md

+#  - https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/main/.gitlab/issue_templates/Release%20Prep%20-%20Tor%20Browser%20Alpha.md

 #

-

-#####################

-### Prepare build ###

-#####################

-

-#. Tag any relevant component versions.

-#  In particular: before tagging the tor-browser tag, the final code from

-#  Torbutton needs to get included and in order to avoid unnecessary commit

-#  inflation the commit for bumping the git submodule should be a fixup commit

-#  to the one with "Bug 25013: Add torbutton as a tor-browser submodule" in

-#  its commit message.

-

-#. Update changelog and relevant config files in tor-browser-build.

-   cd tor-browser-build

-   vim projects/firefox/config

-   vim ChangeLog.txt

-   vim rbm.conf

-

-#. Tag a build tag in tor-browser-build.

-   make signtag-release # or `make signtag-alpha` for an alpha build

-

-#. Push tag and version to tor-browser-build.git. In case of doing a stable

-#  release with a maintenance branch use that one instead of `master`.

-   torsocks git push origin master:master

-   torsocks git push origin --tags

-

-

-#####################

-### Build         ###

-#####################

-

-#. Build and generate incremental MAR files.

-   git tag -v tbb-$TORBROWSER_VERSION-buildN

-   git checkout tbb-$TORBROWSER_VERSION-buildN

-   make && make incrementals-release # `make alpha && make incrementals-alpha`

-

-#. Compare the SHA256 sums of the bundles and MAR files with an independent

-#  builder.

-   sha256sum tor-browser-build/release/unsigned/$TORBROWSER_BUILDDIR/sha256sums-unsigned-build.txt

-   sha256sum tor-browser-build/release/unsigned/$TORBROWSER_BUILDDIR/sha256sums-unsigned-build.incrementals.txt

-

-#. If the sums match (download and) upload the bundles to your build dir on

-#  people.torproject.org. Fix permissions.

-   chmod 755 $TORBROWSER_BUILDDIR

-   chmod 644 $TORBROWSER_BUILDDIR/*

-   chmod 644 $TORBROWSER_BUILDDIR/.htaccess

-   torsocks ssh people.torproject.org "mkdir ~/public_html/builds/${TORBROWSER_BUILDDIR}"

-   torsocks rsync -avP $TORBROWSER_BUILDDIR/ people.torproject.org:public_html/builds/$TORBROWSER_BUILDDIR

-

-#. (Optional): Upload your binaries to people using partial rsync over old version

-   torsocks ssh people.torproject.org "mv ~/public_html/builds/${TORBROWSER_VERSION}-build1 ~/public_html/builds/$TORBROWSER_BUILDDIR"

-   torsocks rsync -avP $TORBROWSER_BUILDDIR/ people.torproject.org:public_html/builds/$TORBROWSER_BUILDDIR

-

-#. Distribute build to tor-qa@lists.torproject.org

-   # XXX: Currently manual email with link to candidate build, important changes,

-   # and changelog.

-   # For stable releases put tails-dev@boum.org into Cc.

-

-

-#######################

-### Android signing ###

-#######################

-

-#. The Android signing is done separately from the desktop signing. To

-#  do the Android signing you'll need to boot from a Tails usb stick,

-#  with persistent storage enabled.

-

-#. Install the openjdk-11-jdk-headless package

-   apt install openjdk-11-jdk-headless

-

-#. Make sure you have the android keys, for example in the

-#  ~/Persistent/android-keys directory. This directory should include

-#  the tba_alpha.p12 and tba_release.p12 files.

-

-#. Clone tor-browser-build, and checkout the tag for the release

-   git tag -v tbb-$TORBROWSER_VERSION-buildN

-   git checkout tbb-$TORBROWSER_VERSION-buildN

-

-#. Update set-config.tbb-version and set-config.android-signing

-   cd tools/signing

-   vim set-config.tbb-version

-   vim set-config.android-signing

-

-#. Run the android-signing script. This script will download the

-#  unsigned apks from the `pkgstage` machine (where `do-all-signing`

-#  is run), signed them, and upload back the signed apks to the

-#  `pkgstage` machine. It should be done before the hash_signed_bundles.sh

-#  step.

-   ./android-signing

-

-

-##########################

-### Signing and upload ###

-##########################

-

-#. You need a `pkgstage` machine to store the bundles during the signing

-#  process. This machine should be:

-#   - secure (you will ssh to the signing machines and staticiforme

-#     from there)

-#   - with good bandwidth (you will download/upload bundles from there)

-#

-#  All steps from the `Signing and upload` are run from the `pkgstage`

-#  machine unless mentioned otherwise.

-#

-#  The signing scripts are located in the tor-browser-build/tools/signing

-#  directory.

-#

-#  Alternatively, you can run the script `./do-all-signing`, which will

-#  do all the signing steps automatically, until the static-update-component

-#  commands on staticiforme. Also the script does not currently handle

-#  removing the old versions on staticiforme, so you have to do that

-#  manually. The script is also not running `./macos-signer-proxy` which

-#  needs to be started and stopped manually.

-

-#. Checkout the tor-browser-build.git commit corresponding to the

-#  tor browser version you need to sign/publish.

-   cd tor-browser-build

-   git tag -v tbb-$TORBROWSER_VERSION-buildN

-   git checkout tbb-$TORBROWSER_VERSION-buildN

-

-#. Change to the tor-browser-build/tools/signing directory. All following

-#  commands are run from this directory.

-   cd tor-browser-build/tools/signing

-

-#. Set hosts information.

-   vim set-config.hosts

-

-#. Set tor browser version information. You should set tbb_version,

-#  tbb_version_build, tbb_version_type. This should match the information

-#  in rbm.conf.

-   vim set-config.tbb-version

-

-#. Download the tor browser build to the `pkgstage` machine from your

-#  build machine (configured in set-config.hosts).

-   ./sync-builder-unsigned-to-local-signed

-

-#. Codesign the macOS dmg files.

-#

-#  Updload the tor browser dmg files to `macos-signer`

-   ./sync-macos-local-to-macos-signer

-

-#  Upload updated signing scripts to `macos-signer`. The scripts are

-#  located in the `signing-release` or `signing-alpha` directory on

-#  `macos-signer`.

-   ./sync-scripts-to-macos-signer

-

-#  Run proxy for `macos-signer`. You may need to kill an old proxy

-#  process on `macos-signer` if it was still running.

-   ./macos-signer-proxy

-

-#  Enable networking on the `macos-signer`.

-   macos-signer$ networksetup -setsecurewebproxystate Ethernet on

-

-#  Sign the bundles. On `macos-signer`.

-#  (replace signing-release with signing-alpha for an alpha release)

-   macos-signer$ ~/signing-release/macos-signer-gatekeeper-signing

-

-#  Notarize the bundles. On `macos-signer`.

-#  (replace signing-release with signing-alpha for an alpha release)

-   macos-signer$ ~/signing-release/macos-signer-notarization

-

-#  Staple the bundles. On `macos-signer`.

-#  (replace signing-release with signing-alpha for an alpha release)

-   macos-signer$ ~/signing-release/macos-signer-stapler

-

-#  Disable networking on the `macos-signer`.

-   macos-signer$ networksetup -setsocksfirewallproxystate Ethernet off

-

-#  Download the stapled bundles to `pkgstage`.

-   ./sync-macos-signer-stapled-to-macos-local-stapled

-

-#. Regenerate macOS DMG files from stapled zip files.

-   ./gatekeeper-bundling.sh

-

-#. Regenerate macOS MAR files from code signed dmg files.

-   ./dmg2mar

-

-#. Sync bundles to `linux-signer`.

-   ./sync-local-to-linux-signer

-

-#. Upload updated signing scripts to `linux-signer`. The scripts are

-#  located in the `signing-release` or `signing-alpha` directory on

-#  `linux-signer`.

-   ./sync-scripts-to-linux-signer

-

-#. Sign the MAR files. On `linux-signer`.

-#  (replace signing-release with signing-alpha for an alpha release)

-   linux-signer$ ~/signing-release/linux-signer-signmars

-

-#. Authenticode signing of exe files. On `linux-signer`.

-   linux-signer$ ~/signing-release/linux-signer-authenticode-signing

-

-#. Authenticode timestamping.

-   ./sync-linux-signer-to-local

-   ./authenticode-timestamping.sh

-

-#. Create sha256sums-signed-build files

-   ./hash_signed_bundles.sh

-

-#. Upload sha256sums-signed-build and updated exe files to `linux-signer`.

-   ./sync-local-to-linux-signer

-

-#. Gpg signing.

-   ./linux-signer-gpg-sign

-

-#. Fetch signatures on unsigned sha256sums from other builds

-   ./download-unsigned-sha256sums-gpg-signatures-from-people-tpo

-

-#. Remove old builds on `staticiforme`

-   # IMPORTANT: Remove the oldest version in a series in case there is more

-   # than 1 available on dist.torproject.org before proceeding

-   # XXX: TORBROWSER_VERSION_OLDEST needs to be set

-   staticiforme$ rm -rf /srv/dist-master.torproject.org/htdocs/torbrowser/$TORBROWSER_VERSION_OLDEST

-   staticiforme$ static-update-component dist.torproject.org

-

-#. Upload new build to `staticiforme`

-   ./sync-local-to-staticiforme

-

-#. Check diskspace available on cdn.tpo

-#  We currently have enough disk space to host two alpha and stable

-#  releases. However with the size of each Tor Browser release increasing,

-#  it may become necessary to increase disk space. The server hosting

-#  the files for cdn.tpo is savii.tpo and its disk usage can be monitored

-#  by going to https://grafana.torproject.org/d/Z7T7Cfemz/node-exporter-full

-#  and selecting the hosts: web-fsn-02.torproject.org, web-cymru-01.torproject.org,

-#  web-fsn-01.torproject.org, and cdn-backend-sunet-01.torproject.org

-

-#. Remove the oldest *.mar files from cdn.tpo to save space

-   staticiforme$ rm -rf /srv/cdn-master.torproject.org/htdocs/aus1/torbrowser/$TORBROWSER_VERSION_OLDEST

-   staticiforme$ static-update-component cdn.torproject.org

-

-#. Sync files to dist.tpo and cdn.tpo mirrored web servers

-#  (replace signing-release with signing-alpha for an alpha release)

-#  It is a good idea to run the static-update-component inside screen

-#  or tmux so that the upload continues if your connection to

-#  staticiforme is closed.

-   $ ./sync-scripts-to-staticiforme

-   staticiforme$ ~/signing-release/staticiforme-prepare-cdn-dist-upload

-   staticiforme$ static-update-component cdn.torproject.org && \

-                 static-update-component dist.torproject.org

-

-#. Make sure we really built from the proper Mozilla build tag by consulting

-   # the respective ESR release branch (for a good overview for ESR78 see

-   # https://hg.mozilla.org/releases/mozilla-esr78/graph/). For the platforms following

-   # rapid release (only Android, currently), consult the beta repo

-   # (https://hg.mozilla.org/releases/mozilla-beta/graph/) or the release repo

-   # (https://hg.mozilla.org/releases/mozilla-release/graph/)

-

-#. Update website's torbrowser versions file in the website git

-   cd tpo

-   torsocks git pull origin

-   # Update `win32` in the `torbrowser-stable` section as well if we

-   # include a new stable tor version (called the Windows Expert Bundle

-   # on the website). See: #14152.

-   vim databags/versions.ini

-   git commit databags/versions.ini -m "Add new Tor Browser version"

-   torsocks git push origin master:master

-   cd ..

-   # Check build success/failure:

-   # https://jenkins.torproject.org/job/lektor-website-tpo-translation/

-   # https://jenkins.torproject.org/job/lektor-website-tpo-translation-install/

-

-#. Add new locales to the download page

-   # If this release is introducing new locales, add them to the

-   # databags/download-alternatives.ini file (for a stable release) or

-   # the databags/download-alternatives-alpha.ini file (for an alpha

-   # release).

-   cd tpo

-   torsocks git pull origin

-   vim databags/download-alternatives.ini # or databags/download-alternatives-alpha.ini

-   git commit databags/download-alternatives.ini -m "Add new Tor Browser locales"

-   torsocks git push origin master:master

-   cd ..

-

-#. Create blog post from changelog

-#  Edit set-config.blog to set you local blog directory.

-#  Don't forget to link to Mozilla's security advisories if this is a security

-#  update.

-  ./create-blog-post

-

-#. Check whether the .exe files got properly signed and timestamped

-   # Point OSSLSIGNCODE to your osslsigncode binary

-   pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION

-   OSSLSIGNCODE=/path/to/osslsigncode

-   ../../../tools/authenticode_check.sh

-   popd

-

-#. Check whether the MAR files got properly signed

-   # Point NSSDB to your nssdb containing the mar signing certificate

-   # Point SIGNMAR to your signmar binary

-   # Point LD_LIBRARY_PATH to your mar-tools directory

-   pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION

-   NSSDB=/path/to/nssdb

-   SIGNMAR=/path/to/mar-tools/signmar

-   LD_LIBRARY_PATH=/path/to/mar-tools/

-   ../../../tools/marsigning_check.sh

-   popd

-

-#. Generate and upload update_responses to staticiforme

-   ./upload-update_responses-to-staticiforme

-

-#. Enable update. On `staticiforme`.

-#  (for alpha release, use deploy_update_responses-alpha.sh instead)

-   staticiforme$ ~/deploy_update_responses-release.sh

-

-#. Clean linux-signer and macos-signer

-   ./finished-signing-clean-linux-signer

-   ./finished-signing-clean-macos-signer

-

-#  Upload APKs to Google Play

-   Log into https://play.google.com/apps/publish

-   Select correct app (Tor Browser or Tor Browser Alpha)

-   Under left-side navigation bar, select "Production" under "Release"

-   Select "Create new release" button at upper right-side

-   Under "App bundles and APKs" section, "Upload" each signed APK

-   After uploading APKs:

-     - The "Release Name" should be automatically filled with the Firefox version

-     - Next to "Release notes" click "Copy from a previous release"

-     - Select the previous release and adjust the blog post url for this release

-     - Save, review, and configure rollout percentage at the bottom

-       - Use 25% rollout when publishing a scheduled update (without a security driver)

-       - Use 100% rollout when publishing an unscheduled update or security-driven release

-   Roll out.

-

-   Note, you may receive three warning messages about:

-     1) app is larger than it is necessary (android app bundle),

-     2) missing deobfuscation file for better crash analysis

-     3) missing debug symbol file

-

-   These warnings are expected and should be accepted.

-

-   See below for updating the rollout percentage.

-

-# Release on F-Droid

-  Publication on F-Droid via the Guardian Project's repository should be

-  automatic. Hans is responsible for maintaining this automation.

-

-# Update rollout percentage

-   After 24 hours, check the Release Daskboard:

-     - "Release"->"Production" and select the "Release Dashboard" tab

-   If the Release Dashboard does not show any concerning trends (significant

-   increase in crashes or ANRs (Application Not Responding)), then continue on

-   to the next paragraph and increase rollout from 25% to 100%. Otherwise

-   consider halting rollout as described in the `RollingBackUpdate` process.

-

-   Select the "Releases" tab on the "Production" page. The current released

-   version should indicate 25% rollout: "Available to 25% of users on Google Play"

-   On the right-side of the "View release details" button of the release there

-   should be a button labeled "Manage rollout" with a down-arrow. Clicking on

-   that button should show two options:

-     - Update rollout

-     - Halt rollout

-

-   Select "Update rollout" and increase to 100% and click "Update". The change

-   should be immediately implemented and the "Manage rollout" button disappears.

-

-#. Write an email to tor-announce in case this release is concerned with getting

-#  a stable version out. Using the contents of the blog entry should do it.
+#  See also the open release prep issues:

+#  https://gitlab.torproject.org/groups/tpo/applications/-/issues/?sort=updated_desc&state=opened&label_name%5B%5D=Release%20Prep&first_page_size=20

richard pushed to branch main at The Tor Project / Applications / tor-browser-spec

Commits:

1 changed file:

Changes: