richard pushed to branch main at The Tor Project / Applications / tor-browser-spec Commits: 2b13c342 by Nicolas Vigier at 2023-07-13T11:20:54+02:00 Bug 40049: processes/ReleaseProcess is replaced by gitlab issue templates - - - - - b79d4e8a by richard at 2023-07-13T20:07:11+00:00 Merge branch 'bug_40049' into 'main' Bug 40049: processes/ReleaseProcess is replaced by gitlab issue templates Closes #40049 See merge request tpo/applications/tor-browser-spec!45 - - - - - 1 changed file: - processes/ReleaseProcess Changes: ===================================== processes/ReleaseProcess ===================================== @@ -1,362 +1,8 @@ # Magical Tor Browser Release Process Incantations # -# "May this part of our job one day be replaced by a small shell script" +# The Tor Browser Release Process is now documented as gitlab issue templates: +# - https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/main... +# - https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/main... # - -##################### -### Prepare build ### -##################### - -#. Tag any relevant component versions. -# In particular: before tagging the tor-browser tag, the final code from -# Torbutton needs to get included and in order to avoid unnecessary commit -# inflation the commit for bumping the git submodule should be a fixup commit -# to the one with "Bug 25013: Add torbutton as a tor-browser submodule" in -# its commit message. - -#. Update changelog and relevant config files in tor-browser-build. - cd tor-browser-build - vim projects/firefox/config - vim ChangeLog.txt - vim rbm.conf - -#. Tag a build tag in tor-browser-build. - make signtag-release # or `make signtag-alpha` for an alpha build - -#. Push tag and version to tor-browser-build.git. In case of doing a stable -# release with a maintenance branch use that one instead of `master`. - torsocks git push origin master:master - torsocks git push origin --tags - - -##################### -### Build ### -##################### - -#. Build and generate incremental MAR files. - git tag -v tbb-$TORBROWSER_VERSION-buildN - git checkout tbb-$TORBROWSER_VERSION-buildN - make && make incrementals-release # `make alpha && make incrementals-alpha` - -#. Compare the SHA256 sums of the bundles and MAR files with an independent -# builder. - sha256sum tor-browser-build/release/unsigned/$TORBROWSER_BUILDDIR/sha256sums-unsigned-build.txt - sha256sum tor-browser-build/release/unsigned/$TORBROWSER_BUILDDIR/sha256sums-unsigned-build.incrementals.txt - -#. If the sums match (download and) upload the bundles to your build dir on -# people.torproject.org. Fix permissions. - chmod 755 $TORBROWSER_BUILDDIR - chmod 644 $TORBROWSER_BUILDDIR/* - chmod 644 $TORBROWSER_BUILDDIR/.htaccess - torsocks ssh people.torproject.org "mkdir ~/public_html/builds/${TORBROWSER_BUILDDIR}" - torsocks rsync -avP $TORBROWSER_BUILDDIR/ people.torproject.org:public_html/builds/$TORBROWSER_BUILDDIR - -#. (Optional): Upload your binaries to people using partial rsync over old version - torsocks ssh people.torproject.org "mv ~/public_html/builds/${TORBROWSER_VERSION}-build1 ~/public_html/builds/$TORBROWSER_BUILDDIR" - torsocks rsync -avP $TORBROWSER_BUILDDIR/ people.torproject.org:public_html/builds/$TORBROWSER_BUILDDIR - -#. Distribute build to tor-qa@lists.torproject.org - # XXX: Currently manual email with link to candidate build, important changes, - # and changelog. - # For stable releases put tails-dev@boum.org into Cc. - - -####################### -### Android signing ### -####################### - -#. The Android signing is done separately from the desktop signing. To -# do the Android signing you'll need to boot from a Tails usb stick, -# with persistent storage enabled. - -#. Install the openjdk-11-jdk-headless package - apt install openjdk-11-jdk-headless - -#. Make sure you have the android keys, for example in the -# ~/Persistent/android-keys directory. This directory should include -# the tba_alpha.p12 and tba_release.p12 files. - -#. Clone tor-browser-build, and checkout the tag for the release - git tag -v tbb-$TORBROWSER_VERSION-buildN - git checkout tbb-$TORBROWSER_VERSION-buildN - -#. Update set-config.tbb-version and set-config.android-signing - cd tools/signing - vim set-config.tbb-version - vim set-config.android-signing - -#. Run the android-signing script. This script will download the -# unsigned apks from the `pkgstage` machine (where `do-all-signing` -# is run), signed them, and upload back the signed apks to the -# `pkgstage` machine. It should be done before the hash_signed_bundles.sh -# step. - ./android-signing - - -########################## -### Signing and upload ### -########################## - -#. You need a `pkgstage` machine to store the bundles during the signing -# process. This machine should be: -# - secure (you will ssh to the signing machines and staticiforme -# from there) -# - with good bandwidth (you will download/upload bundles from there) -# -# All steps from the `Signing and upload` are run from the `pkgstage` -# machine unless mentioned otherwise. -# -# The signing scripts are located in the tor-browser-build/tools/signing -# directory. -# -# Alternatively, you can run the script `./do-all-signing`, which will -# do all the signing steps automatically, until the static-update-component -# commands on staticiforme. Also the script does not currently handle -# removing the old versions on staticiforme, so you have to do that -# manually. The script is also not running `./macos-signer-proxy` which -# needs to be started and stopped manually. - -#. Checkout the tor-browser-build.git commit corresponding to the -# tor browser version you need to sign/publish. - cd tor-browser-build - git tag -v tbb-$TORBROWSER_VERSION-buildN - git checkout tbb-$TORBROWSER_VERSION-buildN - -#. Change to the tor-browser-build/tools/signing directory. All following -# commands are run from this directory. - cd tor-browser-build/tools/signing - -#. Set hosts information. - vim set-config.hosts - -#. Set tor browser version information. You should set tbb_version, -# tbb_version_build, tbb_version_type. This should match the information -# in rbm.conf. - vim set-config.tbb-version - -#. Download the tor browser build to the `pkgstage` machine from your -# build machine (configured in set-config.hosts). - ./sync-builder-unsigned-to-local-signed - -#. Codesign the macOS dmg files. -# -# Updload the tor browser dmg files to `macos-signer` - ./sync-macos-local-to-macos-signer - -# Upload updated signing scripts to `macos-signer`. The scripts are -# located in the `signing-release` or `signing-alpha` directory on -# `macos-signer`. - ./sync-scripts-to-macos-signer - -# Run proxy for `macos-signer`. You may need to kill an old proxy -# process on `macos-signer` if it was still running. - ./macos-signer-proxy - -# Enable networking on the `macos-signer`. - macos-signer$ networksetup -setsecurewebproxystate Ethernet on - -# Sign the bundles. On `macos-signer`. -# (replace signing-release with signing-alpha for an alpha release) - macos-signer$ ~/signing-release/macos-signer-gatekeeper-signing - -# Notarize the bundles. On `macos-signer`. -# (replace signing-release with signing-alpha for an alpha release) - macos-signer$ ~/signing-release/macos-signer-notarization - -# Staple the bundles. On `macos-signer`. -# (replace signing-release with signing-alpha for an alpha release) - macos-signer$ ~/signing-release/macos-signer-stapler - -# Disable networking on the `macos-signer`. - macos-signer$ networksetup -setsocksfirewallproxystate Ethernet off - -# Download the stapled bundles to `pkgstage`. - ./sync-macos-signer-stapled-to-macos-local-stapled - -#. Regenerate macOS DMG files from stapled zip files. - ./gatekeeper-bundling.sh - -#. Regenerate macOS MAR files from code signed dmg files. - ./dmg2mar - -#. Sync bundles to `linux-signer`. - ./sync-local-to-linux-signer - -#. Upload updated signing scripts to `linux-signer`. The scripts are -# located in the `signing-release` or `signing-alpha` directory on -# `linux-signer`. - ./sync-scripts-to-linux-signer - -#. Sign the MAR files. On `linux-signer`. -# (replace signing-release with signing-alpha for an alpha release) - linux-signer$ ~/signing-release/linux-signer-signmars - -#. Authenticode signing of exe files. On `linux-signer`. - linux-signer$ ~/signing-release/linux-signer-authenticode-signing - -#. Authenticode timestamping. - ./sync-linux-signer-to-local - ./authenticode-timestamping.sh - -#. Create sha256sums-signed-build files - ./hash_signed_bundles.sh - -#. Upload sha256sums-signed-build and updated exe files to `linux-signer`. - ./sync-local-to-linux-signer - -#. Gpg signing. - ./linux-signer-gpg-sign - -#. Fetch signatures on unsigned sha256sums from other builds - ./download-unsigned-sha256sums-gpg-signatures-from-people-tpo - -#. Remove old builds on `staticiforme` - # IMPORTANT: Remove the oldest version in a series in case there is more - # than 1 available on dist.torproject.org before proceeding - # XXX: TORBROWSER_VERSION_OLDEST needs to be set - staticiforme$ rm -rf /srv/dist-master.torproject.org/htdocs/torbrowser/$TORBROWSER_VERSION_OLDEST - staticiforme$ static-update-component dist.torproject.org - -#. Upload new build to `staticiforme` - ./sync-local-to-staticiforme - -#. Check diskspace available on cdn.tpo -# We currently have enough disk space to host two alpha and stable -# releases. However with the size of each Tor Browser release increasing, -# it may become necessary to increase disk space. The server hosting -# the files for cdn.tpo is savii.tpo and its disk usage can be monitored -# by going to https://grafana.torproject.org/d/Z7T7Cfemz/node-exporter-full -# and selecting the hosts: web-fsn-02.torproject.org, web-cymru-01.torproject.org, -# web-fsn-01.torproject.org, and cdn-backend-sunet-01.torproject.org - -#. Remove the oldest *.mar files from cdn.tpo to save space - staticiforme$ rm -rf /srv/cdn-master.torproject.org/htdocs/aus1/torbrowser/$TORBROWSER_VERSION_OLDEST - staticiforme$ static-update-component cdn.torproject.org - -#. Sync files to dist.tpo and cdn.tpo mirrored web servers -# (replace signing-release with signing-alpha for an alpha release) -# It is a good idea to run the static-update-component inside screen -# or tmux so that the upload continues if your connection to -# staticiforme is closed. - $ ./sync-scripts-to-staticiforme - staticiforme$ ~/signing-release/staticiforme-prepare-cdn-dist-upload - staticiforme$ static-update-component cdn.torproject.org && \ - static-update-component dist.torproject.org - -#. Make sure we really built from the proper Mozilla build tag by consulting - # the respective ESR release branch (for a good overview for ESR78 see - # https://hg.mozilla.org/releases/mozilla-esr78/graph/). For the platforms following - # rapid release (only Android, currently), consult the beta repo - # (https://hg.mozilla.org/releases/mozilla-beta/graph/) or the release repo - # (https://hg.mozilla.org/releases/mozilla-release/graph/) - -#. Update website's torbrowser versions file in the website git - cd tpo - torsocks git pull origin - # Update `win32` in the `torbrowser-stable` section as well if we - # include a new stable tor version (called the Windows Expert Bundle - # on the website). See: #14152. - vim databags/versions.ini - git commit databags/versions.ini -m "Add new Tor Browser version" - torsocks git push origin master:master - cd .. - # Check build success/failure: - # https://jenkins.torproject.org/job/lektor-website-tpo-translation/ - # https://jenkins.torproject.org/job/lektor-website-tpo-translation-install/ - -#. Add new locales to the download page - # If this release is introducing new locales, add them to the - # databags/download-alternatives.ini file (for a stable release) or - # the databags/download-alternatives-alpha.ini file (for an alpha - # release). - cd tpo - torsocks git pull origin - vim databags/download-alternatives.ini # or databags/download-alternatives-alpha.ini - git commit databags/download-alternatives.ini -m "Add new Tor Browser locales" - torsocks git push origin master:master - cd .. - -#. Create blog post from changelog -# Edit set-config.blog to set you local blog directory. -# Don't forget to link to Mozilla's security advisories if this is a security -# update. - ./create-blog-post - -#. Check whether the .exe files got properly signed and timestamped - # Point OSSLSIGNCODE to your osslsigncode binary - pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION - OSSLSIGNCODE=/path/to/osslsigncode - ../../../tools/authenticode_check.sh - popd - -#. Check whether the MAR files got properly signed - # Point NSSDB to your nssdb containing the mar signing certificate - # Point SIGNMAR to your signmar binary - # Point LD_LIBRARY_PATH to your mar-tools directory - pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION - NSSDB=/path/to/nssdb - SIGNMAR=/path/to/mar-tools/signmar - LD_LIBRARY_PATH=/path/to/mar-tools/ - ../../../tools/marsigning_check.sh - popd - -#. Generate and upload update_responses to staticiforme - ./upload-update_responses-to-staticiforme - -#. Enable update. On `staticiforme`. -# (for alpha release, use deploy_update_responses-alpha.sh instead) - staticiforme$ ~/deploy_update_responses-release.sh - -#. Clean linux-signer and macos-signer - ./finished-signing-clean-linux-signer - ./finished-signing-clean-macos-signer - -# Upload APKs to Google Play - Log into https://play.google.com/apps/publish - Select correct app (Tor Browser or Tor Browser Alpha) - Under left-side navigation bar, select "Production" under "Release" - Select "Create new release" button at upper right-side - Under "App bundles and APKs" section, "Upload" each signed APK - After uploading APKs: - - The "Release Name" should be automatically filled with the Firefox version - - Next to "Release notes" click "Copy from a previous release" - - Select the previous release and adjust the blog post url for this release - - Save, review, and configure rollout percentage at the bottom - - Use 25% rollout when publishing a scheduled update (without a security driver) - - Use 100% rollout when publishing an unscheduled update or security-driven release - Roll out. - - Note, you may receive three warning messages about: - 1) app is larger than it is necessary (android app bundle), - 2) missing deobfuscation file for better crash analysis - 3) missing debug symbol file - - These warnings are expected and should be accepted. - - See below for updating the rollout percentage. - -# Release on F-Droid - Publication on F-Droid via the Guardian Project's repository should be - automatic. Hans is responsible for maintaining this automation. - -# Update rollout percentage - After 24 hours, check the Release Daskboard: - - "Release"->"Production" and select the "Release Dashboard" tab - If the Release Dashboard does not show any concerning trends (significant - increase in crashes or ANRs (Application Not Responding)), then continue on - to the next paragraph and increase rollout from 25% to 100%. Otherwise - consider halting rollout as described in the `RollingBackUpdate` process. - - Select the "Releases" tab on the "Production" page. The current released - version should indicate 25% rollout: "Available to 25% of users on Google Play" - On the right-side of the "View release details" button of the release there - should be a button labeled "Manage rollout" with a down-arrow. Clicking on - that button should show two options: - - Update rollout - - Halt rollout - - Select "Update rollout" and increase to 100% and click "Update". The change - should be immediately implemented and the "Manage rollout" button disappears. - -#. Write an email to tor-announce in case this release is concerned with getting -# a stable version out. Using the contents of the blog entry should do it. +# See also the open release prep issues: +# https://gitlab.torproject.org/groups/tpo/applications/-/issues/?sort=updated... View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-spec/-/compare/2b... -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-spec/-/compare/2b... You're receiving this email because of your account on gitlab.torproject.org.