[tbb-commits] [tor-browser-spec] branch master updated: Bug 40027: Update processes/ReleaseProcess following tor-browser-build#40414
gitolite role
git at cupani.torproject.org
Wed Apr 20 09:41:47 UTC 2022
This is an automated email from the git hooks/post-receive script.
boklm pushed a commit to branch master
in repository tor-browser-spec.
The following commit(s) were added to refs/heads/master by this push:
new 0381b27 Bug 40027: Update processes/ReleaseProcess following tor-browser-build#40414
0381b27 is described below
commit 0381b271b4f4d57a37024a2aeaf21681c59b59c5
Author: Nicolas Vigier <boklm at torproject.org>
AuthorDate: Thu Mar 24 12:47:39 2022 +0100
Bug 40027: Update processes/ReleaseProcess following tor-browser-build#40414
---
processes/ReleaseProcess | 242 +++++++++++++++++++++++++++--------------------
1 file changed, 140 insertions(+), 102 deletions(-)
diff --git a/processes/ReleaseProcess b/processes/ReleaseProcess
index 08db508..21a3b9b 100644
--- a/processes/ReleaseProcess
+++ b/processes/ReleaseProcess
@@ -3,6 +3,10 @@
# "May this part of our job one day be replaced by a small shell script"
#
+#####################
+### Prepare build ###
+#####################
+
#. Tag any relevant component versions.
# In particular: before tagging the tor-browser tag, the final code from
# Torbutton needs to get included and in order to avoid unnecessary commit
@@ -24,7 +28,14 @@
torsocks git push origin master:master
torsocks git push origin --tags
+
+#####################
+### Build ###
+#####################
+
#. Build and generate incremental MAR files.
+ git tag -v tbb-$TORBROWSER_VERSION-buildN
+ git checkout tbb-$TORBROWSER_VERSION-buildN
make && make incrementals-release # `make alpha && make incrementals-alpha`
#. Compare the SHA256 sums of the bundles and MAR files with an independent
@@ -49,96 +60,124 @@
# and changelog.
# For stable releases put tails-dev at boum.org into Cc.
+
+##########################
+### Signing and upload ###
+##########################
+
+#. You need a `pkgstage` machine to store the bundles during the signing
+# process. This machine should be:
+# - secure (you will ssh to the signing machines and staticiforme
+# from there)
+# - with good bandwidth (you will download/upload bundles from there)
+#
+# All steps from the `Signing and upload` are run from the `pkgstage`
+# machine unless mentioned otherwise.
+#
+# The signing scripts are located in the tor-browser-build/tools/signing
+# directory.
+
+#. Checkout the tor-browser-build.git commit corresponding to the
+# tor browser version you need to sign/publish.
+ cd tor-browser-build
+ git tag -v tbb-$TORBROWSER_VERSION-buildN
+ git checkout tbb-$TORBROWSER_VERSION-buildN
+
+#. Change to the tor-browser-build/tools/signing directory. All following
+# commands are run from this directory.
+ cd tor-browser-build/tools/signing
+
+#. Set hosts information.
+ vim set-config.hosts
+
+#. Set tor browser version information. You should set tbb_version,
+# tbb_version_build, tbb_version_type. This should match the information
+# in rbm.conf.
+ vim set-config.tbb-version
+
+#. Download the tor browser build to the `pkgstage` machine from your
+# build machine (configured in set-config.hosts).
+ ./sync-builder-unsigned-to-local-signed
+
#. Codesign the macOS dmg files.
- # setup
- torsocks ssh mac-signer "mkdir $TORBROWSER_VERSION"
- torsocks rsync -avP $TORBROWSER_BUILDDIR/*.dmg mac-signer:$TORBROWSER_VERSION/
- torsocks ssh mac-signer
- # Unlock the keychain and then...
- cd $TORBROWSER_VERSION
- # Enable networking
- networksetup -setsecurewebproxystate Ethernet on
- # Sign the bundles.
- ../gatekeeper-signing.sh $TORBROWSER_VERSION
- # notarize and staple
- ../notarization.sh $TORBROWSER_VERSION
- ../stapler.sh $TORBROWSER_VERSION
- # Check that it worked.
- unzip -d test tb-$TORBROWSER_VERSION-osx_zh-CN-stapled.zip
- pushd test
- # Both should be "Tor Browser.app: Accepted" with "source=Notarized Developer ID"
- spctl -vvvv --assess --type=exec --context context:primary-signature Tor\ Browser.app/
- spctl -vvvv --assess --type=open --context context:primary-signature Tor\ Browser.app/
- popd
- rm -rf test
- # Disable networking
- networksetup -setsocksfirewallproxystate Ethernet off
- exit
- torsocks rsync -avP mac-signer:$TORBROWSER_VERSION/tb-*-stapled.zip /path/to/builddir/$TORBROWSER_VERSION/
+#
+# Updload the tor browser dmg files to `macos-signer`
+ ./sync-macos-local-to-macos-signer
+
+# Upload updated signing scripts to `macos-signer`. The scripts are
+# located in the `signing-release` or `signing-alpha` directory on
+# `macos-signer`.
+ ./sync-scripts-to-macos-signer
+
+# Run proxy for `macos-signer`. You may need to kill an old proxy
+# process on `macos-signer` if it was still running.
+ ./macos-signer-proxy
- cp -rT tor-browser-build/projects/tor-browser/Bundle-Data/mac-applications.dmg dmg
+# Sign the bundles. On `macos-signer`.
+# (replace signing-release with signing-alpha for an alpha release)
+ macos-signer$ ~/signing-release/macos-signer-gatekeeper-signing
+
+# Notarize the bundles. On `macos-signer`.
+# (replace signing-release with signing-alpha for an alpha release)
+ macos-signer$ ~/signing-release/macos-signer-notarization
+
+# Staple the bundles. On `macos-signer`.
+# (replace signing-release with signing-alpha for an alpha release)
+ macos-signer$ ~/signing-release/macos-signer-stapler
+
+# Download the stapled bundles to `pkgstage`.
+ ./sync-macos-signer-stapled-to-macos-local-stapled
+
+#. Regenerate macOS DMG files from stapled zip files.
+ ./gatekeeper-bundling.sh
#. Regenerate macOS MAR files from code signed dmg files.
- # XXX Go to your directory prepared for recreating the .dmg files and containing
- # the uploaded .zip files.
- ./gatekeeper-bundling.sh $TORBROWSER_VERSION
- rsync -avP ../$TORBROWSER_VERSION-signed/*.dmg $TORBROWSER_BUILDDIR/
- cd tor-browser-build
- mv $TORBROWSER_BUILDDIR/ release/signed/ (or alpha)
- # The code signed dmg files should be in the $TORBROWSER_VERSION directory
- # Install a recent p7zip version (see ../tools/dmg2mar for instructions)
- make dmg2mar-release # or `make dmg2mar-alpha`
-
-#. Sign the MAR files
- # First, copy the tor-browser-bundle tree to the signing machine. XXX: This
- # still uses part of the old Gitian related infrastructure.
- torsocks rsync -avP $TORBROWSER_BUILDDIR/../../../ signing-machine
- torsocks ssh signing-machine "mkdir tor-browser-bundle/gitian/$TORBROWSER_VERSION"
- torsocks rsync -avP $TORBROWSER_BUILDDIR/*.mar signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/
- torsocks ssh signing-machine
- cd tor-browser-bundle/gitian
- # XXX Modify the signmars.sh script to comment out the eval call.
- export TORBROWSER_VERSION=$TORBROWSER_VERSION
- export NSS_DB_DIR=/path/to/nssdb
- # Only needed if you are not owner of the marsigner cert
- export NSS_CERTNAME=your_certname
- make signmars
- exit
- torsocks rsync -avP signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/*.mar $TORBROWSER_BUILDDIR/
-
-#. Sign individual bundle files.
- # Authenticode signing first
- torsocks ssh windows-signing-machine "mkdir tor-browser-bundle/gitian/$TORBROWSER_VERSION"
- torsocks rsync -avP $TORBROWSER_BUILDDIR/*.exe windows-signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/
- torsocks ssh windows-signing-machine
- cd tor-browser-bundle/gitian/$TORBROWSER_VERSION
- /path/to/authenticode-signing.sh
- exit
- torsocks rsync -avP window-signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/*.exe $TORBROWSER_BUILDDIR/
- # Authenticode timestamping next
- cd $TORBROWSER_BUILDDIR
- export OSSLSIGNCODE=/path/to/osslsigncode
- /path/to/authenticode-timestamping.sh
- # Hashes of the signed bundles
- ../../../tools/hash_signed_bundles.sh
- # All the GPG signatures at last
- torsocks rsync -avP $TORBROWSER_BUILDDIR/* signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/
- cd tor-browser-bundle/gitian/$TORBROWSER_VERSION
- /path/to/tbb-signing.sh
- exit
- torsocks rsync -avP signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/ $TORBROWSER_BUILDDIR
- # Fetch signatures on unsigned sha256sums from other builds
-
-#. Sync to people.torproject.org
- torsocks rsync -avP $TORBROWSER_BUILDDIR/ people.torproject.org:public_html/builds/$TORBROWSER_BUILDDIR
- torsocks ssh people.torproject.org "mv public_html/$TORBROWSER_BUILDDIR public_html/$TORBROWSER_VERSION"
+ ./dmg2mar
+
+#. Sync bundles to `linux-signer`.
+ ./sync-local-to-linux-signer
+
+#. Upload updated signing scripts to `linux-signer`. The scripts are
+# located in the `signing-release` or `signing-alpha` directory on
+# `linux-signer`.
+ ./sync-scripts-to-linux-signer
+
+#. Sign the MAR files. On `linux-signer`.
+# (replace signing-release with signing-alpha for an alpha release)
+ linux-signer$ ~/signing-release/linux-signer-signmars
-#. Transfer builds to staticiforme
+#. Authenticode signing of exe files. On `linux-signer`.
+ linux-signer$ chgrp -R yubihsm ~/$TORBROWSER_VERSION
+ linux-signer$ chmod -R g+w ~/$TORBROWSER_VERSION
+ linux-signer$ sudo su - yubihsm
+ linux-signer$ cd ~user/$TORBROWSER_VERSION
+ linux-signer$ /path/to/authenticode-signing.sh
+
+#. Authenticode timestamping.
+ ./sync-linux-signer-to-local
+ ./authenticode-timestamping.sh
+
+#. Create sha256sums-signed-build files
+ ./hash_signed_bundles.sh
+
+#. Upload sha256sums-signed-build and updated exe files to `linux-signer`.
+ ./sync-local-to-linux-signer
+
+#. Gpg signing.
+ ./linux-signer-gpg-sign
+
+#. Fetch signatures on unsigned sha256sums from other builds
+ ./download-unsigned-sha256sums-gpg-signatures-from-people-tpo
+
+#. Remove old builds on `staticiforme`
# IMPORTANT: Remove the oldest version in a series in case there is more
# than 1 available on dist.torproject.org before proceeding
# XXX: TORBROWSER_VERSION_OLDEST needs to be set
- rm -rf /srv/dist-master.torproject.org/htdocs/torbrowser/$TORBROWSER_VERSION_OLDEST
- static-update-component dist.torproject.org
+ staticiforme$ rm -rf /srv/dist-master.torproject.org/htdocs/torbrowser/$TORBROWSER_VERSION_OLDEST
+ staticiforme$ static-update-component dist.torproject.org
+
+#. Upload new build to `staticiforme`
+ ./sync-local-to-staticiforme
#. Check diskspace available on cdn.tpo
# We currently have enough disk space to host two alpha and stable
@@ -150,14 +189,15 @@
# web-fsn-01.torproject.org, and cdn-backend-sunet-01.torproject.org
#. Remove the oldest *.mar files from cdn.tpo to save space
- rm -rf /srv/cdn-master.torproject.org/htdocs/aus1/torbrowser/$TORBROWSER_VERSION_OLDEST
- static-update-component cdn.torproject.org
+ staticiforme$ rm -rf /srv/cdn-master.torproject.org/htdocs/aus1/torbrowser/$TORBROWSER_VERSION_OLDEST
+ staticiforme$ static-update-component cdn.torproject.org
#. Sync files to dist.tpo and cdn.tpo mirrored web servers
- # Obtain publish_version.sh from the tor-browser-build repo under tools/update/.
+ # Obtain publish_version.sh from the tor-browser-build repo under
+ # tools/update/ to run it on staticiforme.
# $PREV_TORBROWSER_VERSION is one of the previously published versions remaining
# on staticiforme from where the .htaccess is copied.
- ./publish_version.sh $TORBROWSER_VERSION $PREV_TORBROWSER_VERSION release # or alpha
+ staticiforme$ ./publish_version.sh $TORBROWSER_VERSION $PREV_TORBROWSER_VERSION
#. Make sure we really built from the proper Mozilla build tag by consulting
# the respective ESR release branch (for a good overview for ESR78 see
@@ -193,9 +233,10 @@
cd ..
#. Create blog post from changelog
- # See https://blog.torproject.org/blog/tor-browser-352-released for now
- # Don't forget to link to Mozilla's security advisories if this is a security
- # update.
+# Edit set-config.blog to set you local blog directory.
+# Don't forget to link to Mozilla's security advisories if this is a security
+# update.
+ ./create-blog-post
#. Check whether the .exe files got properly signed and timestamped
# Point OSSLSIGNCODE to your osslsigncode binary
@@ -215,19 +256,16 @@
../../../tools/marsigning_check.sh
popd
-#. Update and upload new update responses for the updater
- # IMPORTANT: Copy the signed MAR files back before creating the update
- # responses!
- export TORBROWSER_UPDATE_CHANNEL=release # or alpha / nightly
- make update_responses-$TORBROWSER_UPDATE_CHANNEL
- cd $TORBROWSER_UPDATE_CHANNEL/update-responses
- tar -xf update-responses-$TORBROWSER_UPDATE_CHANNEL-$TORBROWSER_VERSION.tar
- chmod 664 ${TORBROWSER_UPDATE_CHANNEL}/*
- chmod 664 ${TORBROWSER_UPDATE_CHANNEL}/.htaccess
- chmod 775 ${TORBROWSER_UPDATE_CHANNEL}/
- torsocks rsync -avP $TORBROWSER_UPDATE_CHANNEL staticiforme.torproject.org:/srv/aus1-master.torproject.org/htdocs/torbrowser/update_3/
- torsocks ssh staticiforme.torproject.org "chown -R :torwww /srv/aus1-master.torproject.org/htdocs/torbrowser/update_3/${TORBROWSER_UPDATE_CHANNEL}"
- torsocks ssh staticiforme.torproject.org "static-update-component aus1.torproject.org"
+#. Generate and upload update_responses to staticiforme
+ ./upload-update_responses-to-staticiforme
+
+#. Enable update. On `staticiforme`.
+# (for alpha release, use deploy_update_responses-alpha.sh instead)
+ staticiforme$ ~/deploy_update_responses-release.sh
+
+#. Clean linux-signer and macos-signer
+ ./finished-signing-clean-linux-signer
+ ./finished-signing-clean-macos-signer
# Upload APKs to Google Play
Log into https://play.google.com/apps/publish
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
More information about the tbb-commits
mailing list