This is an automated email from the git hooks/post-receive script. boklm pushed a commit to branch master in repository tor-browser-spec. The following commit(s) were added to refs/heads/master by this push: new 0381b27 Bug 40027: Update processes/ReleaseProcess following tor-browser-build#40414 0381b27 is described below commit 0381b271b4f4d57a37024a2aeaf21681c59b59c5 Author: Nicolas Vigier <boklm@torproject.org> AuthorDate: Thu Mar 24 12:47:39 2022 +0100 Bug 40027: Update processes/ReleaseProcess following tor-browser-build#40414 --- processes/ReleaseProcess | 242 +++++++++++++++++++++++++++-------------------- 1 file changed, 140 insertions(+), 102 deletions(-) diff --git a/processes/ReleaseProcess b/processes/ReleaseProcess index 08db508..21a3b9b 100644 --- a/processes/ReleaseProcess +++ b/processes/ReleaseProcess @@ -3,6 +3,10 @@ # "May this part of our job one day be replaced by a small shell script" # +##################### +### Prepare build ### +##################### + #. Tag any relevant component versions. # In particular: before tagging the tor-browser tag, the final code from # Torbutton needs to get included and in order to avoid unnecessary commit @@ -24,7 +28,14 @@ torsocks git push origin master:master torsocks git push origin --tags + +##################### +### Build ### +##################### + #. Build and generate incremental MAR files. + git tag -v tbb-$TORBROWSER_VERSION-buildN + git checkout tbb-$TORBROWSER_VERSION-buildN make && make incrementals-release # `make alpha && make incrementals-alpha` #. Compare the SHA256 sums of the bundles and MAR files with an independent @@ -49,96 +60,124 @@ # and changelog. # For stable releases put tails-dev@boum.org into Cc. + +########################## +### Signing and upload ### +########################## + +#. You need a `pkgstage` machine to store the bundles during the signing +# process. This machine should be: +# - secure (you will ssh to the signing machines and staticiforme +# from there) +# - with good bandwidth (you will download/upload bundles from there) +# +# All steps from the `Signing and upload` are run from the `pkgstage` +# machine unless mentioned otherwise. +# +# The signing scripts are located in the tor-browser-build/tools/signing +# directory. + +#. Checkout the tor-browser-build.git commit corresponding to the +# tor browser version you need to sign/publish. + cd tor-browser-build + git tag -v tbb-$TORBROWSER_VERSION-buildN + git checkout tbb-$TORBROWSER_VERSION-buildN + +#. Change to the tor-browser-build/tools/signing directory. All following +# commands are run from this directory. + cd tor-browser-build/tools/signing + +#. Set hosts information. + vim set-config.hosts + +#. Set tor browser version information. You should set tbb_version, +# tbb_version_build, tbb_version_type. This should match the information +# in rbm.conf. + vim set-config.tbb-version + +#. Download the tor browser build to the `pkgstage` machine from your +# build machine (configured in set-config.hosts). + ./sync-builder-unsigned-to-local-signed + #. Codesign the macOS dmg files. - # setup - torsocks ssh mac-signer "mkdir $TORBROWSER_VERSION" - torsocks rsync -avP $TORBROWSER_BUILDDIR/*.dmg mac-signer:$TORBROWSER_VERSION/ - torsocks ssh mac-signer - # Unlock the keychain and then... - cd $TORBROWSER_VERSION - # Enable networking - networksetup -setsecurewebproxystate Ethernet on - # Sign the bundles. - ../gatekeeper-signing.sh $TORBROWSER_VERSION - # notarize and staple - ../notarization.sh $TORBROWSER_VERSION - ../stapler.sh $TORBROWSER_VERSION - # Check that it worked. - unzip -d test tb-$TORBROWSER_VERSION-osx_zh-CN-stapled.zip - pushd test - # Both should be "Tor Browser.app: Accepted" with "source=Notarized Developer ID" - spctl -vvvv --assess --type=exec --context context:primary-signature Tor\ Browser.app/ - spctl -vvvv --assess --type=open --context context:primary-signature Tor\ Browser.app/ - popd - rm -rf test - # Disable networking - networksetup -setsocksfirewallproxystate Ethernet off - exit - torsocks rsync -avP mac-signer:$TORBROWSER_VERSION/tb-*-stapled.zip /path/to/builddir/$TORBROWSER_VERSION/ +# +# Updload the tor browser dmg files to `macos-signer` + ./sync-macos-local-to-macos-signer + +# Upload updated signing scripts to `macos-signer`. The scripts are +# located in the `signing-release` or `signing-alpha` directory on +# `macos-signer`. + ./sync-scripts-to-macos-signer + +# Run proxy for `macos-signer`. You may need to kill an old proxy +# process on `macos-signer` if it was still running. + ./macos-signer-proxy - cp -rT tor-browser-build/projects/tor-browser/Bundle-Data/mac-applications.dmg dmg +# Sign the bundles. On `macos-signer`. +# (replace signing-release with signing-alpha for an alpha release) + macos-signer$ ~/signing-release/macos-signer-gatekeeper-signing + +# Notarize the bundles. On `macos-signer`. +# (replace signing-release with signing-alpha for an alpha release) + macos-signer$ ~/signing-release/macos-signer-notarization + +# Staple the bundles. On `macos-signer`. +# (replace signing-release with signing-alpha for an alpha release) + macos-signer$ ~/signing-release/macos-signer-stapler + +# Download the stapled bundles to `pkgstage`. + ./sync-macos-signer-stapled-to-macos-local-stapled + +#. Regenerate macOS DMG files from stapled zip files. + ./gatekeeper-bundling.sh #. Regenerate macOS MAR files from code signed dmg files. - # XXX Go to your directory prepared for recreating the .dmg files and containing - # the uploaded .zip files. - ./gatekeeper-bundling.sh $TORBROWSER_VERSION - rsync -avP ../$TORBROWSER_VERSION-signed/*.dmg $TORBROWSER_BUILDDIR/ - cd tor-browser-build - mv $TORBROWSER_BUILDDIR/ release/signed/ (or alpha) - # The code signed dmg files should be in the $TORBROWSER_VERSION directory - # Install a recent p7zip version (see ../tools/dmg2mar for instructions) - make dmg2mar-release # or `make dmg2mar-alpha` - -#. Sign the MAR files - # First, copy the tor-browser-bundle tree to the signing machine. XXX: This - # still uses part of the old Gitian related infrastructure. - torsocks rsync -avP $TORBROWSER_BUILDDIR/../../../ signing-machine - torsocks ssh signing-machine "mkdir tor-browser-bundle/gitian/$TORBROWSER_VERSION" - torsocks rsync -avP $TORBROWSER_BUILDDIR/*.mar signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/ - torsocks ssh signing-machine - cd tor-browser-bundle/gitian - # XXX Modify the signmars.sh script to comment out the eval call. - export TORBROWSER_VERSION=$TORBROWSER_VERSION - export NSS_DB_DIR=/path/to/nssdb - # Only needed if you are not owner of the marsigner cert - export NSS_CERTNAME=your_certname - make signmars - exit - torsocks rsync -avP signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/*.mar $TORBROWSER_BUILDDIR/ - -#. Sign individual bundle files. - # Authenticode signing first - torsocks ssh windows-signing-machine "mkdir tor-browser-bundle/gitian/$TORBROWSER_VERSION" - torsocks rsync -avP $TORBROWSER_BUILDDIR/*.exe windows-signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/ - torsocks ssh windows-signing-machine - cd tor-browser-bundle/gitian/$TORBROWSER_VERSION - /path/to/authenticode-signing.sh - exit - torsocks rsync -avP window-signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/*.exe $TORBROWSER_BUILDDIR/ - # Authenticode timestamping next - cd $TORBROWSER_BUILDDIR - export OSSLSIGNCODE=/path/to/osslsigncode - /path/to/authenticode-timestamping.sh - # Hashes of the signed bundles - ../../../tools/hash_signed_bundles.sh - # All the GPG signatures at last - torsocks rsync -avP $TORBROWSER_BUILDDIR/* signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/ - cd tor-browser-bundle/gitian/$TORBROWSER_VERSION - /path/to/tbb-signing.sh - exit - torsocks rsync -avP signing-machine:tor-browser-bundle/gitian/$TORBROWSER_VERSION/ $TORBROWSER_BUILDDIR - # Fetch signatures on unsigned sha256sums from other builds - -#. Sync to people.torproject.org - torsocks rsync -avP $TORBROWSER_BUILDDIR/ people.torproject.org:public_html/builds/$TORBROWSER_BUILDDIR - torsocks ssh people.torproject.org "mv public_html/$TORBROWSER_BUILDDIR public_html/$TORBROWSER_VERSION" + ./dmg2mar + +#. Sync bundles to `linux-signer`. + ./sync-local-to-linux-signer + +#. Upload updated signing scripts to `linux-signer`. The scripts are +# located in the `signing-release` or `signing-alpha` directory on +# `linux-signer`. + ./sync-scripts-to-linux-signer + +#. Sign the MAR files. On `linux-signer`. +# (replace signing-release with signing-alpha for an alpha release) + linux-signer$ ~/signing-release/linux-signer-signmars -#. Transfer builds to staticiforme +#. Authenticode signing of exe files. On `linux-signer`. + linux-signer$ chgrp -R yubihsm ~/$TORBROWSER_VERSION + linux-signer$ chmod -R g+w ~/$TORBROWSER_VERSION + linux-signer$ sudo su - yubihsm + linux-signer$ cd ~user/$TORBROWSER_VERSION + linux-signer$ /path/to/authenticode-signing.sh + +#. Authenticode timestamping. + ./sync-linux-signer-to-local + ./authenticode-timestamping.sh + +#. Create sha256sums-signed-build files + ./hash_signed_bundles.sh + +#. Upload sha256sums-signed-build and updated exe files to `linux-signer`. + ./sync-local-to-linux-signer + +#. Gpg signing. + ./linux-signer-gpg-sign + +#. Fetch signatures on unsigned sha256sums from other builds + ./download-unsigned-sha256sums-gpg-signatures-from-people-tpo + +#. Remove old builds on `staticiforme` # IMPORTANT: Remove the oldest version in a series in case there is more # than 1 available on dist.torproject.org before proceeding # XXX: TORBROWSER_VERSION_OLDEST needs to be set - rm -rf /srv/dist-master.torproject.org/htdocs/torbrowser/$TORBROWSER_VERSION_OLDEST - static-update-component dist.torproject.org + staticiforme$ rm -rf /srv/dist-master.torproject.org/htdocs/torbrowser/$TORBROWSER_VERSION_OLDEST + staticiforme$ static-update-component dist.torproject.org + +#. Upload new build to `staticiforme` + ./sync-local-to-staticiforme #. Check diskspace available on cdn.tpo # We currently have enough disk space to host two alpha and stable @@ -150,14 +189,15 @@ # web-fsn-01.torproject.org, and cdn-backend-sunet-01.torproject.org #. Remove the oldest *.mar files from cdn.tpo to save space - rm -rf /srv/cdn-master.torproject.org/htdocs/aus1/torbrowser/$TORBROWSER_VERSION_OLDEST - static-update-component cdn.torproject.org + staticiforme$ rm -rf /srv/cdn-master.torproject.org/htdocs/aus1/torbrowser/$TORBROWSER_VERSION_OLDEST + staticiforme$ static-update-component cdn.torproject.org #. Sync files to dist.tpo and cdn.tpo mirrored web servers - # Obtain publish_version.sh from the tor-browser-build repo under tools/update/. + # Obtain publish_version.sh from the tor-browser-build repo under + # tools/update/ to run it on staticiforme. # $PREV_TORBROWSER_VERSION is one of the previously published versions remaining # on staticiforme from where the .htaccess is copied. - ./publish_version.sh $TORBROWSER_VERSION $PREV_TORBROWSER_VERSION release # or alpha + staticiforme$ ./publish_version.sh $TORBROWSER_VERSION $PREV_TORBROWSER_VERSION #. Make sure we really built from the proper Mozilla build tag by consulting # the respective ESR release branch (for a good overview for ESR78 see @@ -193,9 +233,10 @@ cd .. #. Create blog post from changelog - # See https://blog.torproject.org/blog/tor-browser-352-released for now - # Don't forget to link to Mozilla's security advisories if this is a security - # update. +# Edit set-config.blog to set you local blog directory. +# Don't forget to link to Mozilla's security advisories if this is a security +# update. + ./create-blog-post #. Check whether the .exe files got properly signed and timestamped # Point OSSLSIGNCODE to your osslsigncode binary @@ -215,19 +256,16 @@ ../../../tools/marsigning_check.sh popd -#. Update and upload new update responses for the updater - # IMPORTANT: Copy the signed MAR files back before creating the update - # responses! - export TORBROWSER_UPDATE_CHANNEL=release # or alpha / nightly - make update_responses-$TORBROWSER_UPDATE_CHANNEL - cd $TORBROWSER_UPDATE_CHANNEL/update-responses - tar -xf update-responses-$TORBROWSER_UPDATE_CHANNEL-$TORBROWSER_VERSION.tar - chmod 664 ${TORBROWSER_UPDATE_CHANNEL}/* - chmod 664 ${TORBROWSER_UPDATE_CHANNEL}/.htaccess - chmod 775 ${TORBROWSER_UPDATE_CHANNEL}/ - torsocks rsync -avP $TORBROWSER_UPDATE_CHANNEL staticiforme.torproject.org:/srv/aus1-master.torproject.org/htdocs/torbrowser/update_3/ - torsocks ssh staticiforme.torproject.org "chown -R :torwww /srv/aus1-master.torproject.org/htdocs/torbrowser/update_3/${TORBROWSER_UPDATE_CHANNEL}" - torsocks ssh staticiforme.torproject.org "static-update-component aus1.torproject.org" +#. Generate and upload update_responses to staticiforme + ./upload-update_responses-to-staticiforme + +#. Enable update. On `staticiforme`. +# (for alpha release, use deploy_update_responses-alpha.sh instead) + staticiforme$ ~/deploy_update_responses-release.sh + +#. Clean linux-signer and macos-signer + ./finished-signing-clean-linux-signer + ./finished-signing-clean-macos-signer # Upload APKs to Google Play Log into https://play.google.com/apps/publish -- To stop receiving notification emails like this one, please contact the administrator of this repository.