commit c9e2ee076cafd694e0de664d3fc8f1ab1216ceb1 Author: David Goulet dgoulet@torproject.org Date: Tue Nov 2 11:56:50 2021 -0400
doc: Brand new ReleasingTor.md
Closes #40508
Signed-off-by: David Goulet dgoulet@torproject.org --- doc/HACKING/ReleasingTor.md | 261 +++++++++++----------------------------- doc/HACKING/ReleasingTor.md.old | 255 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 323 insertions(+), 193 deletions(-)
diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md index 490c100fcb..12f5d1b16b 100644 --- a/doc/HACKING/ReleasingTor.md +++ b/doc/HACKING/ReleasingTor.md @@ -1,239 +1,114 @@ -# CHECKLIST - -Here's a summary checklist, with the things that Nick messes up most often. - -Did you: - - * [ ] Copy the ChangeLog to the ReleaseNotes? - * [ ] Check that the new versions got approved? - * [ ] Check the release date in the ChangeLog? - * [ ] Update the GeoIP file? - -# Putting out a new release +# How to Release Tor
Here are the steps that the maintainer should take when putting out a -new Tor release: - -## 0. Preliminaries - -1. Get at least three of weasel/arma/Sebastian/Sina to put the new - version number in their approved versions list. Give them a few - days to do this if you can. - -2. If this is going to be an important security release, give the packagers - advance warning, via `tor-packagers@lists.torproject.org`. - - -3. Given the release date for Tor, ask the TB team about the likely release - date of a TB that contains it. See note below in "commit, upload, - announce". - -## I. Make sure it works - -1. Make sure that CI passes: have a look at the branches on gitlab. - - _Optionally_, have a look at Travis - (https://travis-ci.org/torproject/tor/branches), Appveyor - (https://ci.appveyor.com/project/torproject/tor/history), and - Jenkins (https://jenkins.torproject.org/view/tor/). - Make sure you're looking at the right branches. - - If there are any unexplained failures, try to fix them or figure them - out. - -2. Verify that there are no big outstanding issues. You might find such - issues -- - - * On Gitlab - - * On coverity scan - - * On OSS-Fuzz - -## II. Write a changelog - - -1a. (Alpha release variant) - - Gather the `changes/*` files into a changelog entry, rewriting many - of them and reordering to focus on what users and funders would find - interesting and understandable. - - To do this, run `./scripts/maint/sortChanges.py changes/* > changelog.in` - to combine headings and sort the entries. Copy the changelog.in file into - the ChangeLog. Run `format_changelog.py --inplace` (see below) to clean up - the line breaks. - - Remove the `changes/*` files that you just merged into the ChangeLog. - - After that, it's time to hand-edit and fix the issues that - lintChanges can't find: - - 1. Within each section, sort by "version it's a bugfix on", else by - numerical ticket order. - - 2. Clean them up: - - Make stuff very terse - - Describe the user-visible problem right away - - Mention relevant config options by name. If they're rare or unusual, - remind people what they're for - - Avoid starting lines with open-paren - - Present and imperative tense: not past. - - "Relays", not "servers" or "nodes" or "Tor relays". - - "Onion services", not "hidden services". +new Tor release. It is split in 3 stages and coupled with our Tor CI Release +pipeline.
- "Stop FOOing", not "Fix a bug where we would FOO". +Before we begin, first rule is to make sure:
- Try not to let any given section be longer than about a page. Break up - long sections into subsections by some sort of common subtopic. This - guideline is especially important when organizing Release Notes for - new stable releases. + - Our CI pass for each version to release + - Coverity has no new alerts
- If a given changes stanza showed up in a different release (e.g. - maint-0.2.1), be sure to make the stanzas identical (so people can - distinguish if these are the same change). +## 0. Security Release
- 3. Clean everything one last time. +To start with, if you are doing a security release, this must be done few days +prior to the release:
- 4. Run `./scripts/maint/format_changelog.py --inplace` to make it prettier + 1. If this is going to be an important security release, give the packagers + advance warning, via `tor-packagers@lists.torproject.org`.
-1b. (old-stable release variant)
- For stable releases that backport things from later, we try to compose - their releases, we try to make sure that we keep the changelog entries - identical to their original versions, with a "backport from 0.x.y.z" - note added to each section. So in this case, once you have the items - from the changes files copied together, don't use them to build a new - changelog: instead, look up the corrected versions that were merged - into ChangeLog in the main branch, and use those. +## 1. Preliminaries
- Add "backport from X.Y.Z" in the section header for these entries. +The following must be done 2 days at the very least prior to the release:
-2. Compose a short release blurb to highlight the user-facing - changes. Insert said release blurb into the ChangeLog stanza. If it's - a stable release, add it to the ReleaseNotes file too. If we're adding - to a release-* branch, manually commit the changelogs to the later - git branches too. + 1. Add the version(s) in the dirauth-conf git repository as the + RecommendedVersion and RequiredVersion so they can be approved by the + authorities and be in the consensus before the release.
-3. If there are changes that require or suggest operator intervention - before or during the update, mail operators (either dirauth or relays - list) with a headline that indicates that an action is required or - appreciated. + 2. Send a pre-release announcement to `tor-project@lists.torproject.org` in + order to inform every teams in Tor of the upcoming release. This is so + we can avoid creating release surprises and sync with other teams.
-4. If you're doing the first stable release in a series, you need to - create a ReleaseNotes for the series as a whole. To get started - there, copy all of the Changelog entries from the series into a new - file, and run `./scripts/maint/sortChanges.py` on it. That will - group them by category. Then kill every bugfix entry for fixing - bugs that were introduced within that release series; those aren't - relevant changes since the last series. At that point, it's time - to start sorting and condensing entries. (Generally, we don't edit the - text of existing entries, though.) + 3. Ask the network-team to review the `changes/` files in all versions we + are about to release. This step is encouraged but not mandatory.
-## III. Making the source release.
-1. In `maint-0.?.x`, bump the version number in `configure.ac` and run - `./scripts/main/update_versions.py` to update version numbers in other - places, and commit. Then merge `maint-0.?.x` into `release-0.?.x`. +## 2. Tarballs
- When you merge the maint branch forward to the next maint branch, or into - main, merge it with `-s ours` to avoid conflict with the version - bump. +To build the tarballs to release, we need to launch the CI release pipeline:
-2. In `release-0.?.x`, run `make distcheck`, put the tarball up in somewhere - (how about your homedir on people.torproject.org?) , and tell `#tor-dev` - about it. + https://gitlab.torproject.org/tpo/core/tor-ci-release
- If you want, wait until at least one person has built it - successfully. (We used to say "wait for others to test it", but our - CI has successfully caught these kinds of errors for the last several - years.) +The `versions.yml` needs to be modified with the Tor versions you want to +release. Once done, git commit and push to trigger the release pipeline.
-3. Make sure that the new version is recommended in the latest consensus. - (Otherwise, users will get confused when it complains to them - about its status.) +The first two stages (Preliminary and Patches) will be run automatically. The +Build stage needs to be triggered manually once all generated patches have +been merged upstream.
- If it is not, you'll need to poke Roger, Weasel, Sebastian, and Sina - again: see the note at the start of the document. + 1. Download the generated patches from the `Patches` stage.
-## IV. Commit, upload, announce + 2. For the ChangeLog and ReleaseNotes, you need to write a blurb at the top + explaining a bit the release.
-1. Sign the tarball, then sign and push the git tag: + 3. Review, modify if needed, and merged them upstream.
-```console -$ gpg -ba <the_tarball> -$ git tag -s tor-0.4.x.y-<status> -$ git push origin tag tor-0.4.x.y-<status> -``` + 4. Manually trigger the `maintained` job in the `Build` stage so the CI can + build the tarballs without errors.
- (You must do this before you update the website: the website scripts - rely on finding the version by tag.) +Once this is done, each selected developers need to build the tarballs in a +reproducible way using:
- (If your default PGP key is not the one you want to sign with, then say - "-u <keyid>" instead of "-s".) + https://gitlab.torproject.org/tpo/core/tor-ci-reproducible
-2. scp the tarball and its sig to the dist website, i.e. - `/srv/dist-master.torproject.org/htdocs/` on dist-master. Run - "static-update-component dist.torproject.org" on dist-master. +Simply run the `./build.sh` which will commit interactively the signature for +you. You then only need to git push.
- In the `project/web/tpo.git` repository, update `databags/versions.ini` - to note the new version. Push these changes to `master`. +Once all signatures have been committed:
- (NOTE: Due to #17805, there can only be one stable version listed at - once. Nonetheless, do not call your version "alpha" if it is stable, - or people will get confused.) + 1. Manually trigger the `signature` job in the `Post-process` stage of the + CI release pipeline.
- (NOTE: It will take a while for the website update scripts to update - the website.) + 2. If it passes, the tarball(s) and signature(s) will be available as + artifacts and should be used for the release.
-3. Email the tor-packagers@lists.torproject.org mailing list to tell them - about the new release. + 3. Put them on `dist.torproject.org`:
- Also, email tor-packagers@lists.torproject.org. + Upload the tarball and its sig to the dist website, i.e. + `/srv/dist-master.torproject.org/htdocs/` on dist-master. Run + "static-update-component dist.torproject.org" on dist-master.
- Mention where to download the tarball (`https://dist.torproject.org/%60). + In the `project/web/tpo.git` repository, update `databags/versions.ini` + to note the new version. Push these changes to `master`.
- Include a link to the changelog. + (NOTE: Due to #17805, there can only be one stable version listed at once. + Nonetheless, do not call your version "alpha" if it is stable, or people + will get confused.)
-4. Wait for the download page to be updated. (If you don't do this before you - announce, people will be confused.) + (NOTE: It will take a while for the website update scripts to update the + website.)
-5. Mail the release blurb and ChangeLog to tor-talk (development release) or - tor-announce (stable).
- Post the changelog on the blog as well. You can generate a - blog-formatted version of the changelog with - `./scripts/maint/format_changelog.py -B` +## 3. Post Process
- When you post, include an estimate of when the next TorBrowser - releases will come out that include this Tor release. This will - usually track https://wiki.mozilla.org/RapidRelease/Calendar , but it - can vary. +Once the tarballs have been uploaded and are ready to be announced, we need to +do the following:
- For templates to use when announcing, see: - https://gitlab.torproject.org/tpo/core/team/-/wikis/NetworkTeam/Announcement... + 1. Merge upstream the artifacts from the `patches` job in the + `Post-process` stage of the CI release pipeline.
-## V. Aftermath and cleanup + 2. Write and post the release announcement for the `forum.torproject.net` + in the `News -> Tor Release Announcement` category.
-1. If it's a stable release, bump the version number in the - `maint-x.y.z` branch to "newversion-dev", and do a `merge -s ours` - merge to avoid taking that change into main. + Mention in which Tor Browser version (with dates) the release will be + in. This usually only applies to the latest stable.
-2. If there is a new `maint-x.y.z` branch, create a Travis CI cron job that - builds the release every week. (It's ok to skip the weekly build if the - branch was updated in the last 24 hours.) +### New Stable
-3. Forward-port the ChangeLog (and ReleaseNotes if appropriate) to the - main branch. + 1. Create the `maint-x.y.z` and `release-x.y.z` branches and update the + `./scripts/git/git-list-tor-branches.sh` with the new version.
-4. Keep an eye on the blog post, to moderate comments and answer questions.
## Appendix: An alternative means to notify packagers
diff --git a/doc/HACKING/ReleasingTor.md.old b/doc/HACKING/ReleasingTor.md.old new file mode 100644 index 0000000000..490c100fcb --- /dev/null +++ b/doc/HACKING/ReleasingTor.md.old @@ -0,0 +1,255 @@ +# CHECKLIST + +Here's a summary checklist, with the things that Nick messes up most often. + +Did you: + + * [ ] Copy the ChangeLog to the ReleaseNotes? + * [ ] Check that the new versions got approved? + * [ ] Check the release date in the ChangeLog? + * [ ] Update the GeoIP file? + +# Putting out a new release + +Here are the steps that the maintainer should take when putting out a +new Tor release: + +## 0. Preliminaries + +1. Get at least three of weasel/arma/Sebastian/Sina to put the new + version number in their approved versions list. Give them a few + days to do this if you can. + +2. If this is going to be an important security release, give the packagers + advance warning, via `tor-packagers@lists.torproject.org`. + + +3. Given the release date for Tor, ask the TB team about the likely release + date of a TB that contains it. See note below in "commit, upload, + announce". + +## I. Make sure it works + +1. Make sure that CI passes: have a look at the branches on gitlab. + + _Optionally_, have a look at Travis + (https://travis-ci.org/torproject/tor/branches), Appveyor + (https://ci.appveyor.com/project/torproject/tor/history), and + Jenkins (https://jenkins.torproject.org/view/tor/). + Make sure you're looking at the right branches. + + If there are any unexplained failures, try to fix them or figure them + out. + +2. Verify that there are no big outstanding issues. You might find such + issues -- + + * On Gitlab + + * On coverity scan + + * On OSS-Fuzz + +## II. Write a changelog + + +1a. (Alpha release variant) + + Gather the `changes/*` files into a changelog entry, rewriting many + of them and reordering to focus on what users and funders would find + interesting and understandable. + + To do this, run `./scripts/maint/sortChanges.py changes/* > changelog.in` + to combine headings and sort the entries. Copy the changelog.in file into + the ChangeLog. Run `format_changelog.py --inplace` (see below) to clean up + the line breaks. + + Remove the `changes/*` files that you just merged into the ChangeLog. + + After that, it's time to hand-edit and fix the issues that + lintChanges can't find: + + 1. Within each section, sort by "version it's a bugfix on", else by + numerical ticket order. + + 2. Clean them up: + + Make stuff very terse + + Describe the user-visible problem right away + + Mention relevant config options by name. If they're rare or unusual, + remind people what they're for + + Avoid starting lines with open-paren + + Present and imperative tense: not past. + + "Relays", not "servers" or "nodes" or "Tor relays". + + "Onion services", not "hidden services". + + "Stop FOOing", not "Fix a bug where we would FOO". + + Try not to let any given section be longer than about a page. Break up + long sections into subsections by some sort of common subtopic. This + guideline is especially important when organizing Release Notes for + new stable releases. + + If a given changes stanza showed up in a different release (e.g. + maint-0.2.1), be sure to make the stanzas identical (so people can + distinguish if these are the same change). + + 3. Clean everything one last time. + + 4. Run `./scripts/maint/format_changelog.py --inplace` to make it prettier + +1b. (old-stable release variant) + + For stable releases that backport things from later, we try to compose + their releases, we try to make sure that we keep the changelog entries + identical to their original versions, with a "backport from 0.x.y.z" + note added to each section. So in this case, once you have the items + from the changes files copied together, don't use them to build a new + changelog: instead, look up the corrected versions that were merged + into ChangeLog in the main branch, and use those. + + Add "backport from X.Y.Z" in the section header for these entries. + +2. Compose a short release blurb to highlight the user-facing + changes. Insert said release blurb into the ChangeLog stanza. If it's + a stable release, add it to the ReleaseNotes file too. If we're adding + to a release-* branch, manually commit the changelogs to the later + git branches too. + +3. If there are changes that require or suggest operator intervention + before or during the update, mail operators (either dirauth or relays + list) with a headline that indicates that an action is required or + appreciated. + +4. If you're doing the first stable release in a series, you need to + create a ReleaseNotes for the series as a whole. To get started + there, copy all of the Changelog entries from the series into a new + file, and run `./scripts/maint/sortChanges.py` on it. That will + group them by category. Then kill every bugfix entry for fixing + bugs that were introduced within that release series; those aren't + relevant changes since the last series. At that point, it's time + to start sorting and condensing entries. (Generally, we don't edit the + text of existing entries, though.) + +## III. Making the source release. + +1. In `maint-0.?.x`, bump the version number in `configure.ac` and run + `./scripts/main/update_versions.py` to update version numbers in other + places, and commit. Then merge `maint-0.?.x` into `release-0.?.x`. + + When you merge the maint branch forward to the next maint branch, or into + main, merge it with `-s ours` to avoid conflict with the version + bump. + +2. In `release-0.?.x`, run `make distcheck`, put the tarball up in somewhere + (how about your homedir on people.torproject.org?) , and tell `#tor-dev` + about it. + + If you want, wait until at least one person has built it + successfully. (We used to say "wait for others to test it", but our + CI has successfully caught these kinds of errors for the last several + years.) + +3. Make sure that the new version is recommended in the latest consensus. + (Otherwise, users will get confused when it complains to them + about its status.) + + If it is not, you'll need to poke Roger, Weasel, Sebastian, and Sina + again: see the note at the start of the document. + +## IV. Commit, upload, announce + +1. Sign the tarball, then sign and push the git tag: + +```console +$ gpg -ba <the_tarball> +$ git tag -s tor-0.4.x.y-<status> +$ git push origin tag tor-0.4.x.y-<status> +``` + + (You must do this before you update the website: the website scripts + rely on finding the version by tag.) + + (If your default PGP key is not the one you want to sign with, then say + "-u <keyid>" instead of "-s".) + +2. scp the tarball and its sig to the dist website, i.e. + `/srv/dist-master.torproject.org/htdocs/` on dist-master. Run + "static-update-component dist.torproject.org" on dist-master. + + In the `project/web/tpo.git` repository, update `databags/versions.ini` + to note the new version. Push these changes to `master`. + + (NOTE: Due to #17805, there can only be one stable version listed at + once. Nonetheless, do not call your version "alpha" if it is stable, + or people will get confused.) + + (NOTE: It will take a while for the website update scripts to update + the website.) + +3. Email the tor-packagers@lists.torproject.org mailing list to tell them + about the new release. + + Also, email tor-packagers@lists.torproject.org. + + Mention where to download the tarball (`https://dist.torproject.org/%60). + + Include a link to the changelog. + +4. Wait for the download page to be updated. (If you don't do this before you + announce, people will be confused.) + +5. Mail the release blurb and ChangeLog to tor-talk (development release) or + tor-announce (stable). + + Post the changelog on the blog as well. You can generate a + blog-formatted version of the changelog with + `./scripts/maint/format_changelog.py -B` + + When you post, include an estimate of when the next TorBrowser + releases will come out that include this Tor release. This will + usually track https://wiki.mozilla.org/RapidRelease/Calendar , but it + can vary. + + For templates to use when announcing, see: + https://gitlab.torproject.org/tpo/core/team/-/wikis/NetworkTeam/Announcement... + +## V. Aftermath and cleanup + +1. If it's a stable release, bump the version number in the + `maint-x.y.z` branch to "newversion-dev", and do a `merge -s ours` + merge to avoid taking that change into main. + +2. If there is a new `maint-x.y.z` branch, create a Travis CI cron job that + builds the release every week. (It's ok to skip the weekly build if the + branch was updated in the last 24 hours.) + +3. Forward-port the ChangeLog (and ReleaseNotes if appropriate) to the + main branch. + +4. Keep an eye on the blog post, to moderate comments and answer questions. + +## Appendix: An alternative means to notify packagers + +If for some reason you need to contact a bunch of packagers without +using the publicly archived tor-packagers list, you can try these +people: + + - {weasel,sysrqb,mikeperry} at torproject dot org + - {blueness} at gentoo dot org + - {paul} at invizbox dot io + - {vincent} at invizbox dot com + - {lfleischer} at archlinux dot org + - {Nathan} at freitas dot net + - {mike} at tig dot as + - {tails-rm} at boum dot org + - {simon} at sdeziel.info + - {yuri} at freebsd.org + - {mh+tor} at scrit.ch + - {security} at brave.com