December 2018 - tor-commits - lists.torproject.org

[tor/release-0.2.9] Merge branch 'maint-0.2.9' into release-0.2.9
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit 32ed153282026178da84c8ca6f80d82c97d15e0c Merge: ef87966ec 3b5188576 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 11 10:00:22 2018 -0500 Merge branch 'maint-0.2.9' into release-0.2.9 changes/ticket24803 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

1 0

[tor/release-0.3.3] Merge branch 'maint-0.3.3' into release-0.3.3
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit fd2d7c5d1993b6ff174b705cc0e07e9f0b707685 Merge: 8be8cc50e 93019b478 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 11 10:00:22 2018 -0500 Merge branch 'maint-0.3.3' into release-0.3.3 changes/ticket24803 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

1 0

[tor/release-0.3.3] Merge branch 'maint-0.2.9' into maint-0.3.3
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit 93019b4787e71a56d3d3f3bc89f65919930f1dda Merge: c1f919158 3b5188576 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 11 10:00:22 2018 -0500 Merge branch 'maint-0.2.9' into maint-0.3.3 changes/ticket24803 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

1 0

[tor/release-0.3.4] Note ticket number in changes file for 24803
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit 3b5188576f7cbf20284a9f6e292fd2951504242f Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 11 10:00:19 2018 -0500 Note ticket number in changes file for 24803 --- changes/ticket24803 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changes/ticket24803 b/changes/ticket24803 index c46d3804d..e76a9eeab 100644 --- a/changes/ticket24803 +++ b/changes/ticket24803 @@ -2,4 +2,4 @@ - Replace the 150 fallbacks originally introduced in Tor 0.3.3.1-alpha in January 2018 (of which ~115 were still functional), with a list of 157 fallbacks (92 new, 65 existing, 85 removed) generated in - December 2018. + December 2018. Closes ticket 24803.

1 0

[torspec/master] Mark proposal 297-safer-protocol-shutdowns.txt as implemented (#27735)
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit 18fcb9ab42cca4ce963202a22cd1f93f68ecd57c Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 11 09:52:35 2018 -0500 Mark proposal 297-safer-protocol-shutdowns.txt as implemented (#27735) --- proposals/000-index.txt | 4 ++-- proposals/297-safer-protover-shutdowns.txt | 10 +++++++++- tor-spec.txt | 16 ++++++++++++---- 3 files changed, 23 insertions(+), 7 deletions(-) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 9746434..dda8572 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -217,7 +217,7 @@ Proposals by number: 294 TLS 1.3 Migration [DRAFT] 295 Using ADL-GCM for relay cryptography (solving the crypto-tagging attack) [OPEN] 296 Have Directory Authorities expose raw bandwidth list files [OPEN] -297 Relaxing the protover-based shutdown rules [OPEN] +297 Relaxing the protover-based shutdown rules [CLOSED] 298 Putting family lines in canonical form [CLOSED] @@ -249,7 +249,6 @@ Proposals by status: 289 Authenticating sendme cells to mitigate bandwidth attacks 295 Using ADL-GCM for relay cryptography (solving the crypto-tagging attack) 296 Have Directory Authorities expose raw bandwidth list files - 297 Relaxing the protover-based shutdown rules [for 0.3.5.x] ACCEPTED: 188 Bridge Guards and other anti-enumeration defenses 249 Allow CREATE cells with >505 bytes of handshake data @@ -355,6 +354,7 @@ Proposals by status: 283 Move IPv6 ORPorts from microdescriptors to the microdesc consensus [for 0.3.3.x] [in 0.3.3.1-alpha] 284 Hidden Service v3 Control Port 293 Other ways for relays to know when to publish [for 0.3.5] [in 0.4.0.1-alpha] + 297 Relaxing the protover-based shutdown rules [for 0.3.5.x] [in 0.4.0.x] 298 Putting family lines in canonical form [for 0.3.6.x] [in 0.4.0.1-alpha] SUPERSEDED: 112 Bring Back Pathlen Coin Weight diff --git a/proposals/297-safer-protover-shutdowns.txt b/proposals/297-safer-protover-shutdowns.txt index 0307df3..59fbac1 100644 --- a/proposals/297-safer-protover-shutdowns.txt +++ b/proposals/297-safer-protover-shutdowns.txt @@ -2,8 +2,16 @@ Filename: 297-safer-protover-shutdowns.txt Title: Relaxing the protover-based shutdown rules Author: Nick Mathewson Created: 19-Sep-2018 -Status: Open +Status: Closed Target: 0.3.5.x +Implemented-In: 0.4.0.x + +IMPLEMENTATION NOTE: + + We went with the proposed change in section 2. The "release date" is + now updated by the "make update-versions" target whenever the version + number is incremented. Maintainers may also manually set the "release + date" to the future. 1. Introduction diff --git a/tor-spec.txt b/tor-spec.txt index 97d5159..f4d3f53 100644 --- a/tor-spec.txt +++ b/tor-spec.txt @@ -1935,19 +1935,27 @@ see tor-design.pdf. it should warn its operator that the relay is obsolete. - When a relay lacks a protocol listed in required-relay-protocols, it - must not attempt to join the network. + should warn its operator as above. If the consensus is newer than the + date when the software was released or scheduled for release, it must + not attempt to join the network. - When a client lacks a protocol listed in recommended-client-protocols, it should warn the user that the client is obsolete. - - When a client lacks a protocol listed in required-client-protocols, it - must not connect to the network. This implements a "safe forward - shutdown" mechanism for zombie clients. + - When a client lacks a protocol listed in required-client-protocols, + it should warn the user as above. If the consensus is newer than the + date when the software was released, it must not connect to the + network. This implements a "safe forward shutdown" mechanism for + zombie clients. - If a client or relay has a cached consensus telling it that a given protocol is required, and it does not implement that protocol, it SHOULD NOT try to fetch a newer consensus. + Software release dates SHOULD be automatically updated as part of the + release process, to prevent forgetting to move them forward. Software + release dates MAY be manually adjusted by maintainers if necessary. + Starting in version 0.2.9.4-alpha, the initial required protocols for clients that we will Recommend and Require are:

1 0

[tor/master] Merge remote-tracking branch 'github/prop297'
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit b915b6cd2171dd9b810bd318739e191db1a810b5 Merge: 27e426992 7da06e43d Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 11 09:44:57 2018 -0500 Merge remote-tracking branch 'github/prop297' Makefile.am | 2 +- changes/prop297 | 7 ++ configure.ac | 10 ++- doc/HACKING/ReleasingTor.md | 6 +- scripts/maint/updateVersions.pl.in | 59 ---------------- scripts/maint/update_versions.py | 133 +++++++++++++++++++++++++++++++++++ src/core/or/versions.c | 19 +++++ src/core/or/versions.h | 2 + src/feature/nodelist/networkstatus.c | 5 +- 9 files changed, 176 insertions(+), 67 deletions(-)

1 0

[tor/master] Rewrite updateVersions script in Python, add datestamp functionality.
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit b16d6453adde2b13d2aa95a74ec3c09a259f185f Author: Nick Mathewson <nickm(a)torproject.org> Date: Sat Nov 24 20:03:48 2018 -0500 Rewrite updateVersions script in Python, add datestamp functionality. This updateVersions.pl script was one of the only essential perl scripts left in out maint system, and was the only one that used autoconf to fill in the script. This script adds a feature to define an APPROX_RELEASE_DATE macro that is updated when the version changes. We'll use this to implement prop297, so that we have an accurate view of when a release date happens. --- Makefile.am | 2 +- configure.ac | 10 ++- doc/HACKING/ReleasingTor.md | 6 +- scripts/maint/updateVersions.pl.in | 59 ---------------- scripts/maint/update_versions.py | 133 +++++++++++++++++++++++++++++++++++++ 5 files changed, 144 insertions(+), 66 deletions(-) diff --git a/Makefile.am b/Makefile.am index 803e9d00d..cbe94ad93 100644 --- a/Makefile.am +++ b/Makefile.am @@ -416,7 +416,7 @@ endif .PHONY: update-versions update-versions: - $(PERL) $(top_builddir)/scripts/maint/updateVersions.pl + abs_top_srcdir="$(abs_top_srcdir)" $(PYTHON) $(top_srcdir)/scripts/maint/update_versions.py .PHONY: callgraph callgraph: diff --git a/configure.ac b/configure.ac index 31e41c3bb..7f0d37544 100644 --- a/configure.ac +++ b/configure.ac @@ -8,6 +8,15 @@ AC_INIT([tor],[0.4.0.0-alpha-dev]) AC_CONFIG_SRCDIR([src/app/main/tor_main.c]) AC_CONFIG_MACRO_DIR([m4]) +# DO NOT EDIT THIS DEFINITION BY HAND UNLESS YOU KNOW WHAT YOU'RE DOING. +# +# The update_versions.py script updates this definition when the +# version number changes. Tor uses it to make sure that it +# only shuts down for missing "required protocols" when those protocols +# are listed as required by a consensus after this date. +AC_DEFINE(APPROX_RELEASE_DATE, ["2019-01-15"], # for 0.4.0.0-alpha-dev + [Approximate date when this software was released. (Updated when the version changes.)]) + # "foreign" means we don't follow GNU package layout standards # "1.11" means we require automake version 1.11 or newer # "subdir-objects" means put .o files in the same directory as the .c files @@ -2417,7 +2426,6 @@ AC_CONFIG_FILES([ src/config/torrc.minimal src/rust/.cargo/config scripts/maint/checkOptionDocs.pl - scripts/maint/updateVersions.pl ]) if test "x$asciidoc" = "xtrue" && test "$ASCIIDOC" = "none"; then diff --git a/doc/HACKING/ReleasingTor.md b/doc/HACKING/ReleasingTor.md index b5444afa9..b260cdbb1 100644 --- a/doc/HACKING/ReleasingTor.md +++ b/doc/HACKING/ReleasingTor.md @@ -131,13 +131,9 @@ new Tor release: === III. Making the source release. 1. In `maint-0.?.x`, bump the version number in `configure.ac` and run - `perl scripts/maint/updateVersions.pl` to update version numbers in other + `make update-versions` to update version numbers in other places, and commit. Then merge `maint-0.?.x` into `release-0.?.x`. - (NOTE: To bump the version number, edit `configure.ac`, and then run - either `make`, or `perl scripts/maint/updateVersions.pl`, depending on - your version.) - When you merge the maint branch forward to the next maint branch, or into master, merge it with "-s ours" to avoid a needless version bump. diff --git a/scripts/maint/updateVersions.pl.in b/scripts/maint/updateVersions.pl.in deleted file mode 100755 index 65c51a1f2..000000000 --- a/scripts/maint/updateVersions.pl.in +++ /dev/null @@ -1,59 +0,0 @@ -#!/usr/bin/perl -w - -$CONFIGURE_IN = '@abs_top_srcdir@/configure.ac'; -$ORCONFIG_H = '@abs_top_srcdir@/src/win32/orconfig.h'; -$TOR_NSI = '@abs_top_srcdir@/contrib/win32build/tor-mingw.nsi.in'; - -$quiet = 1; - -sub demand { - my $fn = shift; - die "Missing file $fn" unless (-f $fn); -} - -demand($CONFIGURE_IN); -demand($ORCONFIG_H); -demand($TOR_NSI); - -# extract version from configure.ac - -open(F, $CONFIGURE_IN) or die "$!"; -$version = undef; -while (<F>) { - if (/AC_INIT$\[tor\],\s*\[([^\]]*)\]$/) { - $version = $1; - last; - } -} -die "No version found" unless $version; -print "Tor version is $version\n" unless $quiet; -close F; - -sub correctversion { - my ($fn, $defchar) = @_; - undef $/; - open(F, $fn) or die "$!"; - my $s = <F>; - close F; - if ($s =~ /^$defchar(?:)define\s+VERSION\s+\"([^\"]+)\"/m) { - $oldver = $1; - if ($oldver ne $version) { - print "Version mismatch in $fn: It thinks that the version is $oldver. I think it's $version. Fixing.\n"; - $line = $defchar . "define VERSION \"$version\""; - open(F, ">$fn.bak"); - print F $s; - close F; - $s =~ s/^$defchar(?:)define\s+VERSION.*?$/$line/m; - open(F, ">$fn"); - print F $s; - close F; - } else { - print "$fn has the correct version. Good.\n" unless $quiet; - } - } else { - print "Didn't find a version line in $fn -- uh oh.\n"; - } -} - -correctversion($TOR_NSI, "!"); -correctversion($ORCONFIG_H, "#"); diff --git a/scripts/maint/update_versions.py b/scripts/maint/update_versions.py new file mode 100755 index 000000000..8067f2c6c --- /dev/null +++ b/scripts/maint/update_versions.py @@ -0,0 +1,133 @@ +#!/usr/bin/env python + +from __future__ import print_function + +import io +import os +import re +import sys +import time + +def P(path): + """ + Give 'path' as a path relative to the abs_top_srcdir environment + variable. + """ + return os.path.join( + os.environ.get('abs_top_srcdir', "."), + path) + +def warn(msg): + """ + Print an warning message. + """ + print("WARNING: {}".format(msg), file=sys.stderr) + +def find_version(infile): + """ + Given an open file (or some other iterator of lines) holding a + configure.ac file, find the current version line. + """ + for line in infile: + m = re.search(r'AC_INIT$\[tor\],\s*\[([^\]]*)\]$', line) + if m: + return m.group(1) + + return None + +def update_version_in(infile, outfile, regex, versionline): + """ + Copy every line from infile to outfile. If any line matches 'regex', + replace it with 'versionline'. Return True if any line was changed; + false otherwise. + + 'versionline' is either a string -- in which case it is used literally, + or a function that receives the output of 'regex.match'. + """ + found = False + have_changed = False + for line in infile: + m = regex.match(line) + if m: + found = True + oldline = line + if type(versionline) == type(u""): + line = versionline + else: + line = versionline(m) + if not line.endswith("\n"): + line += "\n" + if oldline != line: + have_changed = True + outfile.write(line) + + if not found: + warn("didn't find any version line to replace in {}".format(infile.name)) + + return have_changed + +def replace_on_change(fname, change): + """ + If "change" is true, replace fname with fname.tmp. Otherwise, + delete fname.tmp. Log what we're doing to stderr. + """ + if not change: + print("No change in {}".format(fname)) + os.unlink(fname+".tmp") + else: + print("Updating {}".format(fname)) + os.rename(fname+".tmp", fname) + + +def update_file(fname, + regex, + versionline, + encoding="utf-8"): + """ + Replace any line matching 'regex' in 'fname' with 'versionline'. + Do not modify 'fname' if there are no changes made. Use the + provided encoding to read and write. + """ + with io.open(fname, "r", encoding=encoding) as f, \ + io.open(fname+".tmp", "w", encoding=encoding) as outf: + have_changed = update_version_in(f, outf, regex, versionline) + + replace_on_change(fname, have_changed) + +# Find out our version +with open("configure.ac") as f: + version = find_version(f) + +# If we have no version, we can't proceed. +if version == None: + print("No version found in configure.ac", file=sys.stderr()) + sys.exit(1) + +print("The version is {}".format(version)) + +today = time.strftime("%Y-%m-%d", time.gmtime()) + +# In configure.ac, we replace the definition of APPROX_RELEASE_DATE +# with "{today} for {version}", but only if the version does not match +# what is already there. +def replace_fn(m): + if m.group(1) != version: + # The version changed -- we change the date. + return u'AC_DEFINE(APPROX_RELEASE_DATE, ["{}"], # for {}'.format(today, version) + else: + # No changes. + return m.group(0) +update_file(P("configure.ac"), + re.compile(r'AC_DEFINE\(APPROX_RELEASE_DATE.* for (.*)'), + replace_fn) + +# In tor-mingw.nsi.in, we replace the definition of VERSION. +update_file(P("contrib/win32build/tor-mingw.nsi.in"), + re.compile(r'!define VERSION .*'), + u'!define VERSION "{}"'.format(version), + encoding="iso-8859-1") + +# In src/win32/orconfig.h, we replace the definition of VERSION. +update_file(P("src/win32/orconfig.h"), + re.compile(r'#define VERSION .*'), + u'#define VERSION "{}"'.format(version))

1 0

[tor/master] No longer exit for missing required protocolversions on an old consensus
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit 7da06e43da96f4253a756af546f27f03141b3784 Author: Nick Mathewson <nickm(a)torproject.org> Date: Sat Nov 24 20:40:48 2018 -0500 No longer exit for missing required protocolversions on an old consensus Specifically, if the consensus is older than the (estimted or measured) release date for this version of tor, we assume that the required versions may have changed in between that consensus and this release. Implements ticket 27735 and proposal 297. --- changes/prop297 | 7 +++++++ src/core/or/versions.c | 19 +++++++++++++++++++ src/core/or/versions.h | 2 ++ src/feature/nodelist/networkstatus.c | 5 ++++- 4 files changed, 32 insertions(+), 1 deletion(-) diff --git a/changes/prop297 b/changes/prop297 new file mode 100644 index 000000000..4f93b232d --- /dev/null +++ b/changes/prop297 @@ -0,0 +1,7 @@ + o Minor features (required protocols): + - Tor no longer exits if it is missing a required protocol, if the + consensus that requires the protocol predates the release date of the + version of Tor. This change prevents Tor releases from exiting because + of an old cached consensus, on the theory that a newer cached + consensus might not require the protocol. Implements proposal 297; + closes ticket 27735. diff --git a/src/core/or/versions.c b/src/core/or/versions.c index 5d4effcaf..d978935f4 100644 --- a/src/core/or/versions.c +++ b/src/core/or/versions.c @@ -16,6 +16,25 @@ #include "core/or/tor_version_st.h" +/** + * Return the approximate date when this release came out, or was + * scheduled to come out, according to the APPROX_RELEASE_DATE set in + * configure.ac + **/ +time_t +tor_get_approx_release_date(void) +{ + char tbuf[ISO_TIME_LEN+1]; + tor_snprintf(tbuf, sizeof(tbuf), + "%s 00:00:00", APPROX_RELEASE_DATE); + time_t result = 0; + int r = parse_iso_time(tbuf, &result); + if (BUG(r < 0)) { + result = 0; + } + return result; +} + /** Return VS_RECOMMENDED if <b>myversion</b> is contained in * <b>versionlist</b>. Else, return VS_EMPTY if versionlist has no * entries. Else, return VS_OLD if every member of diff --git a/src/core/or/versions.h b/src/core/or/versions.h index 4fc50a001..acd899891 100644 --- a/src/core/or/versions.h +++ b/src/core/or/versions.h @@ -26,6 +26,8 @@ typedef enum version_status_t { VS_UNKNOWN, /**< We have no idea. */ } version_status_t; +time_t tor_get_approx_release_date(void); + version_status_t tor_version_is_obsolete(const char *myversion, const char *versionlist); int tor_version_parse_platform(const char *platform, diff --git a/src/feature/nodelist/networkstatus.c b/src/feature/nodelist/networkstatus.c index f1def9afb..a25a539cd 100644 --- a/src/feature/nodelist/networkstatus.c +++ b/src/feature/nodelist/networkstatus.c @@ -2684,6 +2684,9 @@ networkstatus_check_required_protocols(const networkstatus_t *ns, const char *required, *recommended; char *missing = NULL; + const bool consensus_postdates_this_release = + ns->valid_after >= tor_get_approx_release_date(); + tor_assert(warning_out); if (client_mode) { @@ -2701,7 +2704,7 @@ networkstatus_check_required_protocols(const networkstatus_t *ns, "%s on the Tor network. The missing protocols are: %s", func, missing); tor_free(missing); - return 1; + return consensus_postdates_this_release ? 1 : 0; } if (! protover_all_supported(recommended, &missing)) {

1 0

[tor/release-0.3.5] Merge branch 'maint-0.3.3' into maint-0.3.4
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit 6506b1ee9f7d7daf31832c496ae9e30949fec995 Merge: c4f7953d8 c1f919158 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 11 09:41:05 2018 -0500 Merge branch 'maint-0.3.3' into maint-0.3.4 changes/ticket24803 | 5 + src/or/fallback_dirs.inc | 581 ++++++++++++++++++++++++++--------------------- 2 files changed, 333 insertions(+), 253 deletions(-)

1 0

[tor/release-0.3.5] Merge branch 'maint-0.2.9' into maint-0.3.3
by nickm＠torproject.org 11 Dec '18

11 Dec '18

commit c1f9191581eae2d04a4ffacf7a7066a16c9f5eae Merge: 00341d97f 4991b2931 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 11 09:41:04 2018 -0500 Merge branch 'maint-0.2.9' into maint-0.3.3 changes/ticket24803 | 5 + src/or/fallback_dirs.inc | 581 ++++++++++++++++++++++++++--------------------- 2 files changed, 333 insertions(+), 253 deletions(-)

1 0