- tor-commits - lists.torproject.org

[torspec/master] Document which descriptor items aren't allowed to take extra args
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 37def4e5024eda2f067c650f1b8421679fa532e1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 12 09:33:57 2016 -0500 Document which descriptor items aren't allowed to take extra args Closes #16227 --- dir-spec.txt | 32 ++++++++++++++++++++++++++++++++ rend-spec.txt | 15 ++++++++++++++- 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/dir-spec.txt b/dir-spec.txt index 1030c72..66b3421 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -253,6 +253,21 @@ "Once or more": These items MUST occur at least once in any instance of the document type, and MAY occur more. + For forward compatibility, each item MUST allow extra arguments at the + end of the line unless otherwise noted. So if an item's description below + is given as: + "thing" int int int NL + then implementations SHOULD accept this string as well: + "thing 5 9 11 13 16 12" NL + but not this string: + "thing 5" NL + and not this string: + "thing 5 10 thing" NL + . + + Whenever an item DOES NOT allow extra arguments, we will tag it with + "no extra arguments". + 1.3. Signing documents Every signable document below is signed in a similar manner, using a @@ -382,6 +397,7 @@ "-----END ED25519 CERT-----" NL [At most once, in second position in document.] + [No extra arguments] The certificate is a base64-encoded Ed25519 certificate (see cert-spec.txt) terminating =s removed. When this element is @@ -459,6 +475,7 @@ "onion-key" NL a public key in PEM format [Exactly once] + [No extra arguments] This key is used to encrypt CREATE cells for this OR. The key MUST be accepted for at least 1 week after any new key is published in a @@ -471,6 +488,8 @@ "onion-key-crosscert" NL a RSA signature in PEM format. [At most once, required when identity-25519 is present] + [No extra arguments] + This element contains an RSA signature, generated using the onion-key, of the following: @@ -505,6 +524,7 @@ "-----END ED25519 CERT-----" NL [At most once, required when identity-25519 is present] + [No extra arguments] A signature created with the ntor-onion-key, using the certificate format documented in cert-spec.txt, with type @@ -523,6 +543,7 @@ "signing-key" NL a public key in PEM format [Exactly once] + [No extra arguments] The OR's long-term RSA identity key. It MUST be 1024 bits. @@ -571,6 +592,7 @@ "router-signature" NL Signature NL [At end, exactly once] + [No extra arguments] The "SIGNATURE" object contains a signature of the PKCS1-padded hash of the entire server descriptor, taken from the beginning of the @@ -628,6 +650,7 @@ "caches-extra-info" NL [At most once.] + [No extra arguments] Present only if this router is a directory cache that provides extra-info documents. @@ -670,6 +693,7 @@ "allow-single-hop-exits" NL [At most once.] + [No extra arguments] Present only if the router allows single-hop circuits to make exit connections. Most Tor relays do not support this: this is @@ -1097,6 +1121,7 @@ "router-signature" NL Signature NL [At end, exactly once.] + [No extra arguments] A document signature as documented in section 1.3, using the initial item "extra-info" and the final item "router-signature", @@ -1178,6 +1203,7 @@ "dir-identity-key" NL a public key in PEM format [Exactly once.] + [No extra arguments] The long-term authority identity key for this authority. This key SHOULD be at least 2048 bits long; it MUST NOT be shorter than @@ -1199,6 +1225,7 @@ "dir-signing-key" NL a key in PEM format [Exactly once.] + [No extra arguments] The directory server's public signing key. This key MUST be at least 1024 bits, and MAY be longer. @@ -1206,6 +1233,7 @@ "dir-key-crosscert" NL CrossSignature NL [Exactly once.] + [No extra arguments] CrossSignature is a signature, made using the certificate's signing key, of the digest of the PKCS1-padded hash of the certificate's @@ -1220,6 +1248,7 @@ "dir-key-certification" NL Signature NL [At end, exactly once.] + [No extra arguments] A document signature as documented in section 1.3, using the initial item "dir-key-certificate-version" and the final item @@ -1304,6 +1333,7 @@ "onion-key" NL a public key in PEM format [Exactly once, at start] + [No extra arguments] The "onion-key" element as specified in section 2.1.1. @@ -1479,6 +1509,7 @@ "consensus-method" SP Integer NL [At most once for consensuses; does not occur in votes.] + [No extra arguments] See section 3.8.1 for details. @@ -1942,6 +1973,7 @@ consensus method 9 and above with the following: "directory-footer" NL + [No extra arguments] It contains two subsections, a bandwidths-weights line and a directory-signature. (Prior to conensus method 9, footers only contained diff --git a/rend-spec.txt b/rend-spec.txt index 33a9895..6442ecb 100644 --- a/rend-spec.txt +++ b/rend-spec.txt @@ -223,6 +223,7 @@ "rendezvous-service-descriptor" SP descriptor-id NL [At start, exactly once] + [No extra arguments] Indicates the beginning of the descriptor. "descriptor-id" is a periodically changing identifier of 160 bits formatted as 32 base32 @@ -267,7 +268,7 @@ "descriptor-cookie" is an optional secret password of 128 bits that is shared between the hidden service provider and its clients. If the descriptor-cookie is left out, the input to the hash function is 128 - bits shorter. + bits shorter. [No extra arguments] "replica" denotes the number of the replica. A service publishes multiple descriptors with different descriptor IDs in order to @@ -276,6 +277,7 @@ "version" SP version-number NL [Exactly once] + [No extra arguments] The version number of this descriptor's format. Version numbers are a positive integer. @@ -283,6 +285,7 @@ "permanent-key" NL a public key in PEM format [Exactly once] + [No extra arguments] The public key of the hidden service which is required to verify the "descriptor-id" and the "signature". @@ -290,6 +293,7 @@ "secret-id-part" SP secret-id-part NL [Exactly once] + [No extra arguments] The result of the following operation as explained above, formatted as 32 base32 chars. Using this secret id part, everyone can verify that @@ -307,6 +311,7 @@ "protocol-versions" SP version-string NL [Exactly once] + [No extra arguments] A comma-separated list of recognized and permitted version numbers for use in INTRODUCE cells; these versions are described in section @@ -315,6 +320,7 @@ "introduction-points" NL encrypted-string [At most once] + [No extra arguments] A list of introduction points. If the optional "descriptor-cookie" is used, this list is encrypted with AES in CTR mode with a random @@ -332,6 +338,7 @@ "introduction-point" SP identifier NL [At start, exactly once] + [No extra arguments] The identifier of this introduction point: the base32 encoded hash of this introduction point's identity key. @@ -339,12 +346,14 @@ "ip-address" SP ip4 NL [Exactly once] + [No extra arguments] The IP address of this introduction point. "onion-port" SP port NL [Exactly once] + [No extra arguments] The TCP port on which the introduction point is listening for incoming onion requests. @@ -352,6 +361,7 @@ "onion-key" NL a public key in PEM format [Exactly once] + [No extra arguments] The public key that can be used to encrypt messages to this introduction point. @@ -359,6 +369,7 @@ "service-key" NL a public key in PEM format [Exactly once] + [No extra arguments] The public key that can be used to encrypt messages to the hidden service. @@ -383,6 +394,7 @@ "signature" NL signature-string [At end, exactly once] + [No extra arguments] A signature of all fields above with the private key of the hidden service. @@ -994,6 +1006,7 @@ and stores the following data: "client-key" NL a public key in PEM format + [No extra arguments] 2.4. Client configuration

1 0

[torspec/master] One more singleton case.
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit f0b5c9881b83e8373528d0f7ebd8246bdc95a813 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 12 09:38:16 2016 -0500 One more singleton case. --- proposals/237-directory-servers-for-all.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/237-directory-servers-for-all.txt b/proposals/237-directory-servers-for-all.txt index 1ad05d1..04b79fa 100644 --- a/proposals/237-directory-servers-for-all.txt +++ b/proposals/237-directory-servers-for-all.txt @@ -49,6 +49,9 @@ Design: this end, we will introduce a new server descriptor line. "tunnelled-dir-server" NL + [At most once] + [No extra arguments] + The presence of this line indicates that the relay accepts tunnelled directory requests. For a relay that implements this

1 0

[torspec/master] Merge remote-tracking branch 'public/bug16227'
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 7ff38a15c5d6dbefffc256b8df897feade8489a5 Merge: e19176c f0b5c98 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 12 12:15:11 2016 -0500 Merge remote-tracking branch 'public/bug16227' dir-spec.txt | 32 +++++++++++++++++++++++++++ proposals/237-directory-servers-for-all.txt | 3 +++ rend-spec.txt | 15 ++++++++++++- 3 files changed, 49 insertions(+), 1 deletion(-)

1 0

[stem/master] Consensus routers attribute is a dict
by atagar＠torproject.org 12 Jan '16

12 Jan '16

commit f6314efb7b69990d7696088ca315a0a46b0dddcd Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Jan 11 08:52:55 2016 -0800 Consensus routers attribute is a dict Oops, misdocumented class attribute. Most folks don't use this due to the default DocumentHandler being ENTRIES. --- stem/descriptor/networkstatus.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/stem/descriptor/networkstatus.py b/stem/descriptor/networkstatus.py index fe7fb28..436131d 100644 --- a/stem/descriptor/networkstatus.py +++ b/stem/descriptor/networkstatus.py @@ -693,8 +693,8 @@ class NetworkStatusDocumentV3(NetworkStatusDocument): """ Version 3 network status document. This could be either a vote or consensus. - :var tuple routers: :class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3` - contained in the document + :var dict routers: fingerprint to :class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3` + mapping for relays contained in the document :var int version: **\*** document version :var str version_flavor: **\*** flavor associated with the document (such as 'microdesc') @@ -1421,8 +1421,8 @@ class BridgeNetworkStatusDocument(NetworkStatusDocument): Network status document containing bridges. This is only available through the metrics site. - :var tuple routers: :class:`~stem.descriptor.router_status_entry.RouterStatusEntryV2` - contained in the document + :var dict routers: fingerprint to :class:`~stem.descriptor.router_status_entry.RouterStatusEntryV3` + mapping for relays contained in the document :var datetime published: time when the document was published """

1 0

[stem/master] ADD_ONION accepts FQDN targets
by atagar＠torproject.org 12 Jan '16

12 Jan '16

commit f6e4f847ab68b6102891678b3041f9e0ff996ac9 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Jan 12 08:40:12 2016 -0800 ADD_ONION accepts FQDN targets On most systems the following requests balk, but after reformatting for some reason mine thinks its acceptable... >>> ADD_ONION NEW:BEST Port=4567,not_an_address:4567 250-ServiceID=4e5bpsy6e46onv3k 250-PrivateKey=RSA1024:[crypto blob] 250 OK Yawning and I discussed this on... https://trac.torproject.org/projects/tor/ticket/18029 Turns out targets can be a fully qualified domain name, so seems my getaddrinfo() is being particularly liberal with what it accepts. Still unsure if FQDN targets is desirable... https://trac.torproject.org/projects/tor/ticket/18037 ... but regardless, time to fix our tests now that we know what's up. --- test/integ/control/controller.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/integ/control/controller.py b/test/integ/control/controller.py index 7cf16ea..a755540 100644 --- a/test/integ/control/controller.py +++ b/test/integ/control/controller.py @@ -630,7 +630,7 @@ class TestController(unittest.TestCase): runner = test.runner.get_runner() with runner.get_tor_controller() as controller: - for ports in (4567890, [4567, 4567890], {4567: 'not_an_address:4567'}): + for ports in (4567890, [4567, 4567890], {4567: '-:4567'}): try: # try creating a service with an invalid port response = controller.create_ephemeral_hidden_service(ports)

1 0

[translation/tails-perl5lib_completed] Update translations for tails-perl5lib_completed
by translation＠torproject.org 12 Jan '16

12 Jan '16

commit 28ee89fdacc648b4383d7feb5b6b38a01956f5de Author: Translation commit bot <translation(a)torproject.org> Date: Tue Jan 12 16:16:00 2016 +0000 Update translations for tails-perl5lib_completed --- fr_CA.po | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/fr_CA.po b/fr_CA.po index 8574c63..f864dcb 100644 --- a/fr_CA.po +++ b/fr_CA.po @@ -3,23 +3,34 @@ # This file is distributed under the same license as the PACKAGE package. # # Translators: -# Alain-Olivier Breysse, 2014 +# Trans-fr, 2016 +# Trans-fr, 2014 msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: Tails developers <tails(a)boum.org>\n" -"POT-Creation-Date: 2013-11-28 11:56+0100\n" -"PO-Revision-Date: 2014-01-02 18:40+0000\n" -"Last-Translator: Alain-Olivier Breysse\n" -"Language-Team: French (Canada) (http://www.transifex.com/projects/p/torproject/language/fr_CA/)\n" +"POT-Creation-Date: 2016-01-11 15:40+0100\n" +"PO-Revision-Date: 2016-01-12 16:12+0000\n" +"Last-Translator: Trans-fr\n" +"Language-Team: French (Canada) (http://www.transifex.com/otf/torproject/language/fr_CA/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" "Language: fr_CA\n" "Plural-Forms: nplurals=2; plural=(n > 1);\n" -#: ../lib/Tails/RunningSystem.pm:139 +#: ../lib/Tails/RunningSystem.pm:159 +msgid "Error" +msgstr "Erreur" + +#: ../lib/Tails/RunningSystem.pm:161 msgid "" "The device Tails is running from cannot be found. Maybe you used the `toram'" " option?" msgstr "Le périphérique depuis lequel Tails tourne ne peut être trouvé. Peut-être avez-vous utilisé l’option « toram »?" + +#: ../lib/Tails/RunningSystem.pm:192 +msgid "" +"The drive Tails is running from cannot be found. Maybe you used the `toram' " +"option?" +msgstr "Le lecteur à partir duquel Tails fonctionne est introuvable. Peut-être avez-vous utilisé l'option « toram »?"

1 0

[translation/tails-perl5lib] Update translations for tails-perl5lib
by translation＠torproject.org 12 Jan '16

12 Jan '16

commit dc75e33107cfce059576166a4d60ff62ee2cb180 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Jan 12 16:15:56 2016 +0000 Update translations for tails-perl5lib --- fr_CA.po | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/fr_CA.po b/fr_CA.po index 2cf1493..f864dcb 100644 --- a/fr_CA.po +++ b/fr_CA.po @@ -3,14 +3,15 @@ # This file is distributed under the same license as the PACKAGE package. # # Translators: +# Trans-fr, 2016 # Trans-fr, 2014 msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: Tails developers <tails(a)boum.org>\n" "POT-Creation-Date: 2016-01-11 15:40+0100\n" -"PO-Revision-Date: 2016-01-12 08:13+0000\n" -"Last-Translator: carolyn <carolyn(a)anhalt.org>\n" +"PO-Revision-Date: 2016-01-12 16:12+0000\n" +"Last-Translator: Trans-fr\n" "Language-Team: French (Canada) (http://www.transifex.com/otf/torproject/language/fr_CA/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -20,7 +21,7 @@ msgstr "" #: ../lib/Tails/RunningSystem.pm:159 msgid "Error" -msgstr "" +msgstr "Erreur" #: ../lib/Tails/RunningSystem.pm:161 msgid "" @@ -32,4 +33,4 @@ msgstr "Le périphérique depuis lequel Tails tourne ne peut être trouvé. Peut msgid "" "The drive Tails is running from cannot be found. Maybe you used the `toram' " "option?" -msgstr "" +msgstr "Le lecteur à partir duquel Tails fonctionne est introuvable. Peut-être avez-vous utilisé l'option « toram »?"

1 0

[tor/master] Allow fallback directory selection to use day-old data
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 430181fad28fb3275a3771c78453a579681dfa5e Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Fri Dec 18 08:47:03 2015 +1100 Allow fallback directory selection to use day-old data Allow cached or outdated Onionoo data to be used to choose fallback directories, as long as it's less than a day old. Modify last modified date checks in preparation for Onionoo change --- changes/bug17887 | 6 ++++ scripts/maint/updateFallbackDirs.py | 54 +++++++++++++++++++++++------------ 2 files changed, 41 insertions(+), 19 deletions(-) diff --git a/changes/bug17887 b/changes/bug17887 new file mode 100644 index 0000000..423c298 --- /dev/null +++ b/changes/bug17887 @@ -0,0 +1,6 @@ + o Minor fixes (fallback directories): + - Allow cached or outdated Onionoo data to be used to choose + fallback directories, as long as it's less than a day old. + Modify last modified date checks in preparation for Onionoo change + #16907. Closes ticket #17887. Patch by "teor". Not in any released + version of tor. diff --git a/scripts/maint/updateFallbackDirs.py b/scripts/maint/updateFallbackDirs.py index 8fe234f..316962b 100755 --- a/scripts/maint/updateFallbackDirs.py +++ b/scripts/maint/updateFallbackDirs.py @@ -266,6 +266,17 @@ def load_json_from_file(json_file_name): ## OnionOO Functions +def datestr_to_datetime(datestr): + # Parse datetimes like: Fri, 02 Oct 2015 13:34:14 GMT + if datestr is not None: + dt = dateutil.parser.parse(datestr) + else: + # Never modified - use start of epoch + dt = datetime.datetime.utcfromtimestamp(0) + # strip any timezone out (in case they're supported in future) + dt = dt.replace(tzinfo=None) + return dt + def onionoo_fetch(what, **kwargs): params = kwargs params['type'] = 'relay' @@ -304,37 +315,42 @@ def onionoo_fetch(what, **kwargs): if last_mod_date is not None: request.add_header('If-modified-since', last_mod_date) - # Parse datetimes like: Fri, 02 Oct 2015 13:34:14 GMT - if last_mod_date is not None: - last_mod = dateutil.parser.parse(last_mod_date) - else: - # Never modified - use start of epoch - last_mod = datetime.datetime.utcfromtimestamp(0) - # strip any timezone out (in case they're supported in future) - last_mod = last_mod.replace(tzinfo=None) + # Parse last modified date + last_mod = datestr_to_datetime(last_mod_date) + + # Not Modified and still recent enough to be useful + # Onionoo / Globe used to use 6 hours, but we can afford a day + required_freshness = datetime.datetime.utcnow() + # strip any timezone out (to match dateutil.parser) + required_freshness = required_freshness.replace(tzinfo=None) + required_freshness -= datetime.timedelta(hours=24) + # Make the OnionOO request response_code = 0 try: response = urllib2.urlopen(request) response_code = response.getcode() except urllib2.HTTPError, error: response_code = error.code - # strip any timezone out (to match dateutil.parser) - six_hours_ago = datetime.datetime.utcnow() - six_hours_ago = six_hours_ago.replace(tzinfo=None) - six_hours_ago -= datetime.timedelta(hours=6) - # Not Modified and still recent enough to be useful (Globe uses 6 hours) - if response_code == 304: - if last_mod < six_hours_ago: - raise Exception("Outdated data from " + url + ": " - + str(error.code) + ": " + error.reason) - else: - pass + if response_code == 304: # not modified + pass else: raise Exception("Could not get " + url + ": " + str(error.code) + ": " + error.reason) if response_code == 200: # OK + last_mod = datestr_to_datetime(response.info().get('Last-Modified')) + + # Check for freshness + if last_mod < required_freshness: + if last_mod_date is not None: + date_message = "Outdated data: last updated " + last_mod_date + else: + date_message = "No data: never downloaded " + raise Exception(date_message + " from " + url) + + # Process the data + if response_code == 200: # OK response_json = load_possibly_compressed_response_json(response)

1 0

[tor/master] Merge remote-tracking branch 'teor/bug17887-v2'
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 7616922e8136d5d63661cfdfc4802608b16fdef7 Merge: 3074b83 430181f Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 12 10:47:54 2016 -0500 Merge remote-tracking branch 'teor/bug17887-v2' changes/bug17887 | 6 ++++ scripts/maint/updateFallbackDirs.py | 54 +++++++++++++++++++++++------------ 2 files changed, 41 insertions(+), 19 deletions(-)

1 0

[translation/tails-perl5lib_completed] Update translations for tails-perl5lib_completed
by translation＠torproject.org 12 Jan '16

12 Jan '16

commit d0b6ee86df6e55a579b112f952ff402313db49ce Author: Translation commit bot <translation(a)torproject.org> Date: Tue Jan 12 15:46:19 2016 +0000 Update translations for tails-perl5lib_completed --- bg.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bg.po b/bg.po index 015d098..0436ae0 100644 --- a/bg.po +++ b/bg.po @@ -9,7 +9,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: Tails developers <tails(a)boum.org>\n" "POT-Creation-Date: 2016-01-11 15:40+0100\n" -"PO-Revision-Date: 2016-01-12 13:12+0000\n" +"PO-Revision-Date: 2016-01-12 15:38+0000\n" "Last-Translator: Tsvetan Nikolov <slakware(a)gmail.com>\n" "Language-Team: Bulgarian (http://www.transifex.com/otf/torproject/language/bg/)\n" "MIME-Version: 1.0\n"

1 0