January 2016 - tor-commits - lists.torproject.org

[tor/master] Tweak changes file to mention bug number
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 007eb3f8cbfbc696c301ef568a04a012fb847e10 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 12 10:12:08 2016 -0500 Tweak changes file to mention bug number --- changes/log_heartbeat_test | 1 + 1 file changed, 1 insertion(+) diff --git a/changes/log_heartbeat_test b/changes/log_heartbeat_test index f0d143b..7db97ed 100644 --- a/changes/log_heartbeat_test +++ b/changes/log_heartbeat_test @@ -3,3 +3,4 @@ with non-integer offsets. Instead of comparing the end of the time string against a constant, compare it to the output of format_local_iso_time when given the correct input. + Fixes bug 18039.

1 0

[tor/master] Merge remote-tracking branch 'fergus-dall/log_heartbeat_test'
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 91b3fa3636eecc15688263e32133e97fcce36c93 Merge: 1c139f6 9e5a6f0 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 12 10:11:45 2016 -0500 Merge remote-tracking branch 'fergus-dall/log_heartbeat_test' changes/log_heartbeat_test | 5 +++++ src/test/test_status.c | 8 +++++--- 2 files changed, 10 insertions(+), 3 deletions(-)

1 0

[tor/master] Add another safe_str_client to fix bug 17419
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 3074b8365fe5c76e6fc0b9597a99ab2671c7e939 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 12 10:42:01 2016 -0500 Add another safe_str_client to fix bug 17419 --- changes/bug17419 | 4 ++++ src/or/buffers.c | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/changes/bug17419 b/changes/bug17419 new file mode 100644 index 0000000..8ad8edd --- /dev/null +++ b/changes/bug17419 @@ -0,0 +1,4 @@ + + o Minor bugfixes: + - When logging a malformed hostname received through socks4, scrub it + if SafeLogging says we should. Fixes bug 17419; bugfix on 0.1.1.16-rc. diff --git a/src/or/buffers.c b/src/or/buffers.c index 4696bec..cdb499b 100644 --- a/src/or/buffers.c +++ b/src/or/buffers.c @@ -1945,7 +1945,7 @@ parse_socks(const char *data, size_t datalen, socks_request_t *req, log_warn(LD_PROTOCOL, "Your application (using socks4 to port %d) gave Tor " "a malformed hostname: %s. Rejecting the connection.", - req->port, escaped(req->address)); + req->port, escaped_safe_str_client(req->address)); return -1; } if (authend != authstart) {

1 0

[torspec/master] prop250: Update to the latest version from asn/dgoulet
by dgoulet＠torproject.org 12 Jan '16

12 Jan '16

commit e19176c61588eb232ea49a65135844af3c837289 Author: David Goulet <dgoulet(a)ev0ke.net> Date: Tue Jan 12 16:02:47 2016 +0100 prop250: Update to the latest version from asn/dgoulet This version matches the implementation submitted for review. Signed-off-by: David Goulet <dgoulet(a)ev0ke.net> --- proposals/250-commit-reveal-consensus.txt | 304 +++++++++++++++++------------ 1 file changed, 176 insertions(+), 128 deletions(-) diff --git a/proposals/250-commit-reveal-consensus.txt b/proposals/250-commit-reveal-consensus.txt index 526372d..20f5036 100644 --- a/proposals/250-commit-reveal-consensus.txt +++ b/proposals/250-commit-reveal-consensus.txt @@ -12,10 +12,12 @@ Supersedes: 225 1.1. Motivation 1.2. Previous work 2. Overview - 2.1. Ten thousand feet view - 2.2. Commit & Reveal - 2.3. Consensus [CONS] + 2.1. Introduction to our commit-and-reveal protocol + 2.2. Ten thousand feet view of the protocol + 2.3. How we use the consensus [CONS] + 2.3.1. Inserting Shared Random Values in the consensus 2.4. Persistent State of the Protocol [STATE] + 2.5. Protocol Illustration 3. Protocol 3.1 Commitment Phase [COMMITMENTPHASE] 3.1.1. Voting During Commitment Phase @@ -27,26 +29,26 @@ Supersedes: 225 3.3.1. Shared Randomness Calculation [SRCALC] 3.4. Bootstrapping Procedure 3.5. Rebooting Directory Authorities [REBOOT] - 3.6. Shared Randomness Disaster Recovery [SRDISASTER] 4. Specification [SPEC] - 4.1 Voting + 4.1. Voting 4.1.1. Computing commitments and reveals [COMMITREVEAL] 4.1.2. Validating commitments and reveals [VALIDATEVALUES] 4.1.4. Encoding commit/reveal values in votes [COMMITVOTE] 4.1.5. Shared Random Value [SRVOTE] 4.1.6. Including the ed25519 identity key in votes [SRKEY] - 4.2. Persistent state format [STATEFORMAT] - 4.3. Shared Random Value in Consensus [SRCONSENSUS] + 4.2. Encoding Shared Random Values in the consensus [SRCONSENSUS] + 4.3. Persistent state format [STATEFORMAT] 5. Security Analysis 5.1. Security of commit-and-reveal and future directions 5.2. Predicting the shared random value during reveal phase - 5.3. Partition Attack - 5.3.1 During commit phase - 5.3.2 During reveal phase + 5.3. Partition attacks + 5.3.1. Partition attacks during commit phase + 5.3.2. Partition attacks during reveal phase 6. Discussion 6.1. Why the added complexity from proposal 225? 6.2. Why do you do a commit-and-reveal protocol in 24 rounds? - 6.3. Why can't we recover if we fail to do a consensus at 00:00UTC? + 6.3. Why can't we recover if the 00:00UTC consensus fails? + 7. Acknowledgements 1. Introduction @@ -88,7 +90,19 @@ Supersedes: 225 The proposal also specifies how the final shared random value is embedded in consensus documents so that clients who need it can get it. -2.1. Ten thousand feet view +2.1. Introduction to our commit-and-reveal protocol + + Every day, before voting for the consensus at 00:00UTC each authority + generates a new random value and keeps it for the whole day. The authority + cryptographically hashes the random value and calls the output its + "commitment" value. The original random value is called the "reveal" value. + + The idea is that given a reveal value you can cryptographically confirm that + it corresponds to a given commitment value (by hashing it). However given a + commitment value you should not be able to derive the underlying reveal + value. The construction of these values is specified in section [COMMITREVEAL]. + +2.1. Ten thousand feet view of the protocol Our commit-and-reveal protocol aims to produce a fresh shared random value everyday at 00:00UTC. The final fresh random value is embedded in the @@ -101,7 +115,7 @@ Supersedes: 225 Commit phase: Starting at 00:00UTC and for a period of 12 hours, authorities every - hour send their commitments in their votes. They also include any + hour include their commitment in their votes. They also include any received commitments from other authorities, if available. Reveal phase: @@ -117,51 +131,67 @@ Supersedes: 225 At 00:00UTC, the shared random value is computed from the agreed revealed values and added to the consensus. - This concludes the commit-and-reveal procedure at 00:00UTC everyday. + This concludes the commit-and-reveal protocol at 00:00UTC everyday. -2.2. Commit & Reveal +2.3. How we use the consensus [CONS] - Our commit-and-reveal protocol aims to produce a fresh shared random value - everyday at 00:00UTC. + The produced shared random values needs to be readily available to + clients. For this reason we include them in the consensus documents. - In the beginning of that time period, each authority generates a new random - value and keeps it for a whole day. The authority cryptographically hashes - the random value and calls the output its "commitment" value. The original - random value is called the "reveal" value. + Every hour the consensus documents need to include the shared random value + of the day, as well as the shared random value of the previous day. That's + because either of these values might be needed at a given time for a Tor + client to access a hidden service according to section [TIME-OVERLAP] of + proposal 224. This means that both of these two values need to be included + in votes as well. - The idea is that given a reveal value you can cryptographically confirm that - it corresponds to a given commitment value (by hashing it). However given a - commitment value you should not be able to derive the underlying reveal - value. The construction of these values is specified in section [COMMITREVEAL]. + Hence, consensuses need to include: -2.3. Consensus [CONS] + (a) The shared random value of the current time period. + (b) The shared random value of the previous time period. - The produced shared random value needs to be readily available to - clients. For this reason we include it in the consensus documents. + For this, a new SR consensus method will be needed to indicate which + authorities support this new protocol. - Furthermore, every hour the consensus documents need to include the shared - random value of the day, as well as the shared random value of the previous - day. That's because either of these values might be needed at a given time - for a Tor client to access a hidden service according to section - [TIME-OVERLAP] of proposal 224. These means that these two values also need - to be included in votes and in the authority state as well. +2.3.1. Inserting Shared Random Values in the consensus - Hence, the consensuses include: + After voting happens, we need to be careful on how we pick which shared + random values (SRV) to put in the consensus, to avoid breaking the consensus + because of authorities having different views of the commit-and-reveal + protocol (because maybe they missed some rounds of the protocol). - (a) The shared random value of the current time period. This is derived - from the reveal values sent by the authorities during the voting - session. + For this reason, authorities look at the received votes before creating a + consensus and employ the following logic: - (b) The shared random value of the previous time period. + - First of all, they make sure that the agreed upon consensus method is + above the SR consensus method. + + - Authorities include an SRV in the consensus if and only if the SRV has + been voted by at least the majority of authorities. + + - For the consensus at 00:00UTC, authorities include an SRV in the consensus + if and only if the SRV has been voted by at least AuthDirNumAgreements + authorities (where AuthDirNumAgreements is a newly introduced consensus + parameter). + + Authorities include in the consensus the most popular SRV that also + satisfies the above constraints. Otherwise, no SRV should be included. + + The above logic is used to make it harder to break the consensus by natural + partioning causes. - For this, a new consensus method will be needed to indicate which authorities - support this new protocol. + We use the AuthDirNumAgreements consensus parameter to enforce that a + _supermajority_ of dirauths supports the SR protocol during SRV creation, so + that even if a few of those dirauths drop offline in the middle of the run + the SR protocol does not get disturbed. We go to extra lengths to ensure + this because changing SRVs in the middle of the day has terrible + reachability consequences for hidden service clients. 2.4. Persistent State of the Protocol [STATE] A directory authority needs to keep a persistent state on disk of the on - going protocol run. This allows an authority to join the protocol in the - case of a reboot. + going protocol run. This allows an authority to join the protocol seamlessly + in the case of a reboot. During the commitment phase, it is populated with the commitments of all authorities. Then during the reveal phase, the reveal values are also @@ -171,6 +201,28 @@ Supersedes: 225 previous time period must also be present in the state at all times if they are available. +2.5. Protocol Illustration + + An illustration for better understanding the protocol can be found here: + https://people.torproject.org/~asn/hs_notes/shared_rand.jpg + + It reads left-to-right. + + The illustration displays what the authorities (A_1, A_2, A_3) put in their + votes. A chain 'A_1 -> c_1 -> r_1' denotes that authority A_1 committed to + the value c_1 which corresponds to the reveal value r_1. + + The illustration depicts only a few rounds of the whole protocol. It starts + with the first three rounds of the commit phase, then it jumps to the last + round of the commit phase. It continues with the first two rounds of the + reveal phase and then it jumps to the final round of the protocol run. It + finally shows the first round of the commit phase of the next protocol run + (00:00UTC) where the final Shared Random Value is computed. In our fictional + example, the SRV was computed with 3 authority contributions and its value + is "a56fg39h". + + We advice you to revisit this after you have read the whole document. + 3. Protocol In this section we give a detailed specification of the protocol. We @@ -195,16 +247,16 @@ Supersedes: 225 During the commit phase, each authority includes in its votes: - The commitment value for this protocol run. - - Any commitments received from other authorities. + - Any authoritative commitments received from other authorities. - The two previous shared random values produced by the protocol (if any). The commit phase lasts for 12 hours, so authorities have multiple chances to commit their values. An authority MUST NOT commit a second value during a subsequent round of the commit phase. - if an authority publishes a second commitment value in the same - commit phase, only the first commitment should be taken in account - by other authorities. Any subsequent commitments MUST be ignored. + If an authority publishes a second commitment value in the same commit + phase, only the first commitment should be taken in account by other + authorities. Any subsequent commitments MUST be ignored. 3.1.2. Persistent State During Commitment Phase [STATECOMMIT] @@ -255,16 +307,12 @@ Supersedes: 225 Authorities calculate the shared random value using the reveal values in their state as specified in subsection [SRCALC]. - If the shared random value contains reveal contributions by less than 3 - directory authorities, it MUST NOT be created. Instead, the old shared - random value should be used as specified in section [SRDISASTER]. - Authorities at 00:00UTC start including this new shared random value in their votes, replacing the one from two protocol runs ago. Authorities also start including this new shared random value in the consensus as well. - Apart from that, authorities proceed voting normally as they would in the - first round of the commitment phase (section [COMMITMENTPHASE]). + Apart from that, authorities at 00:00UTC proceed voting normally as they + would in the first round of the commitment phase (section [COMMITMENTPHASE]). 3.3.1. Shared Randomness Calculation [SRCALC] @@ -322,31 +370,20 @@ Supersedes: 225 have to reboot in the middle (assuming that their persistent state file is kept). A suggested way to structure the persistent state is found at [STATEFORMAT]. -3.6. Shared Randomness Disaster Recovery [SRDISASTER] - - If the consensus at 00:00UTC fails to be created, then there will be no fresh - shared random value for the day. - - In this case, and assuming there is a previous shared random value, directory - authorities should use the following construction as the shared random value - of the day: - - SRV = HMAC(previous_SRV, "shared-random-disaster") - - where "previous_SRV" is the previous shared random value. - - Clients should keep on using this shared random values. - 4. Specification [SPEC] -4.1 Voting +4.1. Voting - This section describes how commitments, reveals and SR values are encoded - in votes. We describe how to encode both the authority's own - commits/reveals and also the commits/reveals received from the other - authorities. Commits and reveals share the same line, but reveals are + This section describes how commitments, reveals and SR values are encoded in + votes. We describe how to encode both the authority's own + commitments/reveals and also the commitments/reveals received from the other + authorities. Commitments and reveals share the same line, but reveals are optional. + Participating authorities need to include the line: + "shared-rand-participate" + in their votes to announce that they take part in the protocol. + 4.1.1. Computing commitments and reveals [COMMITREVEAL] A directory authority that wants to participate in this protocol needs to @@ -356,29 +393,28 @@ Supersedes: 225 The value REVEAL is computed as follows: - REVEAL = base32-encode( TIMESTAMP || RN ) + REVEAL = base32-encode( TIMESTAMP || H(RN) ) - where RN is a 256-bit random value. + where RN is the sha256 hashed value of a 256-bit random value. We hash + the random value to avoid exposing raw bytes from our PRNG to the network + (see [RANDOM-REFS]). The value COMMIT is computed as follows: - COMMIT = base32-encode( H(RN) || TIMESTAMP ) + COMMIT = base32-encode( H(H(RN)) || TIMESTAMP ) - where H is the hashing algorithm "sha256", and RN is the random value from - REVEAL. + where RN is the random value from REVEAL. 4.1.2. Validating commitments and reveals [VALIDATEVALUES] - Authorities need to verify that the TIMESTAMP field of commitments and - reveals corresponds to the current protocol run. If an outdated COMMIT or - REVEAL value is found, it should be ignored. Corresponding COMMIT and REVEAL - values MUST have the same timestamp value. - Given a COMMIT message and a REVEAL message it should be possible to verify that they indeed correspond. To do so, the client extracts the random value - RN from the REVEAL message, hashes it, and compares it with the H(RN) from - the COMMIT message. We say that the COMMIT and REVEAL messages correspond, if - the comparison was successful. + H(RN) from the REVEAL message, hashes it, and compares it with the H(H(RN)) + from the COMMIT message. We say that the COMMIT and REVEAL messages + correspond, if the comparison was successful. + + Pariticipants MUST also check that corresponding COMMIT and REVEAL values + have the same timestamp value. Authorities should ignore reveal values during the Reveal Phase that don't correspond to commit values published during the Commitment Phase. @@ -389,35 +425,31 @@ Supersedes: 225 seen from the other authorities. To do so, it includes the following in its votes: - "shared-rand-commitment" SP identity SP algname SP COMMIT [SP REVEAL] NL + "shared-rand-commit" SP IDENTITY SP ALGNAME SP COMMIT [SP REVEAL] NL - where "identity" is the authority's identity key fingerprint and COMMIT is - the received encoded commitment [COMMITREVEAL]. Authorities can also - optionally include the encoded reveal value REVEAL. There MUST be only one - line per authority else the vote is considered invalid. Finally, the - "algname" is the hash algorithm that should be used to compute COMMIT and + where IDENTITY is the authority's ed25519 identity key [SRKEY] and COMMIT is + the encoded commit [COMMITREVEAL]. Authorities during the reveal phase can + also optionally include an encoded reveal value REVEAL. There MUST be only + one line per authority else the vote is considered invalid. Finally, the + ALGNAME is the hash algorithm that should be used to compute COMMIT and REVEAL which is "sha256" for version 1. 4.1.5. Shared Random Value [SRVOTE] - Authorities include a shared random value in their votes using the following - encoding for the previous and current value respectively: - - "shared-rand-previous-value" SP status SP value NL - "shared-rand-current-value" SP status SP value NL + Authorities include a shared random value (SRV) in their votes using the + following encoding for the previous and current value respectively: - where "value" is the actual shared random value. It's normally computed as - specified in the section [SRCALC]. + "shared-rand-previous-value" SP NUM_REVEALS SP VALUE NL + "shared-rand-current-value" SP NUM_REVEALS SP VALUE NL - "status" denotes how fresh the value is. If that value was produced as - specified in section [SRCALC] then status is "fresh". If the value was - produced from a failed protocol run (as specified in [SRDISASTER]) then - status is "non-fresh". + where VALUE is the actual shared random value encoded in hex (computed as + specified in section [SRCALC]. NUM_REVEALS is the number of reveal values + used to generate this SRV. To maintain consistent ordering, the shared random values of the previous period should be listed before the values of the current period. -4.1.6. Including the ed25519 shared randomness key in votes [SRKEY] +4.1.6. Including the ed25519 identity key in votes [SRKEY] We don't want to use the legacy RSA keys as part of the shared randomness protocol since they are going to be deprecated soon. @@ -439,7 +471,12 @@ Supersedes: 225 includes the ed25519 master key in it certifying it under the RSA master key (since it's included in a vote which is signed by the RSA key). -4.2. Persistent state format [STATEFORMAT] +4.2. Encoding Shared Random Values in the consensus [SRCONSENSUS] + + Authorities insert the two active shared random values in the consensus + following the same encoding format as in [SRVOTE]. + +4.3. Persistent state format [STATEFORMAT] As a way to keep ground truth state in this protocol, an authority MUST keep a persistent state of the protocol. The next sub-section suggest a @@ -468,7 +505,7 @@ Supersedes: 225 The following details the commitment and reveal section. They are encoded the same as in the vote. This makes it easier for implementation purposes. - "Commitment" SP identity SP algname SP commit [SP reveal] NL + "Commit" SP identity SP algname SP commit [SP reveal] NL [Exactly once per authority] @@ -478,26 +515,20 @@ Supersedes: 225 Finally is the shared random value section. - "SharedRandPreviousValue" SP status SP value NL + "SharedRandPreviousValue" SP num_reveals SP value NL [At most once] This is the previous shared random value agreed on at the previous period. The fields are the same as in section [SRVOTE]. - "SharedRandCurrentValue" SP status SP value NL + "SharedRandCurrentValue" SP num_reveals SP value NL [At most once] This is the latest shared random value. The fields are the same as in section [SRVOTE]. - -4.3. Shared Random Value in Consensus [SRCONSENSUS] - - Authorities insert the two shared random values in the consensus following - the same encoding format as in [SRVOTE]. - 5. Security Analysis 5.1. Security of commit-and-reveal and future directions @@ -513,6 +544,9 @@ Supersedes: 225 crypto and more complex protocols so this seems like an acceptable solution for now. + For alternative approaches on collaborative random number generation also + see the discussion at [RNGMESSAGING]. + 5.2. Predicting the shared random value during reveal phase The reveal phase lasts 12 hours, and most authorities will send their @@ -527,7 +561,7 @@ Supersedes: 225 Any other protocols using the shared random value from this system should be aware of this property. -5.3 Partition attack +5.3. Partition attacks This design is not immune to certain partition attacks. We believe they don't offer much gain to an attacker as they are very easy to detect and @@ -537,7 +571,7 @@ Supersedes: 225 attacks. Nevertheless, this section describes all possible partition attack and how to detect them. -5.3.1 During commit phase +5.3.1. Partition attacks during commit phase A malicious directory authority could send only its commit to one single authority which results in that authority having an extra commit value for @@ -546,15 +580,14 @@ Supersedes: 225 the attacker, using this attack, could remove a single directory authority from the consensus decision at 24:00 when the SRV is computed. - An attacker could also partition the authorities by sending two - different commitment values to different authorities during the - commit phase. + An attacker could also partition the authorities by sending two different + commitment values to different authorities during the commit phase. All of the above is fairly easy to detect. Commitment values in the vote coming from an authority should NEVER be different between authorities. If so, this means an attack is ongoing or very bad bug (highly unlikely). -5.3.2 During reveal phase +5.3.2. Partition attacks during reveal phase Let's consider Alice, a malicious directory authority. Alice could wait until the last reveal round, and reveal its value to half of the @@ -609,13 +642,28 @@ Supersedes: 225 big problem. Also, this way we give more chances for a failing dirauth to recover and rejoin the protocol. -6.3. Why can't we recover if we fail to do a consensus at 00:00UTC? +6.3. Why can't we recover if the 00:00UTC consensus fails? + + If the 00:00UTC consensus fails, there will be no shared random value for + the whole day. In theory, we could recover by calculating the shared + randomness of the day at 01:00UTC instead. However, the engineering issues + with adding such recovery logic are too great. For example, it's not easy + for an authority who just booted to learn whether a specific consensus + failed to be created. + +7. Acknowledgements + + Thanks to everyone who has contributed to this design with feedback and + discussion. + + Thanks go to arma, ioerror, kernelcorn, nickm, s7r, Sebastian, teor, weasel + and everyone else! + +References: - Section [SRDISASTER] specifies that if the 00:00UTC consensus fails to be - created, we simply hash the random value of the previous day and use it as - the new shared random value. This changes the daily value but fails to make - it fresh, which is not optimal. +[RANDOM-REFS]: + http://projectbullrun.org/dual-ec/ext-rand.html + https://lists.torproject.org/pipermail/tor-dev/2015-November/009954.html - Theoretically, we could recover by calculating the shared randomness of the - day at 01:00UTC instead. However, adding such fallback logic would - complicate the protocol even further, so we have not yet considered it. +[RNGMESSAGING]: + https://moderncrypto.org/mail-archive/messaging/2015/002032.html

1 0

[tor/master] Fix various issues with fallback directory data handling
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit da968e558e89b3ee9201d6c9cdd339ef48979071 Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Tue Jan 12 14:15:22 2016 +1100 Fix various issues with fallback directory data handling * support maximum history age in _avg_generic_history() * fix division-by-zero trap in _avg_generic_history() * skip missing (i.e. null/None) intervals in _avg_generic_history() * Python timedelta.total_seconds() function not available in 2.6; replace with equivalent expression * set DEBUG logging level to make relay exclusion reasons visible * move CUTOFF_GUARD test to end in order to expose more exclusion reasons Patch by "starlight", merge modifications by "teor". --- changes/bug18035 | 6 ++++++ scripts/maint/updateFallbackDirs.py | 28 ++++++++++++++++++++-------- 2 files changed, 26 insertions(+), 8 deletions(-) diff --git a/changes/bug18035 b/changes/bug18035 new file mode 100644 index 0000000..31889f5 --- /dev/null +++ b/changes/bug18035 @@ -0,0 +1,6 @@ + o Minor fixes (fallback directories): + - Work around an issue where OnionOO supplies the entire history, + rather than the requested 120 days. Activate debug logging by + default. Fix other minor calculation and compatibility issues. + Closes ticket #18035. Patch by "starlight", merge fixes by + "teor". Not in any released version of tor. diff --git a/scripts/maint/updateFallbackDirs.py b/scripts/maint/updateFallbackDirs.py index 8fe234f..29be433 100755 --- a/scripts/maint/updateFallbackDirs.py +++ b/scripts/maint/updateFallbackDirs.py @@ -27,7 +27,7 @@ import dateutil.parser #from bson import json_util import logging -logging.basicConfig(level=logging.INFO) +logging.basicConfig(level=logging.DEBUG) ## Top-Level Configuration @@ -579,9 +579,15 @@ class Candidate(object): %(p, which)) for v in reversed(h['values']): if (this_ts <= newest): + agt1 = now - this_ts + agt2 = interval + agetmp1 = (agt1.microseconds + (agt1.seconds + agt1.days * 24 * 3600) + * 10**6) / 10**6 + agetmp2 = (agt2.microseconds + (agt2.seconds + agt2.days * 24 * 3600) + * 10**6) / 10**6 generic_history.append( - { 'age': (now - this_ts).total_seconds(), - 'length': interval.total_seconds(), + { 'age': agetmp1, + 'length': agetmp2, 'value': v }) newest = this_ts @@ -599,6 +605,8 @@ class Candidate(object): def _avg_generic_history(generic_history): a = [] for i in generic_history: + if i['age'] > (ADDRESS_AND_PORT_STABLE_DAYS * 24 * 3600): + continue if (i['length'] is not None and i['age'] is not None and i['value'] is not None): @@ -608,7 +616,11 @@ class Candidate(object): sv = math.fsum(map(lambda x: x[0], a)) sw = math.fsum(map(lambda x: x[1], a)) - return sv/sw + if sw == 0.0: + svw = 0.0 + else: + svw = sv/sw + return svw def _add_generic_history(self, history): periods = r['read_history'].keys() @@ -659,10 +671,6 @@ class Candidate(object): logging.debug('%s not a candidate: running avg too low (%lf)', self._fpr, self._running) return False - if self._guard < CUTOFF_GUARD: - logging.debug('%s not a candidate: guard avg too low (%lf)', - self._fpr, self._guard) - return False if self._v2dir < CUTOFF_V2DIR: logging.debug('%s not a candidate: v2dir avg too low (%lf)', self._fpr, self._v2dir) @@ -675,6 +683,10 @@ class Candidate(object): if (not self._data.has_key('recommended_version') or not self._data['recommended_version']): return False + if self._guard < CUTOFF_GUARD: + logging.debug('%s not a candidate: guard avg too low (%lf)', + self._fpr, self._guard) + return False return True def is_in_whitelist(self, relaylist):

1 0

[tor/master] Don't limit fallback directory candidates to 500
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 1c139f60d201a574fc30d8f53786b6727bef762c Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Tue Jan 12 14:50:28 2016 +1100 Don't limit fallback directory candidates to 500 Only impose the limit when selecting fallback directories for inclusion in tor. --- scripts/maint/updateFallbackDirs.py | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/scripts/maint/updateFallbackDirs.py b/scripts/maint/updateFallbackDirs.py index 29be433..20bfd18 100755 --- a/scripts/maint/updateFallbackDirs.py +++ b/scripts/maint/updateFallbackDirs.py @@ -91,7 +91,7 @@ PERMITTED_BADEXIT = .00 FALLBACK_PROPORTION_OF_GUARDS = None if OUTPUT_CANDIDATES else 0.2 # Limit the number of fallbacks (eliminating lowest by weight) -MAX_FALLBACK_COUNT = 500 +MAX_FALLBACK_COUNT = None if OUTPUT_CANDIDATES else 500 # Emit a C #error if the number of fallbacks is below MIN_FALLBACK_COUNT = 100 @@ -1010,7 +1010,8 @@ class CandidateList(dict): # starting with the lowest-weighted fallbacks # total_weight should be recalculated after calling this def exclude_excess_fallbacks(self): - self.fallbacks = self.fallbacks[:MAX_FALLBACK_COUNT] + if MAX_FALLBACK_COUNT is not None: + self.fallbacks = self.fallbacks[:MAX_FALLBACK_COUNT] # Clamp the weight of all fallbacks to MAX_WEIGHT_FRACTION * total_weight # fallbacks are kept sorted, but since excessive weights are reduced to @@ -1081,15 +1082,15 @@ class CandidateList(dict): else: fallback_proportion = ' (%d * %f)'%(guard_count, FALLBACK_PROPORTION_OF_GUARDS) - s += 'Final Count: %d (Eligible %d, Usable %d, Target %d%s, '%( + s += 'Final Count: %d (Eligible %d, Usable %d, Target %d%s'%( min(max_count, fallback_count), eligible_count, fallback_count, target_count, fallback_proportion) - s += 'Clamped to %d)'%( - MAX_FALLBACK_COUNT) - s += '\n' + if MAX_FALLBACK_COUNT is not None: + s += ', Clamped to %d'%(MAX_FALLBACK_COUNT) + s += ')\n' if fallback_count < MIN_FALLBACK_COUNT: s += '*/' s += '\n' @@ -1159,13 +1160,16 @@ def list_fallbacks(): guard_count = candidates.count_guards() if FALLBACK_PROPORTION_OF_GUARDS is None: - target_count = MAX_FALLBACK_COUNT + target_count = guard_count else: target_count = int(guard_count * FALLBACK_PROPORTION_OF_GUARDS) # the maximum number of fallbacks is the least of: # - the target fallback count (FALLBACK_PROPORTION_OF_GUARDS * guard count) # - the maximum fallback count (MAX_FALLBACK_COUNT) - max_count = min(target_count, MAX_FALLBACK_COUNT) + if MAX_FALLBACK_COUNT is None: + max_count = guard_count + else: + max_count = min(target_count, MAX_FALLBACK_COUNT) candidates.compute_fallbacks()

1 0

[torspec/master] Apply updated proposal 263 from tor-dev
by nickm＠torproject.org 12 Jan '16

12 Jan '16

commit 1f087bf0141b4d34d2f19e170a9f2a006d8bb1a7 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 12 10:08:19 2016 -0500 Apply updated proposal 263 from tor-dev --- proposals/263-ntru-for-pq-handshake.txt | 180 ++++++++++++++++--------------- 1 file changed, 94 insertions(+), 86 deletions(-) diff --git a/proposals/263-ntru-for-pq-handshake.txt b/proposals/263-ntru-for-pq-handshake.txt index 0216658..e1bd700 100644 --- a/proposals/263-ntru-for-pq-handshake.txt +++ b/proposals/263-ntru-for-pq-handshake.txt @@ -1,7 +1,8 @@ Filename: 263-ntru-for-pq-handshake.txt -Title: Request to change key exchange protocol for handshake +Title: Request to change key exchange protocol for handshake v1.1 Author: John SCHANCK, William WHYTE and Zhenfei ZHANG Created: 29 Aug 2015 +Updated: 9 Jan 2016 Status: Open 1. Introduction @@ -12,7 +13,7 @@ Status: Open 0x0002 ntor -- the ntor+curve25519+sha256 handshake; Request for a new (set of) handshake type: - 0x010X ntor+qsh -- the hybrid of ntor+curve25519+sha256 handshake + 0x010X ntor+qsh -- the hybrid of ntor+curve25519+sha3 handshake and a quantum-safe key encapsulation mechanism where @@ -20,7 +21,7 @@ Status: Open assigned. 0X0101 ntor+ntru -- the quantum safe KEM is based on NTRUEncrypt, with - parameter ntrueess439ep1 + parameter ntrueess443ep1 0X0102 ntor+ntru -- the quantum safe KEM is based on NTRUEncrypt, with parameter ntrueess743ep1 @@ -31,29 +32,35 @@ Status: Open DEPENDENCY: Proposal 249: Allow CREATE cells with >505 bytes of handshake data - 1.1 Motivation: Quantum resistant key agreement + 1.1 Motivation: Quantum-safe forward-secure key agreement - We are trying to add quantum resistance to the key agreement in Tor - handshake. Current approaches for handling key agreement, for instance, - ntor, are not quantum resistant. ECC will be broken when quantum - computers become available. This allows an adversary with significant - storage capabilities to harvest Tor handshakes now and decrypt them in - the future. + We are trying to add Quantum-safe forward-secrecy to the key agreement in + tor handshake. (Classical) forward-secrecy means that if the long-term key + is compromised, the communication prior to this compromise still stays + secure. Similarly, Quantum-safe forward-secrecy implies if the long-term + key is compromised due to attackers with quantum-computing capabilities, the + prior communication still remains secure. - 1.2 Motivation: Disaster resilience + Current approaches for handling key agreement, for instance, the ntor + handshake protocol, do not have this feature. ntor uses ECC, which will be + broken when quantum computers become available. This allows the simple yet + very effective harvest-then-decrypt attack, where an adversary with + significant storage capabilities harvests Tor handshakes now and decrypt + them in the future. - We are really trying to protect against the disastrous situation of one - key being entirely compromised. By introducing a second cryptographic - primitive, namely, NTRUEncrypt, we ensure that the system remains secure - in those extreme scenarios. + The proposed handshake protocol achieves quantum-safe forward-secrecy and + stops those attacks by introducing a secondary short-term pre-master secret + that is transported via a quantum-safe method. In the case where long-term + key is compromised via quantum algorithm, the attacker still need to recover + the second pre-master secret to be able to decrypt the communication. - 1.3 Motivation: Allowing plug & play for quantum-safe encryption algorithms + 1.2 Motivation: Allowing plug & play for quantum-safe encryption algorithms - We would like to be conservative on the selection of quantum-safe - encryption algorithm. For this purpose, we propose a modular design that - allows any quantum-safe encryption algorithm to be included in this - handshake framework. We will illustrate the proposal with NTRUEncrypt - encryption algorithm. + We would like to be conservative on the selection of quantum-safe encryption + algorithm. For this purpose, we propose a modular design that allows any + quantum-safe encryption algorithm to be included in this handshake + framework. We will illustrate the proposal with NTRUEncrypt encryption + algorithm. 2. Proposal @@ -63,8 +70,8 @@ Status: Open protocol. This proposed new handshake protocol is consistent with that approach. - We aim to provide quantum resistance and disaster resilience to the Tor - network, with the minimum impact on the current version. We aim to use + We aim to provide quantum-safe forward-secrecy and modular design to the Tor + handshake, with the minimum impact on the current version. We aim to use as many existing mechanisms as possible. For purposes of comparison, proposed modifications are indicated with * at @@ -72,50 +79,39 @@ Status: Open are marked with # when applicable. In order to enable variant quantum-safe algorithms for Tor handshake, we - propose a modular approach that allows any quantum-safe encryption - algorithm to be adopted in this framework. Our approach is a hybridization - of ntor protocol and a KEM. We instantiate this framework with NTRUEncrypt, - a lattice-based encryption scheme that is believed to be quantum resistant. - This framework is expandable to other quantum-safe encrypt schemes such as - Ring Learning with Error (R-LWE) based schemes. + propose a modular approach that allows any quantum-safe encryption algorithm + to be adopted in this framework. Our approach is a hybridization of ntor + protocol and a KEM. We instantiate this framework with NTRUEncrypt, a + lattice-based encryption scheme that is believed to be quantum resistant. + This framework is expandable to other quantum-safe encryptions such as Ring + Learning with Error (R-LWE) based schemes. 2.1.1 Achieved Property: - 1) The proposed key exchange method is quantum resistant: The forward - secrecy of the scheme assumes future adversaries are equipped with - quantum computers. + 1) The proposed key exchange method is quantum-safe forward-secure: two + secrets are exchanged, one protected by ECC, one protected by NTRUEncrypt, + and then put through the native Tor Key Derivation Function (KDF) to + derive the encryption and authentication keys. Both secrets are protected + with one-time keys for their respective public key algorithms. - 2) The proposed key exchange method is disaster resilient: The protocol - depends on two cryptographic primitives. Compromising one does not break - the security of the whole system. - - 3) The proposed key exchange method provides one-way authentication: The + 2) The proposed key exchange method provides one-way authentication: The server is authenticated, while the client remains anonymous. - 4) The protocol is almost backward compatible with its previous version: - ntor. By omitting a single field, one obtains the exact ntor + 3) The protocol is almost backward compatible with its previous + version: ntor. By omitting a single field, one obtains the exact ntor protocol. That is, the protocol is at least as secure as ntor. - 5) The protocol provides perfect forward secrecy: two secrets are - exchanged, one protected by ECC, one protected by NTRUEncrypt, and then - put through the native Tor Key Derivation Function (KDF) to derive the - encryption and authentication keys. Both secrets are protected with - one-time keys for their respective public key algorithms. - 2.1.2 General idea: When a client wishes to establish a one-way authenticated key K with a server, through following steps a session key is established: - - 1) Establish a common secret E (classical cryptography, i.e., ECC) using - a one-way authenticated key exchange protocol. #ntor currently uses this - approach#; - - 2) Establish a common "parallel" secret P using a key encapsulation + 1) Establish a common secret E (classical cryptography, i.e., ECC) using + a one-way authenticated key exchange protocol. + #ntor currently uses this approach#; + 2) Establish a common "parallel" secret P using a key encapsulation mechanism similar to TLS_RSA. In this feature request we use NTRUEncrypt as an example. - - 3) Establish a new session key k = KDF(E|P, info, i), where KDF is a Key + 3) Establish a new session key k = KDF(E|P, info, i), where KDF is a Key Derivation Function. 2.1.3 Building Blocks @@ -129,12 +125,14 @@ Status: Open 3) HMAC-based Extract-and-Expand Key Derivation Function - KDF-RFC5869; ##existing approach: See 5.2.2 tor-spec.txt## + Note: a new hash function, SHA3 as in FIPS 202, will be used, rather than + SHA256 as in ntor. 2.2 The protocol 2.2.1 Initialization - H(x,t) as HMAC_SHA256 with message x and key t. + H(x,t) as HMAC_SHA3 with message x and key t. H_LENGTH = 32 ID_LENGTH = 20 G_LENGTH = 32 @@ -142,16 +140,14 @@ Status: Open * QSPK_LENGTH = XXX length of QSE public key * QSC_LENGTH = XXX length of QSE cipher -* PROTOID = "ntor-curve25519-sha256-1-[qseid]" +* PROTOID = "ntor-curve25519-sha3-1-[qseid]" #pre PORTOID = "ntor-curve25519-sha256-1" t_mac = PROTOID | ":mac" t_key = PROTOID | ":key_extract" t_verify = PROTOID | ":verify" - These three variables define three different cryptographic hash - functions: - + These three variables define three different cryptographic hash functions: hash1 = HMAC(*, t_mac); hash2 = HMAC(*, t_key); hash3 = HMAC(*, t_verify); @@ -180,21 +176,21 @@ Status: Open The client generates a temporary key pair: x, X = KEYGEN(); - an NTRU temporary key pair: + an QSE temporary key pair: * QSSK, QSPK = QSKEYGEN(); -============================================================================== +================================================================================ and generates a client-side handshake with contents: NODEID Server identity digest [ID_LENGTH bytes] KEYID KEYID(B) [H_LENGTH bytes] CLIENT_PK X [G_LENGTH bytes] * QSPK QSPK [QSPK_LENGTH bytes] -============================================================================== +================================================================================ The server generates an ephemeral curve25519 keypair: y, Y = KEYGEN(); - a ephemeral "parallel" secret for encryption with NTRU: + a ephemeral "parallel" secret for encryption with QSE: * PAR_SEC P [H_LENGTH bytes] and computes: @@ -214,12 +210,12 @@ Status: Open | ID | PROTOID | "Server" #pre auth_input = verify | B | Y | X | ID | PROTOID | "Server" -============================================================================== +================================================================================ The server's handshake reply is: SERVER_PK Y [G_LENGTH bytes] AUTH H(auth_input, t_mac) [H_LENGTH bytes] * QSCIPHER C [QSPK_LENGTH bytes] -============================================================================== +================================================================================ The client then checks Y is in G^*, and computes E = EXP(Y,x) | EXP(B,x) | B | X | Y @@ -242,18 +238,19 @@ Status: Open 2.3 Instantiation with NTRUEncrypt - The example uses the NTRU parameter set NTRU_EESS439EP1. This has keys - and ciphertexts of length 604 bytes. This parameter set delivers 128 bits - classical security. For 128 bits quantum security, use NTRU_EESS743EP1. + The example uses the NTRU parameter set NTRU_EESS443EP1. This has keys + and ciphertexts of length 610 bytes. This parameter set delivers 128 bits + classical security and quantum security. For 256 bits classcial and quantum + security, use NTRU_EESS743EP1. We adjust the following parameters: handshake type: 0X0101 ntor+ntru the quantum safe KEM is based on NTRUEncrypt, with - parameter ntrueess439ep1 - PROTOID = "ntor-curve25519-sha256-1-ntrueess439ep1" - QSPK_LENGTH = 609 length of NTRU_EESS439EP1 public key - QSC_LENGTH = 604 length of NTRU_EESS439EP1 cipher + parameter ntrueess443ep1 + PROTOID = "ntor-curve25519-sha3-1-ntrueess443ep1" + QSPK_LENGTH = 610 length of NTRU_EESS439EP1 public key + QSC_LENGTH = 610 length of NTRU_EESS439EP1 cipher NTRUEncrypt can be adopted in our framework without further modification. @@ -272,29 +269,40 @@ Status: Open model can be found at https://eprint.iacr.org/2015/287. 3.3 Cryptographic hash function - The default hash function HMAC_SHA256 from Tor suffice all requirements of - our proposal. + The default hash function HMAC_SHA256 from Tor. This is suffice to provide + desired security for the present day. However, to be more future proof, we + propose to use HMAC_SHA3 when Tor starts to migrant to SHA3. 3.4 Key Encapsulation Mechanism The KEM in our protocol can be proved to be KEM-CCA-2 secure. - 3.5 Forward Secrecy - The forward secrecy is achieved. + 3.5 Quantum-safe Forward Secrecy + The quantum-safe forward secrecy is achieved. - Note that although this protocol meets the indicated goals, it is secure - only until a quantum computer is developed that is capable of breaking the - onion keys in real time. Such a computer can compromise the authentication - of ntor online; the security of this approach depends on the authentication - being secure at the time the handshake is executed. This approach is - intended to provide security against the harvest-then-decrypt attack while - an acceptable quantum-safe approach with security against an active - attacker is developed. - + 3.6 Quantum-safe authentication + The proposed protocol is secure only until a quantum computer is developed + that is capable of breaking the onion keys in real time. Such a computer can + compromise the authentication of ntor online; the security of this approach + depends on the authentication being secure at the time the handshake is + executed. This approach is intended to provide security against the + harvest-then-decrypt attack while an acceptable quantum-safe approach with + security against an active attacker is developed. -4. Bibliography +4. Candidate quantum-safe encryption algorithms + + We do not propose any quantum-safe encryption algorithms in this proposal. + This documents focus on the hybrid design. The implementer should modularize + the protocol with appropriate interfaces that allows any quantum-safe + encryption algorithms to be used in this setting. + + Candidate quantum-safe encryption algorithms will be included in another + proposal. This document will refer to the proposal when both proposals are + mature. + + +5. Bibliography [DK05] Y. Dodis, J. Katz, "Chosen-Ciphertext Security of Mulitple Encryption", Theory of Cryptography Conference, 2005. http://link.springer.com/chapter/10.1007%2F978-3-540-30576-7_11 (conference version) or http://cs.nyu.edu/~dodis/ps/2enc.pdf (preprint) -

1 0

[translation/tails-perl5lib_completed] Update translations for tails-perl5lib_completed
by translation＠torproject.org 12 Jan '16

12 Jan '16

commit 3a5c121d46d85f930c30e8e9f32f0cf2af74fb0b Author: Translation commit bot <translation(a)torproject.org> Date: Tue Jan 12 14:16:27 2016 +0000 Update translations for tails-perl5lib_completed --- es.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/es.po b/es.po index 1c88eef..7f1f372 100644 --- a/es.po +++ b/es.po @@ -9,7 +9,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: Tails developers <tails(a)boum.org>\n" "POT-Creation-Date: 2016-01-11 15:40+0100\n" -"PO-Revision-Date: 2016-01-12 11:43+0000\n" +"PO-Revision-Date: 2016-01-12 14:12+0000\n" "Last-Translator: strel\n" "Language-Team: Spanish (http://www.transifex.com/otf/torproject/language/es/)\n" "MIME-Version: 1.0\n"

1 0

[translation/tails-perl5lib] Update translations for tails-perl5lib
by translation＠torproject.org 12 Jan '16

12 Jan '16

commit f7e1bd08750d6f6d437291b2256e98ab6737787a Author: Translation commit bot <translation(a)torproject.org> Date: Tue Jan 12 14:16:19 2016 +0000 Update translations for tails-perl5lib --- es.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/es.po b/es.po index 1c88eef..7f1f372 100644 --- a/es.po +++ b/es.po @@ -9,7 +9,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: Tails developers <tails(a)boum.org>\n" "POT-Creation-Date: 2016-01-11 15:40+0100\n" -"PO-Revision-Date: 2016-01-12 11:43+0000\n" +"PO-Revision-Date: 2016-01-12 14:12+0000\n" "Last-Translator: strel\n" "Language-Team: Spanish (http://www.transifex.com/otf/torproject/language/es/)\n" "MIME-Version: 1.0\n"

1 0

[translation/tails-misc_completed] Update translations for tails-misc_completed
by translation＠torproject.org 12 Jan '16

12 Jan '16

commit 6eb5a9bbf29ba90a8a2d7294b71ba0b1b994cfd6 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Jan 12 14:15:59 2016 +0000 Update translations for tails-misc_completed --- es.po | 4 ++-- ru.po | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/es.po b/es.po index 95c2a48..85be497 100644 --- a/es.po +++ b/es.po @@ -14,8 +14,8 @@ msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2016-01-04 11:56+0100\n" -"PO-Revision-Date: 2016-01-06 19:29+0000\n" +"POT-Creation-Date: 2016-01-11 21:00+0100\n" +"PO-Revision-Date: 2016-01-12 14:12+0000\n" "Last-Translator: strel\n" "Language-Team: Spanish (http://www.transifex.com/otf/torproject/language/es/)\n" "MIME-Version: 1.0\n" diff --git a/ru.po b/ru.po index c8337d4..c66e51e 100644 --- a/ru.po +++ b/ru.po @@ -24,7 +24,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2016-01-11 21:00+0100\n" -"PO-Revision-Date: 2016-01-12 10:58+0000\n" +"PO-Revision-Date: 2016-01-12 14:07+0000\n" "Last-Translator: Antony A. Tolmachev <a.tolmachev(a)richart-consalt.ru>\n" "Language-Team: Russian (http://www.transifex.com/otf/torproject/language/ru/)\n" "MIME-Version: 1.0\n"

1 0