tor-commits October 2015

tor-commits@lists.torproject.org

14 participants
1182 discussions

[torspec/master] fixup name the right function in 210
by nickm＠torproject.org 21 Oct '15

21 Oct '15

commit 5149a07d896e212655992468dbb1edc980c9d0d2 Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Sat Oct 3 22:44:35 2015 +0200 fixup name the right function in 210 --- .../210-faster-headless-consensus-bootstrap.txt | 27 ++++++++++++-------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/proposals/210-faster-headless-consensus-bootstrap.txt b/proposals/210-faster-headless-consensus-bootstrap.txt index d527c2c..8e3cc69 100644 --- a/proposals/210-faster-headless-consensus-bootstrap.txt +++ b/proposals/210-faster-headless-consensus-bootstrap.txt @@ -154,23 +154,30 @@ Implementation Notes: Code Modifications if the purpose is DIR_PURPOSE_FETCH_CONSENSUS and there is no valid (reasonably live) consensus. We can make multiple connections from update_consensus_networkstatus_downloads(), as the sockets are non-blocking. - [ XXX - is this true for all platforms? ] + [ XXX - is this socket actually non-blocking for all platforms? ] As long as we can tolerate a timer resolution of ~1 second (due to the use of time_t), this requires no additional timers or callbacks. We can make 1 - connection for each schedule per second, for a total of 2 per second, or 4 - per second if the IPv4 and IPv6 schedules are implemented separately. + connection for each schedule per second, for a total of 2 per second. update_consensus_networkstatus_downloads() would also check the list of pending connections and, if it is 10 or greater, skip the connection attempt, and leave the retry time constant. - The code in directory_initiate_command_rend() or - connection_dir_finished_connecting() would need to be altered to check that - we are not already downloading the consensus. If we’re not, then call - directory_send_command() to download the consensus, and close any other - pending consensus dircons. (We may still want to check our clock against an - authority by allowing a TLS connection to complete, then immediately closing - it.) + The code in connection_dir_finished_connecting() would need to be altered to + check that we are not already downloading the consensus. If we’re not, then + call directory_send_command() to download the consensus, and close any other + pending consensus dircons. Since we want to check our clock against an + authority at least once per run, we instead mark authority connections so + they only request a HTTP HEAD, and use the first date header we see to + detect if the client’s clock is skewed. + [ XXX - does Tor support HTTP HEAD? ] + + We might also need to make similar changes in authority_certs_fetch_missing(), + as we can’t use a consensus until we have enough authority certificates. + However, Tor already makes multiple requests (one per certificate), and only + needs a majority of certificates to validate a consensus. Therefore, we will + only need to modify authority_certs_fetch_missing() if clients download a + consensus, then end up getting stuck downloading certificates. Reliability Analysis

1 0

[torspec/master] fixup prop 210 split relay and client schedules
by nickm＠torproject.org 21 Oct '15

21 Oct '15

commit e7c05956cf7f175ec3c6c8d7117def3f4d4c649a Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Sun Oct 4 22:16:41 2015 +0200 fixup prop 210 split relay and client schedules Also improve implementation notes. --- .../210-faster-headless-consensus-bootstrap.txt | 103 ++++++++++++-------- 1 file changed, 65 insertions(+), 38 deletions(-) diff --git a/proposals/210-faster-headless-consensus-bootstrap.txt b/proposals/210-faster-headless-consensus-bootstrap.txt index 8e3cc69..380e267 100644 --- a/proposals/210-faster-headless-consensus-bootstrap.txt +++ b/proposals/210-faster-headless-consensus-bootstrap.txt @@ -27,28 +27,51 @@ Design: Bootstrap Process Changes and authority connections are tried. Mirror connections are tried at a faster rate than authority connections. - We specify that mirror connections retry after one second, and then - double the retry time with every connection: + Client Schedules: + + Clients represent the majority of the load on the network. They can use + directory mirrors to download their documents, as the mirrors download + their documents from the authorities early in the consensus validity + period. + + We specify that client mirror connections retry after one second, and + then double the retry time with every connection: 0, 1, 2, 4, 8, 16, 32, ... - We specify that directory authority connections retry after 5 seconds, - and then double the retry time with every connection: + We specify that client directory authority connections retry after + 10 seconds, and then double the retry time with every connection: 0, 10, 20, ... - [ XXX: should we add random noise to these scheduled times? - teor ] + If a client has both an IPv4 and IPv6 address, it will try IPv4 and + IPv6 mirrors and authorities on the following schedule: + IPv4, IPv6, IPv4, IPv6, ... - The maximum retry time for both timers is 3 days + 1 hour. This places a - small load on the mirrors and authorities, while allowing a client that - regains a network connection to eventually download a consensus. + Relay Schedules: - If the client has both an IPv4 and IPv6 address, we try IPv4 and IPv6 - mirrors and authorities on the following schedule: - IPv4, IPv6, IPv4, IPv6, ... + Relays represent a small load on the network, but place a proportionally + greater load on the authorities [citation needed]. They can’t use + directory mirrors to download their documents, as they themselves are + the mirrors. + + We specify that relay directory authority connections retry after + 5 seconds, and then double the retry time with every connection: + 0, 5, 10, ... + + If a relay has both an IPv4 and IPv6 address, it will try IPv4 and + IPv6 mirrors and authorities on the following schedule: + IPv4, IPv4, IPv6, IPv4, IPv6, ... + + [ XXX: should we add random noise to these scheduled times? - teor ] + + The maximum retry time for all these timers is 3 days + 1 hour. This + places a small load on the mirrors and authorities, while allowing a + client that regains a network connection to eventually download a + consensus. We try IPv4 first to avoid overloading IPv6-enabled authorities and - mirrors. Mirrors and auths get a separate IPv4/IPv6 schedule. This - ensures that we try an IPv6 authority within the first 10 seconds. - This helps implement #8374 and related tickets. + mirrors. Each timing schedule uses a separate IPv4/IPv6 schedule. + This ensures that clients and relays try an IPv6 authority within + the first 10 seconds. This helps implement #8374 and related tickets. We don't want to keep on trying an IP version that always fails. Therefore, once sufficient IPv4 and IPv6 connections have been @@ -68,12 +91,6 @@ Design: Bootstrap Process Changes document and the others will be closed, after which bootstrapping will proceed as normal. - A benefit of connecting to directory authorities is that clients are - warned if their clock is wrong. Therefore, when closing a directory - authority connection, we check to see if we have successfully connected - to an authority during this run of the Tor client. If not, we allow the - authority TLS connection to complete, then close the connection. - We expect the vast majority of clients to succeed within 4 seconds, after making up to 4 connection attempts to mirrors and 1 connection attempt to an authority. Clients which can't connect in the first @@ -82,13 +99,18 @@ Design: Bootstrap Process Changes 10 seconds. This is a much better success rate than the current Tor implementation, which fails k/n of clients if k of the n directory authorities are down. (Or, if the connection fails in certain ways, - (k/n)^2.) + it will retry once, failing 1-(1-(k/n)^2).) If at any time, the total outstanding bootstrap connection attempts exceeds 10, no new connection attempts are to be launched until an existing connection attempt experiences full timeout. The retry time is not doubled when a connection is skipped. + A benefit of connecting to directory authorities is that clients are + warned if their clock is wrong. Starting the authority and fallback + schedules at the same time should ensure that some clients check their + clock with an authority at each bootstrap. + Design: Fallback Dir Mirror Selection The set of hard coded directory mirrors from #572 shall be chosen using @@ -111,12 +133,13 @@ Performance: Additional Load with Current Parameter Choices This design and the connection count parameters were chosen such that no additional bandwidth load would be placed on the directory authorities. In fact, the directory authorities should experience less - load, because they will not need to serve the consensus document for a - connection in the event that one of the directory mirrors complete their - connection before the directory authority does. + load, because they will not need to serve the entire consensus document + for a connection in the event that one of the directory mirrors complete + their connection before the directory authority does. (However, they + may need to serve the consensus document HEAD for clock checks.) However, the scheme does place additional TLS connection load on the - fallback dir mirrors. Because bootstrapping is rare and all but one of + fallback dir mirrors. Because bootstrapping is rare, and all but one of the TLS connections will be very short-lived and unused, this should not be a substantial issue. @@ -154,23 +177,26 @@ Implementation Notes: Code Modifications if the purpose is DIR_PURPOSE_FETCH_CONSENSUS and there is no valid (reasonably live) consensus. We can make multiple connections from update_consensus_networkstatus_downloads(), as the sockets are non-blocking. - [ XXX - is this socket actually non-blocking for all platforms? ] - As long as we can tolerate a timer resolution of ~1 second (due to the use - of time_t), this requires no additional timers or callbacks. We can make 1 - connection for each schedule per second, for a total of 2 per second. + (This socket appears to be non-blocking on Unixes (SOCK_NONBLOCK & O_NONBLOCK) + and Windows (FIONBIO).) As long as we can tolerate a timer resolution of + ~1 second (due to the use of time_t), this requires no additional timers or + callbacks. We can make 1 connection for each schedule per second, for a total + of 1-2 per second. + + Since a tor relay fetches its directory material directly from the + authorities, it uses a different schedule. Multiple connections are + only useful to a relay if multiple authorities are down, so we retry + less often than for clients (clients use multiple fallbacks). + update_consensus_networkstatus_downloads() should check + directory_fetches_from_authorities() to choose the appropriate schedule. update_consensus_networkstatus_downloads() would also check the list of pending connections and, if it is 10 or greater, skip the connection attempt, and leave the retry time constant. - The code in connection_dir_finished_connecting() would need to be altered to - check that we are not already downloading the consensus. If we’re not, then - call directory_send_command() to download the consensus, and close any other - pending consensus dircons. Since we want to check our clock against an - authority at least once per run, we instead mark authority connections so - they only request a HTTP HEAD, and use the first date header we see to - detect if the client’s clock is skewed. - [ XXX - does Tor support HTTP HEAD? ] + The code in directory_send_command() would need to be altered to check that we + are not already downloading the consensus. If we’re not, then download the + consensus on this connection, and close any other pending consensus dircons. We might also need to make similar changes in authority_certs_fetch_missing(), as we can’t use a consensus until we have enough authority certificates. @@ -187,6 +213,7 @@ Reliability Analysis uptime.) We expect the first 10 connection retry times to be: + (Research shows users tend to lose interest after 40 seconds.) Mirror: 0s 1s 2s 4s 8s 16s 32s Auth: 0s 10s 20s Success: 90% 95% 97% 98.7% 99.4% 99.89% 99.94% 99.988% 99.994% @@ -196,7 +223,7 @@ Reliability Analysis 99.89% of clients succeed in the first 10 seconds. 0.11% of clients remain, but in this scenario, 2 authorities are unreachable, so the client is most likely blocked from the Tor - network. + network. Alternately, they will likely succeed on relaunch. The current implementation makes 1 or 2 authority connections within the first second, depending on exactly how the first connection fails. Under

1 0

[torspec/master] prop 210 further clarifications
by nickm＠torproject.org 21 Oct '15

21 Oct '15

commit 13c75f888cf9ac401eab674a7b4652bab3d21c5d Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Sat Oct 17 16:33:02 2015 +1100 prop 210 further clarifications --- .../210-faster-headless-consensus-bootstrap.txt | 77 ++++++++++---------- 1 file changed, 38 insertions(+), 39 deletions(-) diff --git a/proposals/210-faster-headless-consensus-bootstrap.txt b/proposals/210-faster-headless-consensus-bootstrap.txt index 380e267..d3c56ff 100644 --- a/proposals/210-faster-headless-consensus-bootstrap.txt +++ b/proposals/210-faster-headless-consensus-bootstrap.txt @@ -27,16 +27,16 @@ Design: Bootstrap Process Changes and authority connections are tried. Mirror connections are tried at a faster rate than authority connections. - Client Schedules: - Clients represent the majority of the load on the network. They can use directory mirrors to download their documents, as the mirrors download their documents from the authorities early in the consensus validity period. We specify that client mirror connections retry after one second, and - then double the retry time with every connection: + then double the retry time with every connection attempt: 0, 1, 2, 4, 8, 16, 32, ... + (The timers currently implemented in Tor increment with every + connection failure.) We specify that client directory authority connections retry after 10 seconds, and then double the retry time with every connection: @@ -46,22 +46,14 @@ Design: Bootstrap Process Changes IPv6 mirrors and authorities on the following schedule: IPv4, IPv6, IPv4, IPv6, ... - Relay Schedules: + [ TODO: should we add random noise to these scheduled times? - teor + Tor doesn’t add random noise to the current failure-based + timers, but as failures are a network event, they are + somewhat random/arbitrary already. These attempt-based timers + will go off every few seconds, exactly erraon the second. ] - Relays represent a small load on the network, but place a proportionally - greater load on the authorities [citation needed]. They can’t use - directory mirrors to download their documents, as they themselves are - the mirrors. - - We specify that relay directory authority connections retry after - 5 seconds, and then double the retry time with every connection: - 0, 5, 10, ... - - If a relay has both an IPv4 and IPv6 address, it will try IPv4 and - IPv6 mirrors and authorities on the following schedule: - IPv4, IPv4, IPv6, IPv4, IPv6, ... - - [ XXX: should we add random noise to these scheduled times? - teor ] + (Relays can’t use directory mirrors to download their documents, + as they *are* the directory mirrors.) The maximum retry time for all these timers is 3 days + 1 hour. This places a small load on the mirrors and authorities, while allowing a @@ -70,8 +62,8 @@ Design: Bootstrap Process Changes We try IPv4 first to avoid overloading IPv6-enabled authorities and mirrors. Each timing schedule uses a separate IPv4/IPv6 schedule. - This ensures that clients and relays try an IPv6 authority within - the first 10 seconds. This helps implement #8374 and related tickets. + This ensures that clients try an IPv6 authority within the first + 10 seconds. This helps implement #8374 and related tickets. We don't want to keep on trying an IP version that always fails. Therefore, once sufficient IPv4 and IPv6 connections have been @@ -85,7 +77,9 @@ Design: Bootstrap Process Changes The retry timers and IP version schedules must reset on HUP and any network reachability events, so that clients that have unreliable networks can recover from network failures. - [ TODO: do we have network reachability events? ] + [ TODO: Do we do this for any other timers? + I think this needs another proposal, it’s out of scope here. + - teor ] The first connection to complete will be used to download the consensus document and the others will be closed, after which bootstrapping will @@ -128,6 +122,8 @@ Design: Fallback Dir Mirror Selection should be set at 20% of the current Guard nodes (approximately 200 as of October 2015), rather than fixed at 100. + [TODO: change the script to dynamically calculate an upper limit.] + Performance: Additional Load with Current Parameter Choices This design and the connection count parameters were chosen such that @@ -135,8 +131,7 @@ Performance: Additional Load with Current Parameter Choices authorities. In fact, the directory authorities should experience less load, because they will not need to serve the entire consensus document for a connection in the event that one of the directory mirrors complete - their connection before the directory authority does. (However, they - may need to serve the consensus document HEAD for clock checks.) + their connection before the directory authority does. However, the scheme does place additional TLS connection load on the fallback dir mirrors. Because bootstrapping is rare, and all but one of @@ -179,31 +174,35 @@ Implementation Notes: Code Modifications update_consensus_networkstatus_downloads(), as the sockets are non-blocking. (This socket appears to be non-blocking on Unixes (SOCK_NONBLOCK & O_NONBLOCK) and Windows (FIONBIO).) As long as we can tolerate a timer resolution of - ~1 second (due to the use of time_t), this requires no additional timers or - callbacks. We can make 1 connection for each schedule per second, for a total - of 1-2 per second. + ~1 second (due to the use of second_elapsed_callback and time_t), this + requires no additional timers or callbacks. We can make 1 connection for each + schedule per second, for a maximum of 2 per second. + + The schedules can be specified in: + TestingClientBootstrapConsensusAuthorityDownloadSchedule + TestingClientBootstrapConsensusFallbackDownloadSchedule + (Similar to the existing TestingClientConsensusDownloadSchedule.) - Since a tor relay fetches its directory material directly from the - authorities, it uses a different schedule. Multiple connections are - only useful to a relay if multiple authorities are down, so we retry - less often than for clients (clients use multiple fallbacks). - update_consensus_networkstatus_downloads() should check - directory_fetches_from_authorities() to choose the appropriate schedule. + TestingServerIPVersionPreferenceSchedule + (Consisting of a CSV like “4,6,4,6”, or perhaps “0,1,0,1”.) - update_consensus_networkstatus_downloads() would also check the list of - pending connections and, if it is 10 or greater, skip the connection - attempt, and leave the retry time constant. + update_consensus_networkstatus_downloads() checks the list of pending + connections and, if it is 10 or greater, skip the connection attempt, + and leave the retry time constant. - The code in directory_send_command() would need to be altered to check that we - are not already downloading the consensus. If we’re not, then download the - consensus on this connection, and close any other pending consensus dircons. + The code in directory_send_command() and connection_finished_connecting() + would need to be altered to check that we are not already downloading the + consensus. If we’re not, then download the consensus on this connection, and + close any other pending consensus dircons. We might also need to make similar changes in authority_certs_fetch_missing(), as we can’t use a consensus until we have enough authority certificates. However, Tor already makes multiple requests (one per certificate), and only needs a majority of certificates to validate a consensus. Therefore, we will only need to modify authority_certs_fetch_missing() if clients download a - consensus, then end up getting stuck downloading certificates. + consensus, then end up getting stuck downloading certificates. (Current tests + show bootstrapping working well without any changes to authority certificate + fetches.) Reliability Analysis

1 0

[torspec/master] Merge remote-tracking branch 'teor/bootstrap-exponential-backoff-v2'
by nickm＠torproject.org 21 Oct '15

21 Oct '15

commit 3bac19d0b31b54156ff42cda68dfa093e8d9552e Merge: bc6855e 13c75f8 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Oct 21 11:55:25 2015 -0400 Merge remote-tracking branch 'teor/bootstrap-exponential-backoff-v2' .../210-faster-headless-consensus-bootstrap.txt | 231 +++++++++++++++----- 1 file changed, 176 insertions(+), 55 deletions(-)

1 0

[tor/master] Check for len < 4 in dn_indicates_v3_cert
by nickm＠torproject.org 21 Oct '15

21 Oct '15

commit 35bf07b8d67d018f7740ca195cf8c7c86b1b4ef9 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Oct 21 11:44:43 2015 -0400 Check for len < 4 in dn_indicates_v3_cert Without this check, we potentially look up to 3 characters before the start of a malloc'd segment, which could provoke a crash under certain (weird afaik) circumstances. Fixes 17404; bugfix on 0.2.6.3-alpha. --- changes/bug17404 | 6 ++++++ src/common/tortls.c | 4 ++++ 2 files changed, 10 insertions(+) diff --git a/changes/bug17404 b/changes/bug17404 new file mode 100644 index 0000000..d524f66 --- /dev/null +++ b/changes/bug17404 @@ -0,0 +1,6 @@ + o Major bugfixes (security, correctness): + - Fix a programming error that could cause us to read 4 bytes before + the beginning of an openssl string. This could be used to provoke + a crash on systems with an unusual malloc implementation, or + systems with unsual hardening installed. Fixes bug 17404; bugfix + on 0.2.3.6-alpha. diff --git a/src/common/tortls.c b/src/common/tortls.c index 4222f6d..75ca47d 100644 --- a/src/common/tortls.c +++ b/src/common/tortls.c @@ -2676,6 +2676,10 @@ dn_indicates_v3_cert(X509_NAME *name) len = ASN1_STRING_to_UTF8(&s, str); if (len < 0) return 0; + if (len < 4) { + OPENSSL_free(s); + return 0; + } r = fast_memneq(s + len - 4, ".net", 4); OPENSSL_free(s); return r;

1 0

[tor/master] Merge remote-tracking branch 'public/bug17404_024' into maint-0.2.7
by nickm＠torproject.org 21 Oct '15

21 Oct '15

commit b809c265e77f4528b59aa3932a3cde8bf5e19fb3 Merge: 9c4a0ae 35bf07b Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Oct 21 11:51:03 2015 -0400 Merge remote-tracking branch 'public/bug17404_024' into maint-0.2.7 changes/bug17404 | 6 ++++++ src/common/tortls.c | 4 ++++ 2 files changed, 10 insertions(+)

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.7'
by nickm＠torproject.org 21 Oct '15

21 Oct '15

commit 895a98dbaf9619c8a24ef872bfeca84a764a6ccb Merge: 52fd384 b809c26 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Oct 21 11:53:00 2015 -0400 Merge remote-tracking branch 'origin/maint-0.2.7' changes/bug17404 | 6 ++++++ src/common/tortls.c | 4 ++++ 2 files changed, 10 insertions(+) diff --cc src/common/tortls.c index 1567508,b597fe2..d863567 --- a/src/common/tortls.c +++ b/src/common/tortls.c @@@ -2453,9 -2519,12 +2453,13 @@@ dn_indicates_v3_cert(X509_NAME *name str = X509_NAME_ENTRY_get_data(entry); len = ASN1_STRING_to_UTF8(&s, str); - if (len < 0) + if (len < 0) { return 0; + } + if (len < 4) { + OPENSSL_free(s); + return 0; + } r = fast_memneq(s + len - 4, ".net", 4); OPENSSL_free(s); return r;

1 0

[stem/master] Mention get_hidden_service_conf() in get_conf_map()
by atagar＠torproject.org 21 Oct '15

21 Oct '15

commit 72fe9ff5299178a4597fdb4bfa28cf9f8c6ca35c Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Oct 21 08:51:37 2015 -0700 Mention get_hidden_service_conf() in get_conf_map() Mentioning that there's another method that's preferable for fetching hidden service options... https://trac.torproject.org/projects/tor/ticket/17378 --- stem/control.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/stem/control.py b/stem/control.py index d9693cc..72f0552 100644 --- a/stem/control.py +++ b/stem/control.py @@ -2044,6 +2044,9 @@ class Controller(BaseController): **HiddenServiceOptions** was the only option that falls into the third category. + **Note:** HiddenServiceOptions are best retrieved via the + :func:`~stem.control.Controller.get_hidden_service_conf` method instead. + :param str,list params: configuration option(s) to be queried :param object default: value for the mappings if the configuration option is either undefined or the query fails

1 0

[tor/maint-0.2.7] Merge remote-tracking branch 'public/bug17404_024' into maint-0.2.7
by nickm＠torproject.org 21 Oct '15

21 Oct '15

1 0

[tor/maint-0.2.7] Check for len < 4 in dn_indicates_v3_cert
by nickm＠torproject.org 21 Oct '15

21 Oct '15

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits October 2015