November 2017 - tor-commits - lists.torproject.org

[tor/maint-0.3.2] Fix DNS resolution on busy exit relays
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit 98275743081728b1538f70111c891ea573127e85 Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Fri Nov 3 03:06:12 2017 +0100 Fix DNS resolution on busy exit relays Thanks to Dhalgren who analyzed this issue two years ago and found a solution! --- changes/bug21394 | 9 +++++++++ src/or/dns.c | 23 ++++++++++++++++++++--- 2 files changed, 29 insertions(+), 3 deletions(-) diff --git a/changes/bug21394 b/changes/bug21394 new file mode 100644 index 000000000..e5452e20b --- /dev/null +++ b/changes/bug21394 @@ -0,0 +1,9 @@ + o Major bugfixes (Exit nodes): + - Fix an issue causing high-bandwidth exit nodes to fail a majority + or all of their DNS requests, making them basically unsuitable for + regular usage in Tor circuits. The problem is related to + libevent's DNS handling, but we can work around it in Tor. Fixes + bugs 21394 and 18580; bugfix on 0.1.2.2-alpha which introduced + eventdns. Credit goes to Dhalgren for identifying and finding a + workaround to this bug and to gamambel, arthuredelstein and + arma in helping to track it down and analyze it. diff --git a/src/or/dns.c b/src/or/dns.c index 0ad4c0f50..c1e3c3256 100644 --- a/src/or/dns.c +++ b/src/or/dns.c @@ -1425,14 +1425,31 @@ configure_nameservers(int force) #define SET(k,v) evdns_base_set_option(the_evdns_base, (k), (v)) + // If we only have one nameserver, it does not make sense to back off + // from it for a timeout. Unfortunately, the value for max-timeouts is + // currently clamped by libevent to 255, but it does not hurt to set + // it higher in case libevent gets a patch for this. + // Reducing attempts in the case of just one name server too, because + // it is very likely to be a local one where a network connectivity + // issue should not cause an attempt to fail. if (evdns_base_count_nameservers(the_evdns_base) == 1) { - SET("max-timeouts:", "16"); - SET("timeout:", "10"); + SET("max-timeouts:", "1000000"); + SET("attempts:", "1"); } else { SET("max-timeouts:", "3"); - SET("timeout:", "5"); } + // Elongate the queue of maximum inflight dns requests, so if a bunch + // time out at the resolver (happens commonly with unbound) we won't + // stall every other DNS request. This potentially means some wasted + // CPU as there's a walk over a linear queue involved, but this is a + // much better tradeoff compared to just failing DNS requests because + // of a full queue. + SET("max-inflight:", "8192"); + + // Time out after 5 seconds if no reply. + SET("timeout:", "5"); + if (options->ServerDNSRandomizeCase) SET("randomize-case:", "1"); else

1 0

[tor/master] Fix DNS resolution on busy exit relays
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit 98275743081728b1538f70111c891ea573127e85 Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Fri Nov 3 03:06:12 2017 +0100 Fix DNS resolution on busy exit relays Thanks to Dhalgren who analyzed this issue two years ago and found a solution! --- changes/bug21394 | 9 +++++++++ src/or/dns.c | 23 ++++++++++++++++++++--- 2 files changed, 29 insertions(+), 3 deletions(-) diff --git a/changes/bug21394 b/changes/bug21394 new file mode 100644 index 000000000..e5452e20b --- /dev/null +++ b/changes/bug21394 @@ -0,0 +1,9 @@ + o Major bugfixes (Exit nodes): + - Fix an issue causing high-bandwidth exit nodes to fail a majority + or all of their DNS requests, making them basically unsuitable for + regular usage in Tor circuits. The problem is related to + libevent's DNS handling, but we can work around it in Tor. Fixes + bugs 21394 and 18580; bugfix on 0.1.2.2-alpha which introduced + eventdns. Credit goes to Dhalgren for identifying and finding a + workaround to this bug and to gamambel, arthuredelstein and + arma in helping to track it down and analyze it. diff --git a/src/or/dns.c b/src/or/dns.c index 0ad4c0f50..c1e3c3256 100644 --- a/src/or/dns.c +++ b/src/or/dns.c @@ -1425,14 +1425,31 @@ configure_nameservers(int force) #define SET(k,v) evdns_base_set_option(the_evdns_base, (k), (v)) + // If we only have one nameserver, it does not make sense to back off + // from it for a timeout. Unfortunately, the value for max-timeouts is + // currently clamped by libevent to 255, but it does not hurt to set + // it higher in case libevent gets a patch for this. + // Reducing attempts in the case of just one name server too, because + // it is very likely to be a local one where a network connectivity + // issue should not cause an attempt to fail. if (evdns_base_count_nameservers(the_evdns_base) == 1) { - SET("max-timeouts:", "16"); - SET("timeout:", "10"); + SET("max-timeouts:", "1000000"); + SET("attempts:", "1"); } else { SET("max-timeouts:", "3"); - SET("timeout:", "5"); } + // Elongate the queue of maximum inflight dns requests, so if a bunch + // time out at the resolver (happens commonly with unbound) we won't + // stall every other DNS request. This potentially means some wasted + // CPU as there's a walk over a linear queue involved, but this is a + // much better tradeoff compared to just failing DNS requests because + // of a full queue. + SET("max-inflight:", "8192"); + + // Time out after 5 seconds if no reply. + SET("timeout:", "5"); + if (options->ServerDNSRandomizeCase) SET("randomize-case:", "1"); else

1 0

[tor/master] Merge branch 'bug21394_029' into maint-0.3.2
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit 9f3d9c68ab21e4856da11580c29f4895bf398a09 Merge: 7c1f33dd0 982757430 Author: Nick Mathewson <nickm(a)torproject.org> Date: Sun Nov 5 14:39:37 2017 -0500 Merge branch 'bug21394_029' into maint-0.3.2 changes/bug21394 | 9 +++++++++ src/or/dns.c | 23 ++++++++++++++++++++--- 2 files changed, 29 insertions(+), 3 deletions(-)

1 0

[tor/master] lintchanges: Allow 'fixes bugs a, b, and c'
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit f46cdb3ff85be4c4241928bfd3f558eb878cff28 Author: Nick Mathewson <nickm(a)torproject.org> Date: Sun Nov 5 14:41:39 2017 -0500 lintchanges: Allow 'fixes bugs a, b, and c' --- scripts/maint/lintChanges.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/maint/lintChanges.py b/scripts/maint/lintChanges.py index c2dda6dc8..d5b8fcae5 100755 --- a/scripts/maint/lintChanges.py +++ b/scripts/maint/lintChanges.py @@ -76,13 +76,13 @@ def lintfile(fname): if isBug and not re.search(r'(\d+)', contents): warn("Ticket marked as bugfix, but does not mention a number.") - elif isBug and not re.search(r'Fixes ([a-z ]*)bug (\d+)', contents): + elif isBug and not re.search(r'Fixes ([a-z ]*)bugs? (\d+)', contents): warn("Ticket marked as bugfix, but does not say 'Fixes bug XXX'") if re.search(r'[bB]ug (\d+)', contents): if not re.search(r'[Bb]ugfix on ', contents): warn("Bugfix does not say 'bugfix on X.Y.Z'") - elif not re.search('[fF]ixes ([a-z ]*)bug (\d+); bugfix on ', + elif not re.search('[fF]ixes ([a-z ]*)bugs? (\d+)((, \d+)* and \d+)?; bugfix on ', contents): warn("Bugfix does not say 'Fixes bug X; bugfix on Y'") elif re.search('tor-([0-9]+)', contents):

1 0

[tor/release-0.3.2] Fix DNS resolution on busy exit relays
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit 98275743081728b1538f70111c891ea573127e85 Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Fri Nov 3 03:06:12 2017 +0100 Fix DNS resolution on busy exit relays Thanks to Dhalgren who analyzed this issue two years ago and found a solution! --- changes/bug21394 | 9 +++++++++ src/or/dns.c | 23 ++++++++++++++++++++--- 2 files changed, 29 insertions(+), 3 deletions(-) diff --git a/changes/bug21394 b/changes/bug21394 new file mode 100644 index 000000000..e5452e20b --- /dev/null +++ b/changes/bug21394 @@ -0,0 +1,9 @@ + o Major bugfixes (Exit nodes): + - Fix an issue causing high-bandwidth exit nodes to fail a majority + or all of their DNS requests, making them basically unsuitable for + regular usage in Tor circuits. The problem is related to + libevent's DNS handling, but we can work around it in Tor. Fixes + bugs 21394 and 18580; bugfix on 0.1.2.2-alpha which introduced + eventdns. Credit goes to Dhalgren for identifying and finding a + workaround to this bug and to gamambel, arthuredelstein and + arma in helping to track it down and analyze it. diff --git a/src/or/dns.c b/src/or/dns.c index 0ad4c0f50..c1e3c3256 100644 --- a/src/or/dns.c +++ b/src/or/dns.c @@ -1425,14 +1425,31 @@ configure_nameservers(int force) #define SET(k,v) evdns_base_set_option(the_evdns_base, (k), (v)) + // If we only have one nameserver, it does not make sense to back off + // from it for a timeout. Unfortunately, the value for max-timeouts is + // currently clamped by libevent to 255, but it does not hurt to set + // it higher in case libevent gets a patch for this. + // Reducing attempts in the case of just one name server too, because + // it is very likely to be a local one where a network connectivity + // issue should not cause an attempt to fail. if (evdns_base_count_nameservers(the_evdns_base) == 1) { - SET("max-timeouts:", "16"); - SET("timeout:", "10"); + SET("max-timeouts:", "1000000"); + SET("attempts:", "1"); } else { SET("max-timeouts:", "3"); - SET("timeout:", "5"); } + // Elongate the queue of maximum inflight dns requests, so if a bunch + // time out at the resolver (happens commonly with unbound) we won't + // stall every other DNS request. This potentially means some wasted + // CPU as there's a walk over a linear queue involved, but this is a + // much better tradeoff compared to just failing DNS requests because + // of a full queue. + SET("max-inflight:", "8192"); + + // Time out after 5 seconds if no reply. + SET("timeout:", "5"); + if (options->ServerDNSRandomizeCase) SET("randomize-case:", "1"); else

1 0

[tor/release-0.3.2] lintchanges: Allow 'fixes bugs a, b, and c'
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit f46cdb3ff85be4c4241928bfd3f558eb878cff28 Author: Nick Mathewson <nickm(a)torproject.org> Date: Sun Nov 5 14:41:39 2017 -0500 lintchanges: Allow 'fixes bugs a, b, and c' --- scripts/maint/lintChanges.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/maint/lintChanges.py b/scripts/maint/lintChanges.py index c2dda6dc8..d5b8fcae5 100755 --- a/scripts/maint/lintChanges.py +++ b/scripts/maint/lintChanges.py @@ -76,13 +76,13 @@ def lintfile(fname): if isBug and not re.search(r'(\d+)', contents): warn("Ticket marked as bugfix, but does not mention a number.") - elif isBug and not re.search(r'Fixes ([a-z ]*)bug (\d+)', contents): + elif isBug and not re.search(r'Fixes ([a-z ]*)bugs? (\d+)', contents): warn("Ticket marked as bugfix, but does not say 'Fixes bug XXX'") if re.search(r'[bB]ug (\d+)', contents): if not re.search(r'[Bb]ugfix on ', contents): warn("Bugfix does not say 'bugfix on X.Y.Z'") - elif not re.search('[fF]ixes ([a-z ]*)bug (\d+); bugfix on ', + elif not re.search('[fF]ixes ([a-z ]*)bugs? (\d+)((, \d+)* and \d+)?; bugfix on ', contents): warn("Bugfix does not say 'Fixes bug X; bugfix on Y'") elif re.search('tor-([0-9]+)', contents):

1 0

[tor/master] Merge branch 'maint-0.3.2'
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit f755f9b9e67232d9d39682cbcdf4433ac738e17a Merge: 9f650b24e f46cdb3ff Author: Nick Mathewson <nickm(a)torproject.org> Date: Sun Nov 5 14:41:53 2017 -0500 Merge branch 'maint-0.3.2' changes/bug21394 | 9 +++++++++ scripts/maint/lintChanges.py | 4 ++-- src/or/dns.c | 23 ++++++++++++++++++++--- 3 files changed, 31 insertions(+), 5 deletions(-)

1 0

[tor/release-0.3.2] Merge branch 'bug21394_029' into maint-0.3.2
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit 9f3d9c68ab21e4856da11580c29f4895bf398a09 Merge: 7c1f33dd0 982757430 Author: Nick Mathewson <nickm(a)torproject.org> Date: Sun Nov 5 14:39:37 2017 -0500 Merge branch 'bug21394_029' into maint-0.3.2 changes/bug21394 | 9 +++++++++ src/or/dns.c | 23 ++++++++++++++++++++--- 2 files changed, 29 insertions(+), 3 deletions(-)

1 0

[tor/release-0.3.2] Merge branch 'maint-0.3.2' into release-0.3.2
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit 6bc1632b724da0593b0d781998bece33a0b6cec6 Merge: ed10b34ef f46cdb3ff Author: Nick Mathewson <nickm(a)torproject.org> Date: Sun Nov 5 14:41:53 2017 -0500 Merge branch 'maint-0.3.2' into release-0.3.2 changes/bug21394 | 9 +++++++++ scripts/maint/lintChanges.py | 4 ++-- src/or/dns.c | 23 ++++++++++++++++++++--- 3 files changed, 31 insertions(+), 5 deletions(-)

1 0

[tor/release-0.3.2] Merge branch 'maint-0.3.2' into release-0.3.2
by nickm＠torproject.org 05 Nov '17

05 Nov '17

commit ed10b34efeb29e75e214e37e04fc01d61fb07612 Merge: 3d966f57a 7c1f33dd0 Author: Nick Mathewson <nickm(a)torproject.org> Date: Sun Nov 5 14:09:21 2017 -0500 Merge branch 'maint-0.3.2' into release-0.3.2 changes/ticket21031 | 7 +++++++ doc/tor.1.txt | 17 ++++++++--------- src/or/config.c | 18 ++++++++++++------ src/or/connection_edge.c | 2 +- src/or/or.h | 2 +- src/or/relay.c | 6 +++--- src/test/test_options.c | 5 +++-- src/test/test_relaycell.c | 4 ++-- 8 files changed, 37 insertions(+), 24 deletions(-)

1 0