tor-commits November 2017

tor-commits@lists.torproject.org

16 participants
2019 discussions

[tor/master] Remove an erroneous 0.5 in compute_weighted_bandwidths()
by nickm＠torproject.org 08 Nov '17

08 Nov '17

commit 3c03e237ab372f495fa2498e925813931ba381da Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Sep 22 15:29:15 2017 -0400 Remove an erroneous 0.5 in compute_weighted_bandwidths() Back in 0.2.4.3-alpha (e106812a778f537), when we switched from using double to using uint64 for selecting by bandwidth, I got the math wrong: I should have used llround(x), or (uint64_t)(x+0.5), but instead I wrote llround(x+0.5). That means we would always round up, rather than rounding to the closest integer Fixes bug 23318; bugfix on 0.2.4.3-alpha. --- changes/bug23318 | 7 +++++++ src/or/routerlist.c | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/changes/bug23318 b/changes/bug23318 new file mode 100644 index 000000000..32c85eb19 --- /dev/null +++ b/changes/bug23318 @@ -0,0 +1,7 @@ + o Minor bugfixes (path selection): + - When selecting relays by bandwidth, avoid a rounding error that + could sometimes cause load to be imbalanced incorrectly. Previously, + we would always round upwards; now, we round towards the nearest + integer. This had the biggest effect when a relay's weight adjustments + should have given it weight 0, but it got weight 1 instead. + Fixes bug 23318; bugfix on 0.2.4.3-alpha. diff --git a/src/or/routerlist.c b/src/or/routerlist.c index 2365f28fd..faf2eeda5 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -2713,7 +2713,7 @@ compute_weighted_bandwidths(const smartlist_t *sl, final_weight = weight*this_bw; } - bandwidths[node_sl_idx] = final_weight + 0.5; + bandwidths[node_sl_idx] = final_weight; } SMARTLIST_FOREACH_END(node); log_debug(LD_CIRC, "Generated weighted bandwidths for rule %s based "

1 0

[tor/master] Merge branch 'bug23318-redux_029' into maint-0.3.2
by nickm＠torproject.org 08 Nov '17

08 Nov '17

commit c74a817e837774a5e488d6c544ea2b45d3e4ddcf Merge: e5a83062e 6e4ebd41b Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Nov 8 10:47:22 2017 -0500 Merge branch 'bug23318-redux_029' into maint-0.3.2 changes/bug23318 | 11 +++++++++++ changes/bug24170 | 3 +++ src/or/routerlist.c | 48 +++++++++++++++++++++++++++++++----------------- 3 files changed, 45 insertions(+), 17 deletions(-)

1 0

[tor/release-0.3.2] Merge remote-tracking branch 'dgoulet/bug23751_032_01' into maint-0.3.2
by nickm＠torproject.org 08 Nov '17

08 Nov '17

commit 02f05f2de2654de3ab24e26ca9f1745389487d32 Merge: c74a817e8 dcabf801e Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Nov 8 10:56:33 2017 -0500 Merge remote-tracking branch 'dgoulet/bug23751_032_01' into maint-0.3.2 changes/bug23751 | 6 ++++++ src/or/scheduler_kist.c | 9 +++++++++ 2 files changed, 15 insertions(+)

1 0

[tor/release-0.3.2] Merge branch 'maint-0.3.2' into release-0.3.2
by nickm＠torproject.org 08 Nov '17

08 Nov '17

commit 04714df9cc2a20f7daae498baee99c5dbf3c4432 Merge: e50da3327 02f05f2de Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Nov 8 11:02:51 2017 -0500 Merge branch 'maint-0.3.2' into release-0.3.2 changes/bug23318 | 11 +++++++++++ changes/bug23751 | 6 ++++++ changes/bug24170 | 3 +++ src/or/routerlist.c | 48 +++++++++++++++++++++++++++++++----------------- src/or/scheduler_kist.c | 9 +++++++++ 5 files changed, 60 insertions(+), 17 deletions(-)

1 0

[tor-browser-build/master] Bug 10394: Don't autoupdate HTTPS Everywhere anymore
by boklm＠torproject.org 08 Nov '17

08 Nov '17

commit 50982eda6d3687aa5bcc2d088546f82c4fa7e53d Author: Georg Koppen <gk(a)torproject.org> Date: Wed Nov 8 14:12:26 2017 +0000 Bug 10394: Don't autoupdate HTTPS Everywhere anymore We use the new --remove-update-channel switch to prevent HTTPS Everywhere from pinging update servers and updating automatically like we do with Torbutton and Tor Launcher. The new HTTPS Everywhere build file makes the workaround for bug 10066 obsolete as well. --- projects/https-everywhere/build | 11 ++--------- projects/https-everywhere/config | 2 +- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/projects/https-everywhere/build b/projects/https-everywhere/build index 71b2a21..49018af 100644 --- a/projects/https-everywhere/build +++ b/projects/https-everywhere/build @@ -2,15 +2,8 @@ [% c("var/set_default_env") -%] tar xf [% project %]-[% c('version') %].tar.gz cd [% project %]-[% c('version') %] -# patch makexpi.sh because we don't have a git repository -sed -i makexpi.sh -e 's/git rev-parse --short HEAD/echo aaaaaa/' -# XXX: Bloody hack to workaround a bug in HTTPS_E's git hash extraction in -# makexpi.sh. See https://trac.torproject.org/projects/tor/ticket/10066 -# The solution there does not work for us as doing something like -# |./makexpi.sh foo| to build tag |foo| is a) trying to download the sources -# during build time (again) and b) broken due to a certificate error. -rm -f .git/refs/heads/master -./makexpi.sh +# Bug 10394: We don't want to autoupdate HTTPS Everywhere anymore. +./make.sh --remove-update-channel # Since 5.0.2 a .xpi for AMO is built, too. We don't need it. rm pkg/*-amo.xpi mv pkg/*.xpi [% dest_dir _ '/' _ c('filename') %] diff --git a/projects/https-everywhere/config b/projects/https-everywhere/config index 99290e7..5225bbc 100644 --- a/projects/https-everywhere/config +++ b/projects/https-everywhere/config @@ -1,5 +1,5 @@ # vim: filetype=yaml sw=2 -version: 2017.10.4 +version: 2017.10.30 git_url: https://git.torproject.org/https-everywhere.git git_hash: '[% c("version") %]' git_submodule: 1

1 0

[tor/release-0.3.2] small edits to changelog
by nickm＠torproject.org 08 Nov '17

08 Nov '17

commit e50da3327f7c76c3f7254ff530ca37d43e228521 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Nov 8 10:38:44 2017 -0500 small edits to changelog --- ChangeLog | 37 +++++++++++++++++-------------------- 1 file changed, 17 insertions(+), 20 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7833490bf..2e796dd04 100644 --- a/ChangeLog +++ b/ChangeLog @@ -3,15 +3,13 @@ Changes in version 0.3.2.4-alpha - 2017-11-08 It fixes several bugs introduced in earlier releases, including stability and reliability fixes on earlier versions of Tor. - o Major bugfixes (Exit nodes): - - Fix an issue causing high-bandwidth exit nodes to fail a majority - or all of their DNS requests, making them basically unsuitable for - regular usage in Tor circuits. The problem is related to - libevent's DNS handling, but we can work around it in Tor. Fixes - bugs 21394 and 18580; bugfix on 0.1.2.2-alpha which introduced - eventdns. Credit goes to Dhalgren for identifying and finding a - workaround to this bug and to gamambel, arthuredelstein and arma - in helping to track it down and analyze it. + o Major bugfixes (exit nodes, DNS): + - Fix an issue causing DNS to fail on high-bandwidth exit nodes, + making them enarly unusable for regular. Fixes bugs 21394 and + 18580; bugfix on 0.1.2.2-alpha, which introduced eventdns. Thanks + to Dhalgren for identifying and finding a workaround to this bug + and to Moritz, Arthur Edelstein and Roger for helping to track it + down and analyze it. o Minor features (logging, scheduler): - Introduce a SCHED_BUG() function to log extra information about @@ -22,7 +20,7 @@ Changes in version 0.3.2.4-alpha - 2017-11-08 - The ClientDNSRejectInternalAddresses flag can once again be set in non-testing Tor networks, so long as they do not use the default directory authorities. This change also removes the deprecation of - this flag in 0.2.9.2-alpha. Closes ticket 21031. + this flag from 0.2.9.2-alpha. Closes ticket 21031. o Minor features (testing): - Our fuzzing tests now test the encrypted portions of the v3 hidden @@ -42,15 +40,15 @@ Changes in version 0.3.2.4-alpha - 2017-11-08 on 0.3.2.1-alpha. o Minor bugfixes (logging, relay): - - Downgrade a warning to a protocol warning in the case the ed25519 - key is not consistent between the descriptor and micro descriptor - of a relay. This can happen for instance if the relay has been - flagged NoEdConsensus. Fixes bug 24025; bugfix on 0.3.2.1-alpha. + - Give only a protocol warning when the ed25519 key is not + consistent between the descriptor and micro descriptor of a relay. + This can happen, for instance, if the relay has been flagged + NoEdConsensus. Fixes bug 24025; bugfix on 0.3.2.1-alpha. o Minor bugfixes (manpage, hidden service): - - Mention that the HiddenServiceNumIntroductionPoints option is 0-10 - for v2 service and 0-20 for v3 service. Fixes bug 24115; bugfix - on 0.3.2.1-alpha. + - Document that the HiddenServiceNumIntroductionPoints option is + 0-10 for v2 service and 0-20 for v3 service. Fixes bug 24115; + bugfix on 0.3.2.1-alpha. o Minor bugfixes (memory leak): - Fix a minor memory-leak-at-exit in the KIST scheduler. This bug @@ -64,9 +62,8 @@ Changes in version 0.3.2.4-alpha - 2017-11-08 - Cache some needed onion service client information instead of continuously computing it over and over again. Fixes bug 23623; bugfix on 0.3.2.1-alpha. - - Properly retry HSv3 descriptor fetches in the case where we were - initially missing required directory information. Fixes bug 23762; - bugfix on 0.3.2.1-alpha. + - Properly retry HSv3 descriptor fetches when missing required + directory information. Fixes bug 23762; bugfix on 0.3.2.1-alpha. o Minor bugfixes (relay, crash): - Avoid a crash when transitioning from client mode to bridge mode.

1 0

[tor/release-0.3.2] start on 0.3.2.4-alpha changelog
by nickm＠torproject.org 08 Nov '17

08 Nov '17

commit b4b2459de0420a30508e9447b75140b538ddf69b Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Nov 8 10:32:42 2017 -0500 start on 0.3.2.4-alpha changelog --- ChangeLog | 84 ++++++++++++++++++++++++++++++++++++++++++++++ changes/bug21394 | 9 ----- changes/bug23623 | 4 --- changes/bug23693 | 6 ---- changes/bug23753 | 4 --- changes/bug23762 | 4 --- changes/bug23774 | 4 --- changes/bug23816 | 6 ---- changes/bug23820 | 5 --- changes/bug24025 | 5 --- changes/bug24082 | 3 -- changes/bug24115 | 4 --- changes/bug24150 | 4 --- changes/hsdescv3_fuzz_more | 3 -- changes/ticket21031 | 7 ---- changes/ticket24109 | 4 --- 16 files changed, 84 insertions(+), 72 deletions(-) diff --git a/ChangeLog b/ChangeLog index 87517a2a5..7833490bf 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,87 @@ +Changes in version 0.3.2.4-alpha - 2017-11-08 + Tor 0.3.2.4-alpha is the fourth alpha release in the 0.3.2.x series. + It fixes several bugs introduced in earlier releases, including + stability and reliability fixes on earlier versions of Tor. + + o Major bugfixes (Exit nodes): + - Fix an issue causing high-bandwidth exit nodes to fail a majority + or all of their DNS requests, making them basically unsuitable for + regular usage in Tor circuits. The problem is related to + libevent's DNS handling, but we can work around it in Tor. Fixes + bugs 21394 and 18580; bugfix on 0.1.2.2-alpha which introduced + eventdns. Credit goes to Dhalgren for identifying and finding a + workaround to this bug and to gamambel, arthuredelstein and arma + in helping to track it down and analyze it. + + o Minor features (logging, scheduler): + - Introduce a SCHED_BUG() function to log extra information about + the scheduler state if we ever catch a bug in the scheduler. + Closes ticket 23753. + + o Minor features (removed deprecations): + - The ClientDNSRejectInternalAddresses flag can once again be set in + non-testing Tor networks, so long as they do not use the default + directory authorities. This change also removes the deprecation of + this flag in 0.2.9.2-alpha. Closes ticket 21031. + + o Minor features (testing): + - Our fuzzing tests now test the encrypted portions of the v3 hidden + service descriptors. Implements more of 21509. + + o Minor bugfixes (directory client): + - On failure to download directory information, delay retry attempts + by a random amount based on the "decorrelated jitter" algorithm. + Our previous delay algorithm tended to produce extra-long delays + too easily. Fixes bug 23816; bugfix on 0.2.9.1-alpha. + + o Minor bugfixes (IPv6, v3 single onion services): + - Remove buggy code for IPv6-only v3 single onion services, and + reject attempts to configure them. This release supports IPv4, + dual-stack, and IPv6-only v3 hidden services; and IPv4 and dual- + stack v3 single onion services. Fixes bug 23820; bugfix + on 0.3.2.1-alpha. + + o Minor bugfixes (logging, relay): + - Downgrade a warning to a protocol warning in the case the ed25519 + key is not consistent between the descriptor and micro descriptor + of a relay. This can happen for instance if the relay has been + flagged NoEdConsensus. Fixes bug 24025; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (manpage, hidden service): + - Mention that the HiddenServiceNumIntroductionPoints option is 0-10 + for v2 service and 0-20 for v3 service. Fixes bug 24115; bugfix + on 0.3.2.1-alpha. + + o Minor bugfixes (memory leak): + - Fix a minor memory-leak-at-exit in the KIST scheduler. This bug + should have no user-visible impact. Fixes bug 23774; bugfix + on 0.3.2.1-alpha. + - Fix a memory leak when decrypting a badly formatted v3 onion + service descriptor. Fixes bug 24150; bugfix on 0.3.2.1-alpha. + Found by OSS-Fuzz; this is OSS-Fuzz issue 3994. + + o Minor bugfixes (onion services): + - Cache some needed onion service client information instead of + continuously computing it over and over again. Fixes bug 23623; + bugfix on 0.3.2.1-alpha. + - Properly retry HSv3 descriptor fetches in the case where we were + initially missing required directory information. Fixes bug 23762; + bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (relay, crash): + - Avoid a crash when transitioning from client mode to bridge mode. + Previously, we would launch the worker threads whenever our + "public server" mode changed, but not when our "server" mode + changed. Fixes bug 23693; bugfix on 0.2.6.3-alpha. + + o Minor bugfixes (testing): + - Fix a spurious fuzzing-only use of an uninitialized value. Found + by Brian Carpenter. Fixes bug 24082; bugfix on 0.3.0.3-alpha. + - Test that IPv6-only clients can use microdescriptors when running + "make test-network-all". Requires chutney master 61c28b9 or later. + Closes ticket 24109. + + Changes in version 0.3.2.3-alpha - 2017-10-27 Tor 0.3.2.3-alpha is the third release in the 0.3.2 series. It fixes numerous small bugs in earlier versions of 0.3.2.x, and adds a new diff --git a/changes/bug21394 b/changes/bug21394 deleted file mode 100644 index e5452e20b..000000000 --- a/changes/bug21394 +++ /dev/null @@ -1,9 +0,0 @@ - o Major bugfixes (Exit nodes): - - Fix an issue causing high-bandwidth exit nodes to fail a majority - or all of their DNS requests, making them basically unsuitable for - regular usage in Tor circuits. The problem is related to - libevent's DNS handling, but we can work around it in Tor. Fixes - bugs 21394 and 18580; bugfix on 0.1.2.2-alpha which introduced - eventdns. Credit goes to Dhalgren for identifying and finding a - workaround to this bug and to gamambel, arthuredelstein and - arma in helping to track it down and analyze it. diff --git a/changes/bug23623 b/changes/bug23623 deleted file mode 100644 index 1e2e5c2ac..000000000 --- a/changes/bug23623 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (onion services): - - Cache some needed onion service client information instead of - continuously computing it over and over again. Fixes bug 23623; bugfix - on 0.3.2.1-alpha. diff --git a/changes/bug23693 b/changes/bug23693 deleted file mode 100644 index 796398be5..000000000 --- a/changes/bug23693 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (relay, crash): - - Avoid a crash when transitioning from client mode to bridge mode. - Previously, we would launch the worker threads whenever our "public - server" mode changed, but not when our "server" mode changed. - Fixes bug 23693; bugfix on 0.2.6.3-alpha. - diff --git a/changes/bug23753 b/changes/bug23753 deleted file mode 100644 index 8782a8e2d..000000000 --- a/changes/bug23753 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (logging, scheduler): - - Introduce a SCHED_BUG() function to log extra information about the - scheduler state if we ever catch a bug in the scheduler. Closes ticket - 23753. diff --git a/changes/bug23762 b/changes/bug23762 deleted file mode 100644 index 741a88e21..000000000 --- a/changes/bug23762 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (hidden service v3): - - Properly retry HSv3 descriptor fetches in the case where we were initially - missing required directory information. Fixes bug 23762; bugfix on - 0.3.2.1-alpha. diff --git a/changes/bug23774 b/changes/bug23774 deleted file mode 100644 index 2ea5c0122..000000000 --- a/changes/bug23774 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (memory leak): - - Fix a minor memory-leak-at-exit in the KIST scheduler. This - bug should have no user-visible impact. Fixes bug 23774; - bugfix on 0.3.2.1-alpha. diff --git a/changes/bug23816 b/changes/bug23816 deleted file mode 100644 index 6139dec9e..000000000 --- a/changes/bug23816 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (directory client): - - On failure to download directory information, delay retry attempts - by a random amount based on the "decorrelated jitter" algorithm. - Our previous delay algorithm tended to produce extra-long delays too - easily. Fixes bug 23816; bugfix on 0.2.9.1-alpha. - diff --git a/changes/bug23820 b/changes/bug23820 deleted file mode 100644 index 4e920d049..000000000 --- a/changes/bug23820 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (IPv6, v3 single onion services): - - Remove buggy code for IPv6-only v3 single onion services, and reject - attempts to configure them. This release supports IPv4, dual-stack, and - IPv6-only v3 hidden services; and IPv4 and dual-stack v3 single onion - services. Fixes bug 23820; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug24025 b/changes/bug24025 deleted file mode 100644 index 1d7841af5..000000000 --- a/changes/bug24025 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (logging, relay): - - Downgrade a warning to a protocol warning in the case the ed25519 key is - not consistent between the descriptor and micro descriptor of a relay. - This can happen for instance if the relay has been flagged - NoEdConsensus. Fixes bug 24025; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug24082 b/changes/bug24082 deleted file mode 100644 index 152323935..000000000 --- a/changes/bug24082 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (testing): - - Fix a spurious fuzzing-only use of an uninitialized value. - Found by Brian Carpenter. Fixes bug 24082; bugfix on 0.3.0.3-alpha. diff --git a/changes/bug24115 b/changes/bug24115 deleted file mode 100644 index 767f13840..000000000 --- a/changes/bug24115 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (manpage, hidden service): - - Mention that the HiddenServiceNumIntroductionPoints option is 0-10 for - v2 service and 0-20 for v3 service. Fixes bug 24115; bugfix on - 0.3.2.1-alpha. diff --git a/changes/bug24150 b/changes/bug24150 deleted file mode 100644 index cfda7c40d..000000000 --- a/changes/bug24150 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (v3 onion services): - - Fix a memory leak when decrypting a badly formatted v3 onion - service descriptor. Fixes bug 24150; bugfix on 0.3.2.1-alpha. - Found by OSS-Fuzz; this is OSS-Fuzz issue 3994. diff --git a/changes/hsdescv3_fuzz_more b/changes/hsdescv3_fuzz_more deleted file mode 100644 index 25626bb9a..000000000 --- a/changes/hsdescv3_fuzz_more +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features (testing): - - Our fuzzing tests now test the encrypted portions of the - v3 hidden service descriptors. Implements more of 21509. diff --git a/changes/ticket21031 b/changes/ticket21031 deleted file mode 100644 index b081fb018..000000000 --- a/changes/ticket21031 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor features (removed deprecations): - - The ClientDNSRejectInternalAddresses flag can once again be set in - non-testing Tor networks, so long as they do not use the default - directory authorities. - This change also removes the deprecation of this - flag in 0.2.9.2-alpha. Closes ticket 21031. - diff --git a/changes/ticket24109 b/changes/ticket24109 deleted file mode 100644 index f66271817..000000000 --- a/changes/ticket24109 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (integration tests): - - Test that IPv6-only clients can use microdescriptors when running - "make test-network-all". Requires chutney master 61c28b9 or later. - Closes ticket 24109.

1 0

[torflow/master] Removed unneeded imports from BwAuthority
by tom＠torproject.org 08 Nov '17

08 Nov '17

commit 8336de82bc451dcbb169c06e95dfa3795f81e744 Author: Edmund Wong <ewongbb(a)pw-wspx.org> Date: Mon Oct 30 12:27:46 2017 +0800 Removed unneeded imports from BwAuthority --- NetworkScanners/BwAuthority/aggregate.py | 1 - NetworkScanners/BwAuthority/bwauthority_child.py | 2 -- NetworkScanners/BwAuthority/extras/parsedireq.py | 1 - NetworkScanners/BwAuthority/extras/parseplatform.py | 2 -- 4 files changed, 6 deletions(-) diff --git a/NetworkScanners/BwAuthority/aggregate.py b/NetworkScanners/BwAuthority/aggregate.py index 5e91bcf..c4326fd 100755 --- a/NetworkScanners/BwAuthority/aggregate.py +++ b/NetworkScanners/BwAuthority/aggregate.py @@ -10,7 +10,6 @@ import traceback sys.path.append("../../") from TorCtl.TorUtil import plog from TorCtl import TorCtl,TorUtil -from TorCtl.PathSupport import VersionRangeRestriction, NodeRestrictionList, NotNodeRestriction bw_files = [] nodes = {} diff --git a/NetworkScanners/BwAuthority/bwauthority_child.py b/NetworkScanners/BwAuthority/bwauthority_child.py index 3ae484d..6db2726 100755 --- a/NetworkScanners/BwAuthority/bwauthority_child.py +++ b/NetworkScanners/BwAuthority/bwauthority_child.py @@ -16,8 +16,6 @@ import sys import urllib2 import os import traceback -import copy -import shutil import threading import ConfigParser import sqlalchemy diff --git a/NetworkScanners/BwAuthority/extras/parsedireq.py b/NetworkScanners/BwAuthority/extras/parsedireq.py index 1920da7..1cb5774 100755 --- a/NetworkScanners/BwAuthority/extras/parsedireq.py +++ b/NetworkScanners/BwAuthority/extras/parsedireq.py @@ -14,7 +14,6 @@ # n-v2-ip import time -import math import sys def total_countries(l): diff --git a/NetworkScanners/BwAuthority/extras/parseplatform.py b/NetworkScanners/BwAuthority/extras/parseplatform.py index 17f9cf0..dbdff7f 100755 --- a/NetworkScanners/BwAuthority/extras/parseplatform.py +++ b/NetworkScanners/BwAuthority/extras/parseplatform.py @@ -4,8 +4,6 @@ # distribution parameters based on this data. import time -import math - import logistic f = open("platforms.csv", "r")

1 0

[torflow/master] Remove the FastFirstHopPK option, which is a deprecated option. The value we used is the default (bwauths were an early adoptor) Closes #21393
by tom＠torproject.org 08 Nov '17

08 Nov '17

commit 1288389e699c03702aeae06b84d3c7905fe98387 Author: Tom Ritter <tom(a)ritter.vg> Date: Wed Nov 8 09:07:55 2017 -0600 Remove the FastFirstHopPK option, which is a deprecated option. The value we used is the default (bwauths were an early adoptor) Closes #21393 --- NetworkScanners/BwAuthority/data/tor.1/torrc | 4 ---- NetworkScanners/BwAuthority/data/tor.2/torrc | 4 ---- 2 files changed, 8 deletions(-) diff --git a/NetworkScanners/BwAuthority/data/tor.1/torrc b/NetworkScanners/BwAuthority/data/tor.1/torrc index 95f8bd4..10cf915 100644 --- a/NetworkScanners/BwAuthority/data/tor.1/torrc +++ b/NetworkScanners/BwAuthority/data/tor.1/torrc @@ -13,9 +13,5 @@ __LeaveStreamsUnattached 1 FetchDirInfoExtraEarly 1 FetchDirInfoEarly 1 -# Use CREATE cells for first hop, so we can still measure CPU overload via -# circ failure -FastFirstHopPK 0 - # Turn pathbias off, so it doesn't flood the logs UseEntryGuards 0 diff --git a/NetworkScanners/BwAuthority/data/tor.2/torrc b/NetworkScanners/BwAuthority/data/tor.2/torrc index 0232163..979a687 100755 --- a/NetworkScanners/BwAuthority/data/tor.2/torrc +++ b/NetworkScanners/BwAuthority/data/tor.2/torrc @@ -13,9 +13,5 @@ __LeaveStreamsUnattached 1 FetchDirInfoExtraEarly 1 FetchDirInfoEarly 1 -# Use CREATE cells for first hop, so we can still measure CPU overload via -# circ failure -FastFirstHopPK 0 - # Turn pathbias off, so it doesn't flood the logs UseEntryGuards 0

1 0

[tor/master] Replace our random-exponential-delay algorithm.
by nickm＠torproject.org 08 Nov '17

08 Nov '17

commit cb29687e93169e9615de3bc5adcbf6d31551b32c Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Oct 17 13:24:40 2017 -0400 Replace our random-exponential-delay algorithm. This patch has implementations of the "decorrelated" and "full" algorithms from https://www.awsarchitectureblog.com/2015/03/backoff.html --- changes/bug23816 | 6 +++++ src/or/directory.c | 77 ++++++++++++++++++++++++++++++++--------------------- src/or/directory.h | 10 +++++-- src/test/test_dir.c | 42 ++++++++++++++++++++++++++--- 4 files changed, 98 insertions(+), 37 deletions(-) diff --git a/changes/bug23816 b/changes/bug23816 new file mode 100644 index 000000000..6139dec9e --- /dev/null +++ b/changes/bug23816 @@ -0,0 +1,6 @@ + o Minor bugfixes (directory client): + - On failure to download directory information, delay retry attempts + by a random amount based on the "decorrelated jitter" algorithm. + Our previous delay algorithm tended to produce extra-long delays too + easily. Fixes bug 23816; bugfix on 0.2.9.1-alpha. + diff --git a/src/or/directory.c b/src/or/directory.c index fce48c6e9..758a29b05 100644 --- a/src/or/directory.c +++ b/src/or/directory.c @@ -3776,53 +3776,68 @@ find_dl_min_and_max_delay(download_status_t *dls, const or_options_t *options, *max = INT_MAX; } -/** Advance one delay step. The algorithm is to use the previous delay to - * compute an increment, we construct a value uniformly at random between - * delay and MAX(delay*2,delay+1). We then clamp that value to be no larger - * than max_delay, and return it. +/** As next_random_exponential_delay() below, but does not compute a random + * value. Instead, compute the range of values that + * next_random_exponential_delay() should use when computing its random value. + * Store the low bound into *low_bound_out, and the high bound into + * *high_bound_out. Guarantees that the low bound is strictly less + * than the high bound. */ +STATIC void +next_random_exponential_delay_range(int *low_bound_out, + int *high_bound_out, + int delay, + int base_delay) +{ + // This is the "decorrelated jitter" approach, from + // https://www.awsarchitectureblog.com/2015/03/backoff.html + // The formula is + // sleep = min(cap, random_between(base, sleep * 3)) + + const int delay_times_3 = delay < INT_MAX/3 ? delay * 3 : INT_MAX; + *low_bound_out = base_delay; + if (delay_times_3 > base_delay) { + *high_bound_out = delay_times_3; + } else { + *high_bound_out = base_delay+1; + } +} + +/** Advance one delay step. The algorithm will generate a random delay, + * such that each failure is possibly (random) longer than the ones before. + * + * We then clamp that value to be no larger than max_delay, and return it. + * + * The base_delay parameter is lowest possible delay time (can't be + * zero); the backoff_position parameter is the number of times we've + * generated a delay; and the delay argument is the most recently used + * delay. * * Requires that delay is less than INT_MAX, and delay is in [0,max_delay]. */ STATIC int -next_random_exponential_delay(int delay, int max_delay) +next_random_exponential_delay(int delay, + int base_delay, + int max_delay) { /* Check preconditions */ if (BUG(max_delay < 0)) max_delay = 0; if (BUG(delay > max_delay)) delay = max_delay; - if (delay == INT_MAX) - return INT_MAX; /* prevent overflow */ if (BUG(delay < 0)) delay = 0; - /* How much are we willing to add to the delay? */ - int max_increment; - int multiplier = 3; /* no more than quadruple the previous delay */ - if (get_options()->TestingTorNetwork) { - /* Decrease the multiplier in testing networks. This reduces the variance, - * so that bootstrap is more reliable. */ - multiplier = 2; /* no more than triple the previous delay */ - } + if (base_delay < 1) + base_delay = 1; - if (delay && delay < (INT_MAX-1) / multiplier) { - max_increment = delay * multiplier; - } else if (delay) { - max_increment = INT_MAX-1; - } else { - max_increment = 1; - } + int low_bound=0, high_bound=max_delay; - if (BUG(max_increment < 1)) - max_increment = 1; + next_random_exponential_delay_range(&low_bound, &high_bound, + delay, base_delay); - /* the + 1 here is so that we always wait longer than last time. */ - int increment = crypto_rand_int(max_increment)+1; + int rand_delay = crypto_rand_int_range(low_bound, high_bound); - if (increment < max_delay - delay) - return delay + increment; - else - return max_delay; + return MIN(rand_delay, max_delay); } /** Find the current delay for dls based on schedule or min_delay/ @@ -3871,7 +3886,7 @@ download_status_schedule_get_delay(download_status_t *dls, while (dls->last_backoff_position < dls_schedule_position) { /* Do one increment step */ - delay = next_random_exponential_delay(delay, max_delay); + delay = next_random_exponential_delay(delay, min_delay, max_delay); /* Update our position */ ++(dls->last_backoff_position); } diff --git a/src/or/directory.h b/src/or/directory.h index 629b3ead9..c055e60d9 100644 --- a/src/or/directory.h +++ b/src/or/directory.h @@ -167,8 +167,14 @@ STATIC const smartlist_t *find_dl_schedule(download_status_t *dls, STATIC void find_dl_min_and_max_delay(download_status_t *dls, const or_options_t *options, int *min, int *max); -STATIC int next_random_exponential_delay(int delay, int max_delay); - +STATIC int next_random_exponential_delay(int delay, + int base_delay, + int max_delay); + +STATIC void next_random_exponential_delay_range(int *low_bound_out, + int *high_bound_out, + int delay, + int base_delay); #endif #endif diff --git a/src/test/test_dir.c b/src/test/test_dir.c index cdc56acb8..1c6662147 100644 --- a/src/test/test_dir.c +++ b/src/test/test_dir.c @@ -3617,6 +3617,7 @@ test_dir_download_status_random_backoff(void *arg) /* Check the random backoff cases */ old_increment = 0; + int n_attempts = 0; do { increment = download_status_schedule_get_delay(&dls_random, NULL, @@ -3625,9 +3626,9 @@ test_dir_download_status_random_backoff(void *arg) /* Test */ tt_int_op(increment, OP_GE, min_delay); tt_int_op(increment, OP_LE, max_delay); - tt_int_op(increment, OP_GE, old_increment); - /* We at most quadruple, and maybe add one */ - tt_int_op(increment, OP_LE, 4 * old_increment + 1); + + if (old_increment) + tt_int_op(increment, OP_LE, old_increment * 3); /* Advance */ current_time += increment; @@ -3636,13 +3637,45 @@ test_dir_download_status_random_backoff(void *arg) /* Try another maybe */ old_increment = increment; - } while (increment < max_delay); + } while (increment < max_delay && ++n_attempts < 1000); done: return; } static void +test_dir_download_status_random_backoff_ranges(void *arg) +{ + (void)arg; + int lo, hi; + next_random_exponential_delay_range(&lo, &hi, 0, 10); + tt_int_op(lo, OP_EQ, 10); + tt_int_op(hi, OP_EQ, 11); + + next_random_exponential_delay_range(&lo, &hi, 6, 10); + tt_int_op(lo, OP_EQ, 10); + tt_int_op(hi, OP_EQ, 6*3); + + next_random_exponential_delay_range(&lo, &hi, 13, 10); + tt_int_op(lo, OP_EQ, 10); + tt_int_op(hi, OP_EQ, 13 * 3); + + next_random_exponential_delay_range(&lo, &hi, 37, 10); + tt_int_op(lo, OP_EQ, 10); + tt_int_op(hi, OP_EQ, 111); + + next_random_exponential_delay_range(&lo, &hi, 123, 10); + tt_int_op(lo, OP_EQ, 10); + tt_int_op(hi, OP_EQ, 369); + + next_random_exponential_delay_range(&lo, &hi, INT_MAX-5, 10); + tt_int_op(lo, OP_EQ, 10); + tt_int_op(hi, OP_EQ, INT_MAX); + done: + ; +} + +static void test_dir_download_status_increment(void *arg) { (void)arg; @@ -5475,6 +5508,7 @@ struct testcase_t dir_tests[] = { DIR(packages, 0), DIR(download_status_schedule, 0), DIR(download_status_random_backoff, 0), + DIR(download_status_random_backoff_ranges, 0), DIR(download_status_increment, 0), DIR(authdir_type_to_string, 0), DIR(conn_purpose_to_string, 0),

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits November 2017