July 2016 - tor-commits - lists.torproject.org

[webwml/master] Merge https://github.com/isabelabagueros/tpo
by sebastian＠torproject.org 18 Jul '16

18 Jul '16

commit e4ec0c33c817eb50e3449919fe4a20c8f9188fa9 Merge: c4cd913 1a75d7f Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Tue Jul 19 00:46:13 2016 +0200 Merge https://github.com/isabelabagueros/tpo If you don't like "Tor is a free software that" provide something that's shorter and better :) include/head.wmi | 1 + 1 file changed, 1 insertion(+)

1 0

[webwml/master] Merge pull request #1 from isabelabagueros/description
by sebastian＠torproject.org 18 Jul '16

18 Jul '16

commit 8f2b64ac18da6e6ff27a5f7e3e2490f66ee5d247 Merge: c4cd913 870750e Author: isabelabagueros <isabelabagueros(a)gmail.com> Date: Sat Jul 16 17:42:22 2016 -0700 Merge pull request #1 from isabelabagueros/description adding gettor info on meta description for folks who can't open our site include/head.wmi | 1 + 1 file changed, 1 insertion(+)

1 0

[webwml/master] adding gettor info on meta description for folks who can't open our website
by sebastian＠torproject.org 18 Jul '16

18 Jul '16

commit 870750e14de3155782c8e9a19ab2563d072c008d Author: toya <toya(a)muamba.muamba> Date: Sat Jul 16 17:42:33 2016 -0700 adding gettor info on meta description for folks who can't open our website --- include/head.wmi | 1 + 1 file changed, 1 insertion(+) diff --git a/include/head.wmi b/include/head.wmi index f354839..15eee5b 100644 --- a/include/head.wmi +++ b/include/head.wmi @@ -14,6 +14,7 @@ <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="author" content="The Tor Project, Inc."> + <meta name="description" content="A free software implementation of second-generation onion routing, a system enabling its users to communicate anonymously on the Internet. If you can't open the website, email gettor(a)torproject.org for instruction on how to get the Tor Browser."> <meta name="keywords" content="anonymity online, tor, tor project, censorship circumvention, traffic analysis, anonymous communications research"> <meta property="og:image" content="https://www.torproject.org/images/tor-logo.jpg">

1 0

[webwml/master] Merge pull request #2 from isabelabagueros/new_description
by sebastian＠torproject.org 18 Jul '16

18 Jul '16

commit 1a75d7f83aa0785561d8bda3e8dae6864fa455bf Merge: 8f2b64a 3900758 Author: isabelabagueros <isabelabagueros(a)gmail.com> Date: Sat Jul 16 18:00:35 2016 -0700 Merge pull request #2 from isabelabagueros/new_description changed for an easier phrase include/head.wmi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

1 0

[chutney/master] Allow tor and tor-gencert to be specified on the chutney command-line
by teor＠torproject.org 18 Jul '16

18 Jul '16

commit 1ac58478c113b40c17d3bd43ebe00639d419eb51 Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Mon Jul 18 13:15:40 2016 +1000 Allow tor and tor-gencert to be specified on the chutney command-line Now works via --tor and --tor-gencert. Overridden by the build directory set via --tor-path and $TOR_DIR. --- README | 6 +++++- tools/test-network.sh | 16 +++++++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/README b/README index 826e5dd..0ae9380 100644 --- a/README +++ b/README @@ -19,8 +19,12 @@ Stuff to try: Automated Setup, Verification, and Shutdown: ./tools/test-network.sh --flavor basic-min - ./tools/test-network.sh --tor-path <tor-build-directory> ./tools/test-network.sh --coverage + ./tools/test-network.sh --tor-path <tor-build-directory> + ./tools/test-network.sh --tor <name-or-path> --tor-gencert <name-or-path> + (--tor-path and $TOR_DIR override --tor and --tor-gencert.) + +The environmental variables documented below also work with test-network.sh. Standard Actions: ./chutney configure networks/basic diff --git a/tools/test-network.sh b/tools/test-network.sh index 3cd6092..2e0cd7c 100755 --- a/tools/test-network.sh +++ b/tools/test-network.sh @@ -13,9 +13,21 @@ do shift ;; --tor-path) + # the path of a tor build directory + # --tor-path overrides --tor and --tor-gencert export TOR_DIR="$2" shift ;; + --tor) + # the name or path of a tor binary + export CHUTNEY_TOR="$2" + shift + ;; + --tor-gencert) + # the name or path of a tor-gencert binary + export CHUTNEY_TOR_GENCERT="$2" + shift + ;; --flavor|--flavour|--network-flavor|--network-flavour) export NETWORK_FLAVOUR="$2" shift @@ -113,7 +125,9 @@ CHUTNEY_PATH=\`pwd\`/chutney" fi # For picking up the right tor binaries. -# If these varibles aren't set, chutney looks for tor binaries in $PATH +# If $TOR_DIR isn't set, chutney looks for tor binaries by name or path +# using $CHUTNEY_TOR and $CHUTNEY_TOR_GENCERT, and then falls back to +# looking for tor and tor-gencert in $PATH if [ -d "$TOR_DIR" ]; then tor_name=tor tor_gencert_name=tor-gencert

1 0

[chutney/master] Fix some typos in test-network.sh
by teor＠torproject.org 18 Jul '16

18 Jul '16

commit 6bd1d12e1ebce8e487393699a8ef83bc4eaecab6 Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Mon Jul 18 13:07:09 2016 +1000 Fix some typos in test-network.sh --- tools/test-network.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/test-network.sh b/tools/test-network.sh index 38a0477..3cd6092 100755 --- a/tools/test-network.sh +++ b/tools/test-network.sh @@ -55,7 +55,7 @@ do *) echo "$myname: Sorry, I don't know what to do with '$1'." # continue processing arguments during a dry run - if [ "$NETWORK_DRY_RUN" != true]; then + if [ "$NETWORK_DRY_RUN" != true ]; then exit 2 fi ;; @@ -141,7 +141,7 @@ cd "$CHUTNEY_PATH" CHUTNEY_BOOTSTRAP_TIME=${CHUTNEY_BOOTSTRAP_TIME:-60} # but even the fastest tor networks take 5 seconds for their first consensus # and then 10 seconds after that for relays to bootstrap and upload descriptors -echo "Waiting 15 seconds for a consensus containing relaysto be generated..." +echo "Waiting 15 seconds for a consensus containing relays to be generated..." sleep 15 ./chutney verify $CHUTNEY_NETWORK VERIFY_EXIT_STATUS=$?

1 0

[chutney/master] Don't burn through a client's HSDirs too fast in hs-min
by teor＠torproject.org 18 Jul '16

18 Jul '16

commit aad22037a2498697b81fdf74c9ece4dc21a08d30 Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Mon Jul 18 12:53:46 2016 +1000 Don't burn through a client's HSDirs too fast in hs-min --- lib/chutney/Traffic.py | 7 ++++--- networks/hs-min | 4 +++- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/lib/chutney/Traffic.py b/lib/chutney/Traffic.py index c118289..881cac7 100644 --- a/lib/chutney/Traffic.py +++ b/lib/chutney/Traffic.py @@ -291,11 +291,12 @@ class Source(Peer): else: debug("BUG: sent no bytes") self._sent_no_bytes += 1 - if self._sent_no_bytes >= 10: + # We can't retry too fast, otherwise clients burn all their HSDirs + if self._sent_no_bytes >= 3: print("Send no data %d times. Stalled." % (self._sent_no_bytes)) - sys.exit(-1) - time.sleep(1) + return -1 + time.sleep(3) self.outbuf = self.outbuf[n:] if self.state == self.CONNECTING_THROUGH_PROXY: return 1 # Keep us around. diff --git a/networks/hs-min b/networks/hs-min index e7b7403..73873a8 100644 --- a/networks/hs-min +++ b/networks/hs-min @@ -11,7 +11,9 @@ HS = Node(tag="h", hs=1, torrc="hs.tmpl") # connections: # a minimum path length of 3, plus the client-nominated rendezvous point, # plus a seperate introduction point -NODES = Authority.getN(3) + NonExitRelay.getN(2) + \ +# But we use 6 HSDirs: when the hidden service is not up, the client fails to +# find the descriptor at the first 3-5 HSDirs +NODES = Authority.getN(3) + NonExitRelay.getN(3) + \ Client.getN(1) + HS.getN(1) ConfigureNodes(NODES)

1 0

[chutney/master] Make chutney verify retry until success
by teor＠torproject.org 18 Jul '16

18 Jul '16

commit 1592d53a91a4b9421ad9eb5567ba3028a0b85fe4 Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Mon Jul 18 12:51:42 2016 +1000 Make chutney verify retry until success Return immediately on success. Only return failure after 60 seconds with no successes. This makes networks that bootstrap quickly verify much faster, and networks that bootstrap slowly are much more reliable. --- README | 10 ++++++++++ lib/chutney/TorNet.py | 7 +++++++ scripts/chutney_tests/verify.py | 16 ++++++++++++++-- tools/test-network.sh | 15 +++++++-------- 4 files changed, 38 insertions(+), 10 deletions(-) diff --git a/README b/README index f945d91..826e5dd 100644 --- a/README +++ b/README @@ -64,6 +64,16 @@ HS Connection Tests: # Default behavior is one client connects to each HS ./chutney stop networks/hs-025 +Waiting for the network to bootstrap: + + Commands like "chutney verify" start immediately, and keep trying for + CHUTNEY_BOOTSTRAP_TIME seconds. If they haven't been successful after that + time, they fail. + + The tools/test-network.sh script waits 15 seconds before calling chutney + verify, because that's the minimum amount of time it takes to bootstrap a + consensus containing relays. + Changing the network address: Chutney defaults to binding to localhost. To change the bind address, diff --git a/lib/chutney/TorNet.py b/lib/chutney/TorNet.py index 87c5dab..12fc19e 100644 --- a/lib/chutney/TorNet.py +++ b/lib/chutney/TorNet.py @@ -700,6 +700,13 @@ DEFAULTS = { # to connect to each HS? # (Clients choose an exit at random, so this doesn't apply to exits.) 'hs_multi_client': int(os.environ.get('CHUTNEY_HS_MULTI_CLIENT', 0)), + # How long should verify (and similar commands) wait for a successful + # outcome? (seconds) + # We check BOOTSTRAP_TIME for compatibility with old versions of + # test-network.sh + 'bootstrap_time': int(os.environ.get('CHUTNEY_BOOTSTRAP_TIME', + os.environ.get('BOOTSTRAP_TIME', + 60))), } diff --git a/scripts/chutney_tests/verify.py b/scripts/chutney_tests/verify.py index 480cf97..8734b60 100644 --- a/scripts/chutney_tests/verify.py +++ b/scripts/chutney_tests/verify.py @@ -3,8 +3,20 @@ import chutney def run_test(network): - print("Verifying data transmission:") - status = _verify_traffic(network) + wait_time = network._dfltEnv['bootstrap_time'] + start_time = time.time() + end_time = start_time + wait_time + print("Verifying data transmission: (retrying for up to %d seconds)" + % wait_time) + status = False + # Keep on retrying the verify until it succeeds or times out + while not status and time.time() < end_time: + # TrafficTester connections time out after ~3 seconds + # a TrafficTester times out after ~10 seconds if no data is being sent + status = _verify_traffic(network) + # Avoid madly spewing output if we fail immediately each time + if not status: + time.sleep(2) print("Transmission: %s" % ("Success" if status else "Failure")) if not status: # TODO: allow the debug flag to be passed as an argument to diff --git a/tools/test-network.sh b/tools/test-network.sh index 6707e2d..38a0477 100755 --- a/tools/test-network.sh +++ b/tools/test-network.sh @@ -21,7 +21,7 @@ do shift ;; --delay|--sleep|--bootstrap-time|--time) - export BOOTSTRAP_TIME="$2" + export CHUTNEY_BOOTSTRAP_TIME="$2" shift ;; # Environmental variables used by chutney verify performance tests @@ -137,13 +137,12 @@ fi cd "$CHUTNEY_PATH" ./tools/bootstrap-network.sh $NETWORK_FLAVOUR || exit 2 -# Sleep some, waiting for the network to bootstrap. -# TODO: Add chutney command 'bootstrap-status' and use that instead. -BOOTSTRAP_TIME=${BOOTSTRAP_TIME:-35} -$ECHO_N "$myname: sleeping for $BOOTSTRAP_TIME seconds" -n=$BOOTSTRAP_TIME; while [ $n -gt 0 ]; do - sleep 1; n=$(expr $n - 1); $ECHO_N . -done; echo "" +# chutney verify starts immediately, and keeps on trying for 60 seconds +CHUTNEY_BOOTSTRAP_TIME=${CHUTNEY_BOOTSTRAP_TIME:-60} +# but even the fastest tor networks take 5 seconds for their first consensus +# and then 10 seconds after that for relays to bootstrap and upload descriptors +echo "Waiting 15 seconds for a consensus containing relaysto be generated..." +sleep 15 ./chutney verify $CHUTNEY_NETWORK VERIFY_EXIT_STATUS=$? # work around a bug/feature in make -j2 (or more)

1 0

[chutney/master] Apparently time.time() is a more reliable time.clock()
by teor＠torproject.org 18 Jul '16

18 Jul '16

commit 89e8ec1cc2f3f346cf3dae071486f4f03f1c6799 Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Mon Jul 18 12:54:39 2016 +1000 Apparently time.time() is a more reliable time.clock() --- scripts/chutney_tests/verify.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/chutney_tests/verify.py b/scripts/chutney_tests/verify.py index 8734b60..ac601c9 100644 --- a/scripts/chutney_tests/verify.py +++ b/scripts/chutney_tests/verify.py @@ -94,9 +94,9 @@ def _verify_traffic(network): LISTEN_PORT, connection_count, network._dfltEnv['hs_multi_client']) print("Transmitting Data:") - start_time = time.clock() + start_time = time.time() status = tt.run() - end_time = time.clock() + end_time = time.time() # if we fail, don't report the bandwidth if not status: return status

1 0

[chutney/master] We fixed #15937, update the README
by teor＠torproject.org 18 Jul '16

18 Jul '16

commit 595fdecb3588aa5cd37ece3fc76df084b70df35f Author: teor (Tim Wilson-Brown) <teor2345(a)gmail.com> Date: Mon Jul 18 12:48:51 2016 +1000 We fixed #15937, update the README --- README | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README b/README index 97aad0a..f945d91 100644 --- a/README +++ b/README @@ -52,8 +52,8 @@ Connection Tests: ./chutney stop networks/basic-025 Note: If you create 7 or more connections to a hidden service from a single -client, you'll likely get a verification failure due to -https://trac.torproject.org/projects/tor/ticket/15937 +Tor 0.2.7 client, you'll likely get a verification failure due to #15937. +This is fixed in 0.2.8. HS Connection Tests: ./chutney configure networks/hs-025

1 0