July 2017 - tor-commits - lists.torproject.org

[stem/master] Test HiddenServiceNonAnonymousMode and HiddenServiceSingleHopMode must match
by atagar＠torproject.org 06 Jul '17

06 Jul '17

commit d6378fabc7caac3e1362684f98e9b0ad25c87072 Author: Damian Johnson <atagar(a)torproject.org> Date: Thu Jul 6 09:26:20 2017 -0700 Test HiddenServiceNonAnonymousMode and HiddenServiceSingleHopMode must match Another simple ask on https://trac.torproject.org/projects/tor/ticket/20245 --- test/integ/process.py | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/test/integ/process.py b/test/integ/process.py index 10f74a1..a7e2bc3 100644 --- a/test/integ/process.py +++ b/test/integ/process.py @@ -336,6 +336,33 @@ class TestProcess(unittest.TestCase): assert_in('[notice] Configuration file "/path/that/really/shouldnt/exist" not present, using reasonable defaults.', output, 'Missing torrc should be allowed with --ignore-missing-torrc') @asynchronous + def test_unanonymous_hidden_service_config_must_match(tor_cmd): + """ + Checking that startup fails if HiddenServiceNonAnonymousMode and + HiddenServiceSingleHopMode don't match. + """ + + try: + stem.process.launch_tor_with_config( + tor_cmd = tor_cmd, + config = {'HiddenServiceNonAnonymousMode': '1'}, + ) + + raise AssertionError("Tor shouldn't start with 'HiddenServiceNonAnonymousMode' set but not 'HiddenServiceSingleHopMode'") + except OSError as exc: + assert_equal('Process terminated: HiddenServiceNonAnonymousMode does not provide any server anonymity. It must be used with HiddenServiceSingleHopMode set to 1.', str(exc)) + + try: + stem.process.launch_tor_with_config( + tor_cmd = tor_cmd, + config = {'HiddenServiceSingleHopMode': '1'}, + ) + + raise AssertionError("Tor shouldn't start with 'HiddenServiceSingleHopMode' set but not 'HiddenServiceNonAnonymousMode'") + except OSError as exc: + assert_equal('Process terminated: HiddenServiceSingleHopMode does not provide any server anonymity. It must be used with HiddenServiceNonAnonymousMode set to 1.', str(exc)) + + @asynchronous def test_can_run_multithreaded(tor_cmd): """ Our launch_tor() function uses signal to support its timeout argument.

1 0

[stem/master] Update CollecTor links
by atagar＠torproject.org 06 Jul '17

06 Jul '17

commit c4abd00933fd176e70d968bdec7639d0714c7614 Author: Damian Johnson <atagar(a)torproject.org> Date: Thu Jul 6 08:22:56 2017 -0700 Update CollecTor links CollecTor's home just changed. Updating our links so they still work. --- stem/descriptor/__init__.py | 4 ++-- stem/descriptor/extrainfo_descriptor.py | 5 ++--- stem/descriptor/networkstatus.py | 2 +- stem/descriptor/server_descriptor.py | 7 +++---- 4 files changed, 8 insertions(+), 10 deletions(-) diff --git a/stem/descriptor/__init__.py b/stem/descriptor/__init__.py index e61d205..66dc573 100644 --- a/stem/descriptor/__init__.py +++ b/stem/descriptor/__init__.py @@ -122,7 +122,7 @@ def parse_file(descriptor_file, descriptor_type = None, validate = False, docume tries to determine the descriptor type based on the following... * The @type annotation on the first line. These are generally only found in - the `CollecTor archives <https://collector.torproject.org/formats.html#relay-descriptors>`_. + the `CollecTor archives <https://metrics.torproject.org/collector.html#relay-descriptors>`_. * The filename if it matches something from tor's data directory. For instance, tor's 'cached-descriptors' contains server descriptors. @@ -166,7 +166,7 @@ def parse_file(descriptor_file, descriptor_type = None, validate = False, docume my_descriptor_file = open(descriptor_path, 'rb') :param str,file,tarfile descriptor_file: path or opened file with the descriptor contents - :param str descriptor_type: `descriptor type <https://collector.torproject.org/formats.html>`_, this is guessed if not provided + :param str descriptor_type: `descriptor type <https://metrics.torproject.org/collector.html#data-formats>`_, this is guessed if not provided :param bool validate: checks the validity of the descriptor's content if **True**, skips these checks otherwise :param stem.descriptor.__init__.DocumentHandler document_handler: method in diff --git a/stem/descriptor/extrainfo_descriptor.py b/stem/descriptor/extrainfo_descriptor.py index bfea309..3d6770f 100644 --- a/stem/descriptor/extrainfo_descriptor.py +++ b/stem/descriptor/extrainfo_descriptor.py @@ -19,8 +19,7 @@ Extra-info descriptors are available from a few sources... * control port via 'GETINFO extra-info/digest/\*' queries * the 'cached-extrainfo' file in tor's data directory -* Archived descriptors provided by CollecTor - (https://collector.torproject.org/) +* Archived descriptors provided by `CollecTor <https://metrics.torproject.org/collector.html>`_. * Directory authorities and mirrors via their DirPort. @@ -997,7 +996,7 @@ class RelayExtraInfoDescriptor(ExtraInfoDescriptor): class BridgeExtraInfoDescriptor(ExtraInfoDescriptor): """ Bridge extra-info descriptor (`bridge descriptor specification - <https://collector.torproject.org/formats.html#bridge-descriptors>`_) + <https://metrics.torproject.org/collector.html#bridge-descriptors>`_) :var str ed25519_certificate_hash: sha256 hash of the original identity-ed25519 :var str router_digest_sha256: sha256 digest of this document diff --git a/stem/descriptor/networkstatus.py b/stem/descriptor/networkstatus.py index 1c0acab..9e7b74b 100644 --- a/stem/descriptor/networkstatus.py +++ b/stem/descriptor/networkstatus.py @@ -9,7 +9,7 @@ Documents can be obtained from a few sources... * The 'cached-consensus' file in Tor's data directory. * Archived descriptors provided by `CollecTor - <https://collector.torproject.org/>`_. + <https://metrics.torproject.org/collector.html>`_. * Directory authorities and mirrors via their DirPort. diff --git a/stem/descriptor/server_descriptor.py b/stem/descriptor/server_descriptor.py index 2e1c776..8489b33 100644 --- a/stem/descriptor/server_descriptor.py +++ b/stem/descriptor/server_descriptor.py @@ -10,8 +10,7 @@ etc). This information is provided from a few sources... * The 'cached-descriptors' file in Tor's data directory. -* Archived descriptors provided by CollecTor - (https://collector.torproject.org/) +* Archived descriptors provided by `CollecTor <https://metrics.torproject.org/collector.html>`_. * Directory authorities and mirrors via their DirPort. @@ -949,7 +948,7 @@ class RelayDescriptor(ServerDescriptor): class BridgeDescriptor(ServerDescriptor): """ Bridge descriptor (`bridge descriptor specification - <https://collector.torproject.org/formats.html#bridge-descriptors>`_) + <https://metrics.torproject.org/collector.html#bridge-descriptors>`_) :var str ed25519_certificate_hash: sha256 hash of the original identity-ed25519 :var str router_digest_sha256: sha256 digest of this document @@ -992,7 +991,7 @@ class BridgeDescriptor(ServerDescriptor): """ Checks if we've been properly scrubbed in accordance with the `bridge descriptor specification - <https://collector.torproject.org/formats.html#bridge-descriptors>`_. + <https://metrics.torproject.org/collector.html#bridge-descriptors>`_. Validation is a moving target so this may not be fully up to date. :returns: **True** if we're scrubbed, **False** otherwise

1 0

[metrics-web/master] Update @type bridge-network-status to 1.2 everywhere.
by karsten＠torproject.org 06 Jul '17

06 Jul '17

commit 4f11d77d8afd81f76a97915e660358c186f989a8 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jul 6 15:38:04 2017 +0200 Update @type bridge-network-status to 1.2 everywhere. --- website/src/main/resources/web/WEB-INF/collector.jsp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/src/main/resources/web/WEB-INF/collector.jsp b/website/src/main/resources/web/WEB-INF/collector.jsp index 5f078c6..4b37e5e 100644 --- a/website/src/main/resources/web/WEB-INF/collector.jsp +++ b/website/src/main/resources/web/WEB-INF/collector.jsp @@ -118,7 +118,7 @@ </tr> <tr> <td><a href="#type-bridge-network-status">Bridge Network Statuses</a></td> - <td><code>@type bridge-network-status 1.1</code></td> + <td><code>@type bridge-network-status 1.2</code></td> <td><a href="https://collector.torproject.org/recent/bridge-descriptors/statuses/" target="_blank" class="btn btn-primary btn-xs pull-left"> recent</a> <a href="https://collector.torproject.org/archive/bridge-descriptors/statuses/" target="_blank" class="btn btn-primary btn-xs pull-right"> archive</a></td> </tr> @@ -450,7 +450,7 @@ descriptor including all signatures.</li> </ol> <h3 id="type-bridge-network-status" class="hover">Bridge Network Statuses -<code>@type bridge-network-status 1.1</code> +<code>@type bridge-network-status 1.2</code> <a href="https://collector.torproject.org/recent/bridge-descriptors/statuses/" target="_blank" class="btn btn-primary btn-xs"> recent</a> <a href="https://collector.torproject.org/archive/bridge-descriptors/" target="_blank" class="btn btn-primary btn-xs"> archive</a> <a href="#type-bridge-network-status" class="anchor">#</a>

1 0

[tor/master] Merge remote-tracking branch 'public/bug19648'
by nickm＠torproject.org 06 Jul '17

06 Jul '17

commit 86eb63deb45f2d21375bee74ef4758c24974a97e Merge: 720552b 994595a Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jul 6 09:32:22 2017 -0400 Merge remote-tracking branch 'public/bug19648' changes/bug19648 | 5 +++++ src/or/routerlist.c | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-)

1 0

[tor/master] Don't say "downloading 1 descriptor, 4 at a time"
by nickm＠torproject.org 06 Jul '17

06 Jul '17

commit 994595ae5d0bc5e72bdeb9a2e7889b9f52b8b3c8 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue May 23 09:47:11 2017 -0400 Don't say "downloading 1 descriptor, 4 at a time" Fixes bug 19648, bugfix on 0.1.1.8-alpha. --- changes/bug19648 | 5 +++++ src/or/routerlist.c | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/changes/bug19648 b/changes/bug19648 new file mode 100644 index 0000000..e8c2a6a --- /dev/null +++ b/changes/bug19648 @@ -0,0 +1,5 @@ + o Minor bugfixes (logging): + - When logging the number of descriptors we intend to download per + directory request, do not log a number higher than then the + number of descriptors we're fetching in total. Fixes bug 19648; + bugfix on 0.1.1.8-alpha. diff --git a/src/or/routerlist.c b/src/or/routerlist.c index a790265..b7ed87a 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -5054,8 +5054,9 @@ launch_descriptor_downloads(int purpose, if (n_per_request > max_dl_per_req) n_per_request = max_dl_per_req; - if (n_per_request < MIN_DL_PER_REQUEST) - n_per_request = MIN_DL_PER_REQUEST; + if (n_per_request < MIN_DL_PER_REQUEST) { + n_per_request = MIN(MIN_DL_PER_REQUEST, n_downloadable); + } if (n_downloadable > n_per_request) req_plural = rtr_plural = "s";

1 0

[metrics-web/master] Describe new keys in @type torperf 1.1.
by karsten＠torproject.org 06 Jul '17

06 Jul '17

commit acf61979af922d6948a10e286e1cff5ed1c9579b Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jul 6 15:28:46 2017 +0200 Describe new keys in @type torperf 1.1. Implements #22263. --- .../src/main/resources/web/WEB-INF/collector.jsp | 39 +++++++++++----------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/website/src/main/resources/web/WEB-INF/collector.jsp b/website/src/main/resources/web/WEB-INF/collector.jsp index a6085f4..9edad80 100644 --- a/website/src/main/resources/web/WEB-INF/collector.jsp +++ b/website/src/main/resources/web/WEB-INF/collector.jsp @@ -160,11 +160,11 @@ <a href="https://collector.torproject.org/archive/exit-lists/" target="_blank" class="btn btn-primary btn-xs pull-right"> archive</a></td> </tr> <tr class="tableHeadline"> - <td colspan="3"><a href="#torperf">Torperf's Performance Data</a></td> + <td colspan="3"><a href="#torperf">Torperf's and OnionPerf's Performance Data</a></td> </tr> <tr> <td><a href="#type-torperf">Torperf Measurement Results</a></td> - <td><code>@type torperf 1.0</code></td> + <td><code>@type torperf 1.1</code></td> <td><a href="https://collector.torproject.org/recent/torperf/" target="_blank" class="btn btn-primary btn-xs pull-left"> recent</a> <a href="https://collector.torproject.org/archive/torperf/" target="_blank" class="btn btn-primary btn-xs pull-right"> archive</a></td> </tr> @@ -679,38 +679,38 @@ ExitAddress 91.102.152.227 2010-12-28 10:35:30 </pre> -<h2 id="torperf" class="hover">Torperf's Performance Data +<h2 id="torperf" class="hover">Torperf's and OnionPerf's Performance Data <a href="#torperf" class="anchor">#</a> </h2> -The performance measurement service Torperf publishes performance data +The performance measurement services Torperf and OnionPerf publish performance data from making simple HTTP requests over the Tor network. -Torperf uses a trivial SOCKS client to download files of various sizes +Torperf/OnionPerf use a SOCKS client to download files of various sizes over the Tor network and notes how long substeps take. -<h3 id="type-torperf" class="hover">Torperf Measurement Results -<code>@type torperf 1.0</code> +<h3 id="type-torperf" class="hover">Torperf and OnionPerf Measurement Results +<code>@type torperf 1.1</code> <a href="https://collector.torproject.org/recent/torperf/" target="_blank" class="btn btn-primary btn-xs"> recent</a> <a href="https://collector.torproject.org/archive/torperf/" target="_blank" class="btn btn-primary btn-xs"> archive</a> <a href="#type-torperf" class="anchor">#</a> </h3> -A Torperf results file contains a single line per Torperf run with +A Torperf or OnionPerf results file contains a single line per Torperf/OnionPerf run with <code>key=value</code> pairs. -Such a result line is sufficient to learn about 1) the Tor and Torperf +Such a result line is sufficient to learn about 1) the Tor and Torperf/OnionPerf configuration, 2) measurement results, and 3) additional information that might help explain the results. -Known keys are explained below. +Known keys in <code>@type torperf 1.0</code> are explained below. <ul> <li>Configuration <ul> <li><code>SOURCE:</code> Configured name of the data source; required.</li> <li><code>FILESIZE:</code> Configured file size in bytes; required.</li> -<li>Other meta data describing the Tor or Torperf configuration, e.g., +<li>Other meta data describing the Tor or Torperf/OnionPerf configuration, e.g., GUARD for a custom guard choice; optional.</li> </ul></li> <li>Measurement results @@ -759,14 +759,15 @@ measurement; optional.</li> </ul></li> </ul> - -Recently published Torperf measurement result files accumulate -all new Torperf measurements of a given day, which means that -they may change throughout the day. -This is different from some of the other recently published files provided here -which do not change once they are -written. - +OnionPerf adds a few more keys in <code>@type torperf 1.1</code>: +<ul> +<li><code>ENDPOINTLOCAL:</code> Hostname, IP address, and port that the TGen client used to connect to the local tor SOCKS port, formatted as <code>hostname:ip:port</code>, which may be <code>"NULL:0.0.0.0:0"</code> if TGen was not able to find this information; optional.</li> +<li><code>ENDPOINTPROXY:</code> Hostname, IP address, and port that the TGen client used to connect to the SOCKS proxy server that tor runs, formatted as <code>hostname:ip:port</code>, which may be <code>"NULL:0.0.0.0:0"</code> if TGen was not able to find this information; optional.</li> +<li><code>ENDPOINTREMOTE:</code> Hostname, IP address, and port that the TGen client used to connect to the remote server, formatted as <code>hostname:ip:port</code>, which may be <code>"NULL:0.0.0.0:0"</code> if TGen was not able to find this information; optional.</li> +<li><code>HOSTNAMELOCAL:</code> Client machine hostname, which may be <code>"(NULL)"</code> if the TGen client was not able to find this information; optional.</li> +<li><code>HOSTNAMEREMOTE:</code> Server machine hostname, which may be <code>"(NULL)"</code> if the TGen server was not able to find this information; optional.</li> +<li><code>SOURCEADDRESS:</code> Public IP address of the OnionPerf host obtained by connecting to well-known servers and finding the IP address in the result, which may be <code>"unknown"</code> if OnionPerf was not able to find this information; optional.</li> +</ul> </div>

1 0

[metrics-web/master] Fix two typographical errors in the documentation
by karsten＠torproject.org 06 Jul '17

06 Jul '17

commit fa69933511b1adff0d9753b53595dfe148b4475c Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jul 6 15:21:41 2017 +0200 Fix two typographical errors in the documentation Closes #22169. Adapted from onionoo.git commit 1e506dd. --- website/src/main/resources/web/WEB-INF/onionoo.jsp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/src/main/resources/web/WEB-INF/onionoo.jsp b/website/src/main/resources/web/WEB-INF/onionoo.jsp index b349b7a..3fa4c5e 100644 --- a/website/src/main/resources/web/WEB-INF/onionoo.jsp +++ b/website/src/main/resources/web/WEB-INF/onionoo.jsp @@ -320,7 +320,7 @@ Parameter values are case-insensitive. Return only running (parameter value true) or only non-running relays and/or bridges -(paramter value +(parameter value false). Parameter values are case-insensitive. @@ -921,7 +921,7 @@ This information may be helpful to decide whether a relay that is not running anymore has reached its accounting limit and has not dropped out of the network for another, unknown reason. Omitted if either the relay is not hibernating, or if no information is -available about the hiberation status of the relay. +available about the hibernation status of the relay. </li>

1 0

[metrics-web/master] Document new `fingerprint` line in bridge-network-status documents.
by karsten＠torproject.org 06 Jul '17

06 Jul '17

commit f170d03a7c10fbfbefa3ae0a64d27773121329a3 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Thu Jul 6 15:31:21 2017 +0200 Document new `fingerprint` line in bridge-network-status documents. * FIXES part of #22207: https://bugs.torproject.org/22207 Adapted from isislovecruft/collector commit 71bc054. --- website/src/main/resources/web/WEB-INF/collector.jsp | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/website/src/main/resources/web/WEB-INF/collector.jsp b/website/src/main/resources/web/WEB-INF/collector.jsp index 9edad80..5f078c6 100644 --- a/website/src/main/resources/web/WEB-INF/collector.jsp +++ b/website/src/main/resources/web/WEB-INF/collector.jsp @@ -458,7 +458,8 @@ descriptor including all signatures.</li> Sanitized bridge network statuses are similar to version 2 relay network -statuses, but with only a <code>published</code> line in the header and +statuses, but with only a <code>published</code> line and a +<code>fingerprint</code> line in the header, and without any lines in the footer. The format has changed over time to accomodate changes to the sanitizing process, with earlier versions being: @@ -468,6 +469,9 @@ process, with earlier versions being: <li><code>@type bridge-network-status 1.0</code> was the first version.</li> <li><code>@type bridge-network-status 1.1</code> introduced sanitized TCP ports.</li> +<li><code>@type bridge-network-status 1.2</code> introduced the +<code>fingerprint</code> line, containing the fingerprint of the bridge +authority which produced the document, to the header.</li> </ul> <h3 id="type-bridge-server-descriptor" class="hover">Bridge Server descriptors

1 0

[tor/master] Merge remote-tracking branch 'chelseakomlo/coding-standards-22762'
by nickm＠torproject.org 06 Jul '17

06 Jul '17

commit 720552b04c86e1b823d12bfde71ba052f4246e1c Merge: 139799c 24d29c7 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jul 6 09:30:40 2017 -0400 Merge remote-tracking branch 'chelseakomlo/coding-standards-22762' doc/HACKING/CodingStandards.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)

1 0

[tor/master] make check is required for all code submissions
by nickm＠torproject.org 06 Jul '17

06 Jul '17

commit 24d29c7ae0c523da7d59495438bbea10114da5e1 Author: Chelsea H. Komlo <chelsea.komlo(a)gmail.com> Date: Wed Jun 28 20:36:59 2017 -0400 make check is required for all code submissions integration tests should be run for feature changes/major patches --- doc/HACKING/CodingStandards.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/doc/HACKING/CodingStandards.md b/doc/HACKING/CodingStandards.md index c7787a7..da8ce62 100644 --- a/doc/HACKING/CodingStandards.md +++ b/doc/HACKING/CodingStandards.md @@ -6,7 +6,7 @@ tl;dr: - Run configure with `--enable-fatal-warnings` - Document your functions - Write unit tests - - Run `make test-full` to test against all unit and integration tests. + - Run `make check` before submitting a patch - Run `make distcheck` to ensure the distribution works - Add a file in `changes` for your branch. @@ -32,6 +32,11 @@ Did you remember... - To base your code on the appropriate branch? - To include a file in the `changes` directory as appropriate? +If you are submitting a major patch or new feature, or want to in the future... + + - Set up Chutney and Stem, see HACKING/WritingTests.md + - Run `make test-full` to test against all unit and integration tests. + How we use Git branches =======================

1 0