June 2019 - tor-commits - lists.torproject.org

[stem/master] Fix test_close_stream assertion
by atagar＠torproject.org 02 Jun '19

02 Jun '19

commit e5aba03229f163ba73e88a20f0da277fad1c368a Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jun 2 10:53:53 2019 -0700 Fix test_close_stream assertion When running our online target we might have multiple streams in process. Fixing our assertion so we check if the stream we expect is present rather than the only one. https://trac.torproject.org/projects/tor/ticket/30696 ====================================================================== FAIL: test_close_stream ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/atagar/Desktop/stem/test/require.py", line 57, in wrapped return func(self, *args, **kwargs) File "/home/atagar/Desktop/stem/test/require.py", line 57, in wrapped return func(self, *args, **kwargs) File "/home/atagar/Desktop/stem/test/integ/control/controller.py", line 1072, in test_close_stream self.assertEqual([built_stream.id], [stream.id for stream in controller.get_streams()]) AssertionError: Lists differ: ['48'] != ['48', '58', '473'] Second list contains 2 additional elements. First extra element 1: '58' - ['48'] + ['48', '58', '473'] --- test/integ/control/controller.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/test/integ/control/controller.py b/test/integ/control/controller.py index fd4c81dc..ef90de80 100644 --- a/test/integ/control/controller.py +++ b/test/integ/control/controller.py @@ -1069,15 +1069,15 @@ class TestController(unittest.TestCase): # Make sure we have the stream for which we asked, otherwise # the next assertion would be a false positive. - self.assertEqual([built_stream.id], [stream.id for stream in controller.get_streams()]) + self.assertTrue(built_stream.id in [stream.id for stream in controller.get_streams()]) # Try to close our stream... controller.close_stream(built_stream.id) - # ...which means there are zero streams. + # ... after which the stream should no longer be present. - self.assertEqual([], controller.get_streams()) + self.assertFalse(built_stream.id in [stream.id for stream in controller.get_streams()]) # unknown stream

1 0

[tor/master] fix some simple typos
by arma＠torproject.org 01 Jun '19

01 Jun '19

commit 180048e013c06ee67c053186aab46ff94cea0489 Author: Roger Dingledine <arma(a)torproject.org> Date: Sat Jun 1 15:37:51 2019 -0400 fix some simple typos --- ChangeLog | 6 +++--- ReleaseNotes | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/ChangeLog b/ChangeLog index cda4c11e0..86f39bfd7 100644 --- a/ChangeLog +++ b/ChangeLog @@ -641,7 +641,7 @@ Changes in version 0.4.0.3-alpha - 2019-03-22 Changes in version 0.3.5.8 - 2019-02-21 - Tor 0.3.5.8 backports serveral fixes from later releases, including fixes + Tor 0.3.5.8 backports several fixes from later releases, including fixes for an annoying SOCKS-parsing bug that affected users in earlier 0.3.5.x releases. @@ -660,7 +660,7 @@ Changes in version 0.3.5.8 - 2019-02-21 o Major bugfixes (networking, backport from 0.4.0.2-alpha): - Gracefully handle empty username/password fields in SOCKS5 - username/password auth messsage and allow SOCKS5 handshake to + username/password auth message and allow SOCKS5 handshake to continue. Previously, we had rejected these handshakes, breaking certain applications. Fixes bug 29175; bugfix on 0.3.5.1-alpha. @@ -867,7 +867,7 @@ Changes in version 0.4.0.2-alpha - 2019-02-21 - Check that bugfix versions in changes files look like Tor versions from the versions spec. Warn when bugfixes claim to be on a future release. Closes ticket 27761. - - Provide a git pre-commit hook that disallows commiting if we have + - Provide a git pre-commit hook that disallows committing if we have any failures in our code and changelog formatting checks. It is now available in scripts/maint/pre-commit.git-hook. Implements feature 28976. diff --git a/ReleaseNotes b/ReleaseNotes index badc5e6d0..788804236 100644 --- a/ReleaseNotes +++ b/ReleaseNotes @@ -71,7 +71,7 @@ Changes in version 0.4.0.5 - 2019-05-02 o Major bugfixes (networking): - Gracefully handle empty username/password fields in SOCKS5 - username/password auth messsage and allow SOCKS5 handshake to + username/password auth message and allow SOCKS5 handshake to continue. Previously, we had rejected these handshakes, breaking certain applications. Fixes bug 29175; bugfix on 0.3.5.1-alpha. @@ -140,7 +140,7 @@ Changes in version 0.4.0.5 - 2019-05-02 - Check that bugfix versions in changes files look like Tor versions from the versions spec. Warn when bugfixes claim to be on a future release. Closes ticket 27761. - - Provide a git pre-commit hook that disallows commiting if we have + - Provide a git pre-commit hook that disallows committing if we have any failures in our code and changelog formatting checks. It is now available in scripts/maint/pre-commit.git-hook. Implements feature 28976. @@ -602,7 +602,7 @@ Changes in version 0.4.0.5 - 2019-05-02 Changes in version 0.3.5.8 - 2019-02-21 - Tor 0.3.5.8 backports serveral fixes from later releases, including fixes + Tor 0.3.5.8 backports several fixes from later releases, including fixes for an annoying SOCKS-parsing bug that affected users in earlier 0.3.5.x releases.

1 0

[stem/master] Add stem.descriptor.get_bandwidth_file()
by atagar＠torproject.org 01 Jun '19

01 Jun '19

commit a4ecadb65c467fe277f205b47a2e3c93897d9891 Author: Damian Johnson <atagar(a)torproject.org> Date: Thu May 30 11:00:31 2019 -0700 Add stem.descriptor.get_bandwidth_file() Tor now provides bandwidth files over its DirPort. Adding our corresponding function to download them... https://trac.torproject.org/projects/tor/ticket/26902 From what I can tell these statistics are available in practice for our next hour's consensus, but not the present one... atagar@morrigan:~$ curl -s 128.31.0.34:9131/tor/status-vote/next/bandwidth | wc -l 8987 atagar@morrigan:~$ curl -s 128.31.0.34:9131/tor/status-vote/current/bandwidth | wc -l 0 --- docs/_static/example/bandwidth_stats.py | 11 +++++ docs/change_log.rst | 1 + docs/tutorials/double_double_toil_and_trouble.rst | 4 ++ docs/tutorials/examples/bandwidth_stats.rst | 48 ++++++++++++++++++++ stem/descriptor/remote.py | 53 +++++++++++++++++++---- 5 files changed, 109 insertions(+), 8 deletions(-) diff --git a/docs/_static/example/bandwidth_stats.py b/docs/_static/example/bandwidth_stats.py new file mode 100644 index 00000000..e7e11ceb --- /dev/null +++ b/docs/_static/example/bandwidth_stats.py @@ -0,0 +1,11 @@ +import stem.descriptor.remote + +bandwidth_file = stem.descriptor.remote.get_bandwidth_file().run()[0] + +for fingerprint, measurement in bandwidth_file.measurements.items(): + print('Relay %s' % fingerprint) + + for attr, value in measurement.items(): + print(' %s = %s' % (attr, value)) + + print('') diff --git a/docs/change_log.rst b/docs/change_log.rst index fc09eabc..fec98f5b 100644 --- a/docs/change_log.rst +++ b/docs/change_log.rst @@ -60,6 +60,7 @@ The following are only available within Stem's `git repository * Ed25519 validity checks are now done though the cryptography module rather than PyNaCl (:trac:`22022`) * Download compressed descriptors by default (:trac:`29186`) * Added :func:`stem.descriptor.remote.get_microdescriptors` + * Added :func:`stem.descriptor.remote.get_bandwidth_file` (:trac:`26902`) * Added :class:`~stem.descriptor.networkstatus.DetachedSignature` parsing (:trac:`28495`) * Added :func:`~stem.descriptor.__init__.Descriptor.from_str` method (:trac:`28450`) * Added :func:`~stem.descriptor.__init__.Descriptor.type_annotation` method (:trac:`28397`) diff --git a/docs/tutorials/double_double_toil_and_trouble.rst b/docs/tutorials/double_double_toil_and_trouble.rst index 6e5df2cf..b631d479 100644 --- a/docs/tutorials/double_double_toil_and_trouble.rst +++ b/docs/tutorials/double_double_toil_and_trouble.rst @@ -134,6 +134,10 @@ Descriptors Example for writing a Tor consensus to disk, and reading it back. +* `Bandwidth Heuristics <examples/bandwidth_stats.html>`_ + + Download bandwidth authority heuristics about relay capacity. + * `Checking Digests <examples/check_digests.html>`_ Looking for additional integrity that your descriptor is properly signed? diff --git a/docs/tutorials/examples/bandwidth_stats.rst b/docs/tutorials/examples/bandwidth_stats.rst new file mode 100644 index 00000000..3cfddfea --- /dev/null +++ b/docs/tutorials/examples/bandwidth_stats.rst @@ -0,0 +1,48 @@ +Bandwidth Heuristics +==================== + +.. image:: /_static/buttons/back.png + :target: ../double_double_toil_and_trouble.html + +To select the relays it will use Tor consults several factors. Exit policies, +flags, as well as bandwidth heuristics so our circuits are zippy without +overtaxing individual relays. + +These statistics are collected by a special subset of our directory authorites +called **bandwidth authorities**. See our `bandwidth file specification +<https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt>`_ for +details. Statistics are publicly available and generated each hour... + +.. literalinclude:: /_static/example/bandwidth_stats.py + :language: python + +:: + + % python bandwidth_stats.py + + Relay 6AD3EA55B87C80971F353EBA710F6550202A9355 + scanner = /scanner.5/scan-data/bws-59.4:60.1-done-2019-05-29-05:44:10 + measured_at = 1559123050 + pid_delta = -0.360692869958 + updated_at = 1559123050 + pid_error_sum = -0.178566523071 + nick = OrphanOrOften + node_id = $6AD3EA55B87C80971F353EBA710F6550202A9355 + pid_bw = 538334 + bw = 538 + pid_error = -0.178566523071 + circ_fail = 0.0 + + Relay 11B6727E38D249C83E20EEB0647BAD4FACECBEB6 + scanner = /scanner.8/scan-data/bws-92.4:93.1-done-2019-05-23-16:06:26 + measured_at = 1558641986 + pid_delta = 0.0352270644197 + updated_at = 1558641986 + pid_error_sum = -0.822158700788 + nick = snap269 + node_id = $11B6727E38D249C83E20EEB0647BAD4FACECBEB6 + pid_bw = 21124 + bw = 21 + pid_error = -0.822158700788 + circ_fail = 0.0 + diff --git a/stem/descriptor/remote.py b/stem/descriptor/remote.py index c663276b..ab2fc091 100644 --- a/stem/descriptor/remote.py +++ b/stem/descriptor/remote.py @@ -61,6 +61,8 @@ content. For example... |- get_extrainfo_descriptors - provides present extrainfo descriptors |- get_microdescriptors - provides present microdescriptors with the given digests |- get_consensus - provides the present consensus or router status entries + |- get_bandwidth_file - provies bandwidth heuristics used to make the next consensus + |- get_detached_signatures - authority signatures used to make the next consensus |- get_key_certificates - provides present authority key certificates +- query - request an arbitrary descriptor resource @@ -224,6 +226,18 @@ def get_consensus(authority_v3ident = None, microdescriptor = False, **query_arg return get_instance().get_consensus(authority_v3ident, microdescriptor, **query_args) +def get_bandwidth_file(**query_args): + """ + Shorthand for + :func:`~stem.descriptor.remote.DescriptorDownloader.get_bandwidth_file` + on our singleton instance. + + .. versionadded:: 1.8.0 + """ + + return get_instance().get_bandwidth_file(**query_args) + + def get_detached_signatures(**query_args): """ Shorthand for @@ -291,6 +305,7 @@ class Query(object): /tor/micro/d/<hash1>-<hash2> microdescriptors with the given hashes /tor/status-vote/current/consensus present consensus /tor/status-vote/current/consensus-microdesc present microdescriptor consensus + /tor/status-vote/next/bandwidth bandwidth authority heuristics for the next consenus /tor/status-vote/next/consensus-signatures detached signature, used for making the next consenus /tor/keys/all key certificates for the authorities /tor/keys/fp/<v3ident1>+<v3ident2> key certificates for specific authorities @@ -849,6 +864,22 @@ class DescriptorDownloader(object): return self.query(resource, **query_args) + def get_bandwidth_file(self, **query_args): + """ + Provides the bandwidth authority heuristics used to make the next + consensus. + + .. versionadded:: 1.8.0 + + :param query_args: additional arguments for the + :class:`~stem.descriptor.remote.Query` constructor + + :returns: :class:`~stem.descriptor.remote.Query` for the bandwidth + authority heuristics + """ + + return self.query('/tor/status-vote/next/bandwidth', **query_args) + def get_detached_signatures(self, **query_args): """ Provides the detached signatures that will be used to make the next @@ -1070,16 +1101,22 @@ def _guess_descriptor_type(resource): return 'extra-info 1.0' elif resource.startswith('/tor/micro/'): return 'microdescriptor 1.0' - elif resource.startswith('/tor/status-vote/next/consensus-signatures'): - return '%s 1.0' % DETACHED_SIGNATURE_TYPE - elif resource.startswith('/tor/status-vote/current/consensus-microdesc'): - return 'network-status-microdesc-consensus-3 1.0' - elif resource.startswith('/tor/status-vote/'): - return 'network-status-consensus-3 1.0' elif resource.startswith('/tor/keys/'): return 'dir-key-certificate-3 1.0' - else: - raise ValueError("Unable to determine the descriptor type for '%s'" % resource) + elif resource.startswith('/tor/status-vote/'): + # The following resource urls can be for the present consensus + # (/tor/status-vote/current/*) or the next (/tor/status-vote/next/*). + + if resource.endswith('/consensus'): + return 'network-status-consensus-3 1.0' + elif resource.endswith('/consensus-microdesc'): + return 'network-status-microdesc-consensus-3 1.0' + elif resource.endswith('/consensus-signatures'): + return '%s 1.0' % DETACHED_SIGNATURE_TYPE + elif resource.endswith('/bandwidth'): + return 'bandwidth-file 1.0' + + raise ValueError("Unable to determine the descriptor type for '%s'" % resource) def get_authorities():

1 0