August 2019 - tor-commits - lists.torproject.org

[translation/tbmanual-contentspot] Update translations for tbmanual-contentspot
by translation＠torproject.org 18 Aug '19

18 Aug '19

commit e1bf628b2d3f4eb609152bab6411f1f14b66a0e8 Author: Translation commit bot <translation(a)torproject.org> Date: Sun Aug 18 03:29:40 2019 +0000 Update translations for tbmanual-contentspot --- contents+fa.po | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/contents+fa.po b/contents+fa.po index 2c9f8ec0d..3c2209def 100644 --- a/contents+fa.po +++ b/contents+fa.po @@ -607,6 +607,9 @@ msgid "" "Then, select 'Provide a bridge I know' and enter each bridge address on a " "separate line." msgstr "" +"در پنجره تنظیمات شبکه تور، گزینه 'تور در کشور من سانسور شده است' را انتخاب " +"کنید. سپس گزینه 'من از قبل، یک آدرس پل دارم' را انتخاب کرده و آدرس هر پل را " +"در خطی جدا وارد کنید." #: https//tb-manual.torproject.org/bridges/ #: (content/bridges/contents+en.lrtopic.body) @@ -623,6 +626,8 @@ msgid "" "Click “OK” to save your settings. Using bridges may slow down the connection" " compared to using ordinary Tor relays." msgstr "" +"برای ذخیره تنظیمات، بر روی دکمه \"قبول\" کلیک کنید. استفاده از پل ها در " +"مقایسه با ایستگاه های عادی تور، ممکن است موجب کندشدن ارتباط اینترنتی شود." #: https//tb-manual.torproject.org/bridges/ #: (content/bridges/contents+en.lrtopic.body) @@ -630,6 +635,9 @@ msgid "" "If the connection fails, the bridges you received may be down. Please use " "one of the above methods to obtain more bridge addresses, and try again." msgstr "" +"در صورتی که ارتباط اینترنتی برقرار نشد، ممکن است ایستگاه پلی که در حال " +"استفاده از آن هستید، غیرفعال شده باشد. در این حالت، برای دریافت آدرس های پل " +"دیگر، می توان از یکی از روش هایی که در بالا توضیح داده شد، استفاده کنید." #: https//tb-manual.torproject.org/transports/ #: (content/transports/contents+en.lrtopic.title) @@ -642,6 +650,8 @@ msgid "" "Pluggable transports are tools that Tor can use to disguise the traffic it " "sends out." msgstr "" +"Pluggable Transport ها ابزاری هستند که تور به وسیله آن ها می تواند ترافیک " +"خروجی از دستگاه شما را مبهم و غیرقابل شناسایی کند." #: https//tb-manual.torproject.org/transports/ #: (content/transports/contents+en.lrtopic.body) @@ -651,6 +661,10 @@ msgid "" "Provider or other authority is actively blocking connections to the Tor " "network." msgstr "" +"Pluggable Transport ها ابزاری هستند که تور به وسیله آن ها می تواند ترافیک " +"خروجی از دستگاه شما را مبهم و غیرقابل شناسایی کند. این کار برای مواقعی که " +"اتصال شما به شبکه تور توسط شرکتی که از آن اینترنت خریداری کرده اید - ISP - " +"یا سایر مراجع دولتی، مسدود شده باشد." #: https//tb-manual.torproject.org/transports/ #: (content/transports/contents+en.lrtopic.body) @@ -663,6 +677,8 @@ msgid "" "Currently there are five pluggable transports available, but more are being " "developed." msgstr "" +"در حال حاضر، پنج نوع Pluggable Transport در دسترس است اما موارد بیشتری نیز " +"در دست توسعه می باشد." #: https//tb-manual.torproject.org/transports/ #: (content/transports/contents+en.lrtopic.body)

1 0

[stem/master] Travis: Add 0.4.0-nightly and 0.4.1-nightly jobs
by atagar＠torproject.org 18 Aug '19

18 Aug '19

commit 43f67be8b8acea21918bbb225c6fc90913f63455 Author: teor <teor(a)torproject.org> Date: Fri Aug 2 09:31:38 2019 +1000 Travis: Add 0.4.0-nightly and 0.4.1-nightly jobs Closes 30827. --- .travis.yml | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index e3b7bdfb..b3b0d9ec 100644 --- a/.travis.yml +++ b/.travis.yml @@ -58,8 +58,23 @@ matrix: packages: - tor env: TOR="stable-release" TARGET="" - ## TODO: add 0.4.0-nightly and experimental-0.4.1 when available - ## See #30827. + ## We also have 0.4.0 and 0.4.1 nightlies + - addons: + apt: + sources: + - sourceline: 'deb https://deb.torproject.org/torproject.org tor-nightly-0.4.0.x-xenial main' + key_url: 'https://deb.torproject.org/torproject.org/A3C4F0F979CAA22CDBA8F512EE8CBC9E8…' + packages: + - tor + env: TOR="0.4.0-nightly" TARGET="" + - addons: + apt: + sources: + - sourceline: 'deb https://deb.torproject.org/torproject.org tor-nightly-0.4.1.x-xenial main' + key_url: 'https://deb.torproject.org/torproject.org/A3C4F0F979CAA22CDBA8F512EE8CBC9E8…' + packages: + - tor + env: TOR="0.4.1-nightly" TARGET="" ## Test all supported python releases ## Pre-installed in Travis xenial:

1 0

[stem/master] Travis: fix a comment typo
by atagar＠torproject.org 18 Aug '19

18 Aug '19

commit 8ca077aed21ca4b72f39ea44a0135899805ec7c2 Author: teor <teor(a)torproject.org> Date: Fri Aug 2 09:31:49 2019 +1000 Travis: fix a comment typo --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index b3b0d9ec..fe96ccf6 100644 --- a/.travis.yml +++ b/.travis.yml @@ -93,7 +93,7 @@ matrix: env: TOR="master-nightly" TARGET="" ## Stable: 20 October 2019 ## (Switch from 3.8-dev to 3.8, and check for 3.9-dev) - ## End of Life: October2024 + ## End of Life: October 2024 ## https://www.python.org/dev/peps/pep-0569/#lifespan - python: "3.8-dev" env: TOR="master-nightly" TARGET=""

1 0

[stem/master] Exemplify collector functions rather than methods
by atagar＠torproject.org 17 Aug '19

17 Aug '19

commit 7033ab1327f5a12c75558b531222d0048b410c71 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jun 23 13:31:43 2019 -0700 Exemplify collector functions rather than methods Like stem.descriptor.remote I plan to add function counterparts of our CollecTor class methods. If users don't need to customize their instance (which will be the case for almost all users) the functions will be more convenient than a class. --- stem/descriptor/collector.py | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/stem/descriptor/collector.py b/stem/descriptor/collector.py index 4ccf9395..7a3d04ff 100644 --- a/stem/descriptor/collector.py +++ b/stem/descriptor/collector.py @@ -16,12 +16,11 @@ With this you can either download and read directly from CollecTor... import datetime import stem.descriptor.collector - collector = stem.descriptor.collector.CollecTor() yesterday = datetime.date.today() - datetime.timedelta(1) # provide yesterday's exits - for desc in collector.get_server_descriptors(start = yesterday): + for desc in stem.descriptor.collector.get_server_descriptors(start = yesterday): if desc.exit_policy.is_exiting_allowed(): print(' %s (%s)' % (desc.nickname, desc.fingerprint)) @@ -33,10 +32,9 @@ With this you can either download and read directly from CollecTor... import stem.descriptor import stem.descriptor.collector - collector = stem.descriptor.collector.CollecTor() yesterday = datetime.date.today() - datetime.timedelta(1) - collector.download_server_descriptors( + stem.descriptor.collector.download_server_descriptors( destination = '~/descriptor_cache', start = yesterday, ).join()

1 0

[stem/master] Remove DescriptorReader from collector example
by atagar＠torproject.org 17 Aug '19

17 Aug '19

commit 3996b3260aa702b9303bdba7c813fb30b5f932c5 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jun 23 13:28:37 2019 -0700 Remove DescriptorReader from collector example We're deprecating DescriptorReader, so best not to use it to exemplify usage of this new module. --- stem/descriptor/collector.py | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/stem/descriptor/collector.py b/stem/descriptor/collector.py index bf98dc78..4ccf9395 100644 --- a/stem/descriptor/collector.py +++ b/stem/descriptor/collector.py @@ -30,8 +30,8 @@ With this you can either download and read directly from CollecTor... :: import datetime + import stem.descriptor import stem.descriptor.collector - import stem.descriptor.reader collector = stem.descriptor.collector.CollecTor() yesterday = datetime.date.today() - datetime.timedelta(1) @@ -41,9 +41,7 @@ With this you can either download and read directly from CollecTor... start = yesterday, ).join() - reader = stem.descriptor.reader.DescriptorReader('~/descriptor_cache') - - for desc in reader: + for desc in stem.descriptor.parse_file('~/descriptor_cache', descriptor_type = 'server-descriptor 1.0'): if desc.exit_policy.is_exiting_allowed(): print(' %s (%s)' % (desc.nickname, desc.fingerprint))

1 0

[stem/master] Change collector module copyright date
by atagar＠torproject.org 17 Aug '19

17 Aug '19

commit 9dcb2ca14b12cdaaaf03e8546f88fb55ea7dab79 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jun 23 12:15:29 2019 -0700 Change collector module copyright date I began this module in 2017, but it languished since then and has only been dusted off recently in earnest. Not sure if the date should be 2017 or 2019 so gonna lean toward the later (... shouldn't matter either way). --- stem/descriptor/collector.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stem/descriptor/collector.py b/stem/descriptor/collector.py index b3f99241..bf98dc78 100644 --- a/stem/descriptor/collector.py +++ b/stem/descriptor/collector.py @@ -1,4 +1,4 @@ -# Copyright 2017-2019, Damian Johnson and The Tor Project +# Copyright 2019, Damian Johnson and The Tor Project # See LICENSE for licensing information """

1 0

[stem/master] Allow plaintext enum value
by atagar＠torproject.org 17 Aug '19

17 Aug '19

commit 45b98cac45f3f80f6bb319d898ad12775b30823a Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jun 23 14:29:14 2019 -0700 Allow plaintext enum value Our CollecTor module used None to specify plaintext but we should accept the Compression.PLAINTEXT value as well. --- stem/descriptor/collector.py | 22 +++++++++++----------- test/unit/descriptor/collector.py | 5 +++-- 2 files changed, 14 insertions(+), 13 deletions(-) diff --git a/stem/descriptor/collector.py b/stem/descriptor/collector.py index 7a3d04ff..21c9d91d 100644 --- a/stem/descriptor/collector.py +++ b/stem/descriptor/collector.py @@ -65,7 +65,7 @@ COLLECTOR_URL = 'https://collector.torproject.org/' REFRESH_INDEX_RATE = 3600 # get new index if cached copy is an hour old -def url(resource, compression = None): +def url(resource, compression = Compression.PLAINTEXT): """ Provides CollecTor url for the given resource. @@ -84,8 +84,8 @@ def url(resource, compression = None): else: raise ValueError("'%s' isn't a recognized resource type" % resource) - suffix = compression.extension if compression else '' - return COLLECTOR_URL + '/'.join(path) + suffix + extension = compression.extension if compression not in (None, Compression.PLAINTEXT) else '' + return COLLECTOR_URL + '/'.join(path) + extension class CollecTor(object): @@ -102,22 +102,22 @@ class CollecTor(object): """ def __init__(self, compression = 'best', retries = 2, timeout = None): + self.compression = Compression.PLAINTEXT + self.retries = retries + self.timeout = timeout + + self._cached_index = None + self._cached_index_at = 0 + if compression == 'best': - self.compression = None for option in (Compression.LZMA, Compression.BZ2, Compression.GZIP): if option.available: self.compression = option break - else: + elif compression is not None: self.compression = compression - self.retries = retries - self.timeout = timeout - - self._cached_index = None - self._cached_index_at = 0 - def index(self): """ Provides the archives available in CollecTor. diff --git a/test/unit/descriptor/collector.py b/test/unit/descriptor/collector.py index b2b464ce..6aeda8f0 100644 --- a/test/unit/descriptor/collector.py +++ b/test/unit/descriptor/collector.py @@ -23,6 +23,7 @@ class TestCollector(unittest.TestCase): def test_url(self): self.assertEqual('https://collector.torproject.org/index/index.json', url('index')) self.assertEqual('https://collector.torproject.org/index/index.json', url('index', compression = None)) + self.assertEqual('https://collector.torproject.org/index/index.json', url('index', compression = Compression.PLAINTEXT)) self.assertEqual('https://collector.torproject.org/index/index.json.gz', url('index', compression = Compression.GZIP)) self.assertEqual('https://collector.torproject.org/index/index.json.bz2', url('index', compression = Compression.BZ2)) self.assertEqual('https://collector.torproject.org/index/index.json.xz', url('index', compression = Compression.LZMA)) @@ -35,12 +36,12 @@ class TestCollector(unittest.TestCase): 'path': 'https://collector.torproject.org' } - collector = CollecTor(compression = None) + collector = CollecTor(compression = Compression.PLAINTEXT) self.assertEqual(expected, collector.index()) @patch(URL_OPEN, Mock(return_value = io.BytesIO(b'not json'))) def test_index_malformed_json(self): - collector = CollecTor(compression = None) + collector = CollecTor(compression = Compression.PLAINTEXT) if stem.prereq.is_python_3(): self.assertRaisesRegexp(ValueError, 'Expecting value: line 1 column 1', collector.index)

1 0

[stem/master] Implement CollecTor download retries
by atagar＠torproject.org 17 Aug '19

17 Aug '19

commit d3de8b55528f4ed186fb9f99d032f421a4c1f301 Author: Damian Johnson <atagar(a)torproject.org> Date: Thu Jul 4 17:27:32 2019 -0700 Implement CollecTor download retries --- stem/descriptor/collector.py | 63 +++++++++++++++++++++++++++++++-------- test/unit/descriptor/collector.py | 10 ++++++- 2 files changed, 60 insertions(+), 13 deletions(-) diff --git a/stem/descriptor/collector.py b/stem/descriptor/collector.py index 21c9d91d..0bbf4251 100644 --- a/stem/descriptor/collector.py +++ b/stem/descriptor/collector.py @@ -47,9 +47,11 @@ With this you can either download and read directly from CollecTor... """ import json +import sys import time from stem.descriptor import Compression +from stem.util import log try: # account for urllib's change between python 2.x and 3.x @@ -88,6 +90,53 @@ def url(resource, compression = Compression.PLAINTEXT): return COLLECTOR_URL + '/'.join(path) + extension +def _download(url, compression, timeout, retries): + """ + Download from the given url. + + :param str url: url to download from + :param descriptor.Compression compression: decompression type + :param int timeout: timeout when connection becomes idle, no timeout applied + if **None** + :param int retires: maximum attempts to impose + + :returns: content of the given url + + :raises: + * **IOError** if unable to decompress + * **socket.timeout** if our request timed out + * **urllib2.URLError** for most request failures + + Note that the urllib2 module may fail with other exception types, in + which case we'll pass it along. + """ + + start_time = time.time() + + try: + response = urllib.urlopen(url, timeout = timeout).read() + except: + exc = sys.exc_info()[1] + + if timeout is not None: + timeout -= time.time() - start_time + + if retries > 0 and (timeout is None or timeout > 0): + log.debug("Failed to download from CollecTor at '%s' (%i retries remaining): %s" % (url, retries, exc)) + return _download(url, compression, timeout, retries - 1) + else: + log.debug("Failed to download from CollecTor at '%s': %s" % (url, exc)) + raise + + if compression not in (None, Compression.PLAINTEXT): + try: + response = compression.decompress(response) + except Exception as exc: + raise IOError('Unable to decompress %s response from %s: %s' % (compression, url, exc)) + + return stem.util.str_tools._to_unicode(response) + + class CollecTor(object): """ Downloader for descriptors from CollecTor. The contents of CollecTor are @@ -110,7 +159,6 @@ class CollecTor(object): self._cached_index_at = 0 if compression == 'best': - for option in (Compression.LZMA, Compression.BZ2, Compression.GZIP): if option.available: self.compression = option @@ -134,17 +182,8 @@ class CollecTor(object): """ if not self._cached_index or time.time() - self._cached_index_at >= REFRESH_INDEX_RATE: - # TODO: add retry support - - response = urllib.urlopen(url('index', self.compression), timeout = self.timeout).read() - - if self.compression: - try: - response = self.compression.decompress(response) - except Exception as exc: - raise IOError('Unable to decompress response as %s: %s' % (self.compression, exc)) - - self._cached_index = json.loads(stem.util.str_tools._to_unicode(response)) + response = _download(url('index', self.compression), self.compression, self.timeout, self.retries) + self._cached_index = json.loads(response) self._cached_index_at = time.time() return self._cached_index diff --git a/test/unit/descriptor/collector.py b/test/unit/descriptor/collector.py index 6aeda8f0..c46fb60c 100644 --- a/test/unit/descriptor/collector.py +++ b/test/unit/descriptor/collector.py @@ -28,6 +28,14 @@ class TestCollector(unittest.TestCase): self.assertEqual('https://collector.torproject.org/index/index.json.bz2', url('index', compression = Compression.BZ2)) self.assertEqual('https://collector.torproject.org/index/index.json.xz', url('index', compression = Compression.LZMA)) + @patch(URL_OPEN) + def test_retries(self, urlopen_mock): + collector = CollecTor(retries = 4) + urlopen_mock.side_effect = IOError('boom') + + self.assertRaisesRegexp(IOError, 'boom', collector.index) + self.assertEqual(5, urlopen_mock.call_count) + @patch(URL_OPEN, Mock(return_value = io.BytesIO(b'{"index_created":"2017-12-25 21:06","build_revision":"56a303e","path":"https://collector.torproject.org"}'))) def test_index(self): expected = { @@ -52,4 +60,4 @@ class TestCollector(unittest.TestCase): for compression in (Compression.GZIP, Compression.BZ2, Compression.LZMA): with patch(URL_OPEN, Mock(return_value = io.BytesIO(b'not compressed'))): collector = CollecTor(compression = compression) - self.assertRaisesRegexp(IOError, 'Unable to decompress response as %s' % compression, collector.index) + self.assertRaisesRegexp(IOError, 'Unable to decompress %s response' % compression, collector.index)

1 0

[stem/master] Drop url helper function
by atagar＠torproject.org 17 Aug '19

17 Aug '19

commit a60efda6275fbee76349896ba6f1ed63415c4b94 Author: Damian Johnson <atagar(a)torproject.org> Date: Fri Jul 5 14:12:52 2019 -0700 Drop url helper function Originally I was unsure if I'd want this. Turns out nope - it helped with url testability, but we can exercise the same with better mocks. --- stem/descriptor/collector.py | 31 +++--------- test/unit/descriptor/collector.py | 102 +++++++++++++++++++++++++++++++------- 2 files changed, 89 insertions(+), 44 deletions(-) diff --git a/stem/descriptor/collector.py b/stem/descriptor/collector.py index 746e4b1d..d723e97a 100644 --- a/stem/descriptor/collector.py +++ b/stem/descriptor/collector.py @@ -70,34 +70,11 @@ COLLECTOR_URL = 'https://collector.torproject.org/' REFRESH_INDEX_RATE = 3600 # get new index if cached copy is an hour old -def url(resource, compression = Compression.PLAINTEXT): - """ - Provides CollecTor url for the given resource. - - :param str resource: resource type of the url - :param descriptor.Compression compression: compression type to - download from - - :returns: **str** with the CollecTor url - """ - - # TODO: Unsure how to most elegantly map resources to urls. No doubt - # this'll change as we add more types. - - if resource == 'index': - path = ('index', 'index.json') - else: - raise ValueError("'%s' isn't a recognized resource type" % resource) - - extension = compression.extension if compression not in (None, Compression.PLAINTEXT) else '' - return COLLECTOR_URL + '/'.join(path) + extension - - def _download(url, compression, timeout, retries): """ Download from the given url. - :param str url: url to download from + :param str url: uncompressed url to download from :param descriptor.Compression compression: decompression type :param int timeout: timeout when connection becomes idle, no timeout applied if **None** @@ -115,6 +92,10 @@ def _download(url, compression, timeout, retries): """ start_time = time.time() + extension = compression.extension if compression not in (None, Compression.PLAINTEXT) else '' + + if not url.endswith(extension): + url += extension try: response = urllib.urlopen(url, timeout = timeout).read() @@ -185,7 +166,7 @@ class CollecTor(object): """ if not self._cached_index or time.time() - self._cached_index_at >= REFRESH_INDEX_RATE: - response = _download(url('index', self.compression), self.compression, self.timeout, self.retries) + response = _download(COLLECTOR_URL + 'index/index.json', self.compression, self.timeout, self.retries) self._cached_index = json.loads(response) self._cached_index_at = time.time() diff --git a/test/unit/descriptor/collector.py b/test/unit/descriptor/collector.py index c46fb60c..cfb57c3d 100644 --- a/test/unit/descriptor/collector.py +++ b/test/unit/descriptor/collector.py @@ -8,7 +8,7 @@ import unittest import stem.prereq from stem.descriptor import Compression -from stem.descriptor.collector import CollecTor, url +from stem.descriptor.collector import CollecTor try: # added in python 3.3 @@ -18,34 +18,95 @@ except ImportError: URL_OPEN = 'urllib.request.urlopen' if stem.prereq.is_python_3() else 'urllib2.urlopen' +MINIMAL_INDEX = { + 'index_created': '2017-12-25 21:06', + 'build_revision': '56a303e', + 'path': 'https://collector.torproject.org' +} + +MINIMAL_INDEX_JSON = b'{"index_created":"2017-12-25 21:06","build_revision":"56a303e","path":"https://collector.torproject.org"}' + class TestCollector(unittest.TestCase): - def test_url(self): - self.assertEqual('https://collector.torproject.org/index/index.json', url('index')) - self.assertEqual('https://collector.torproject.org/index/index.json', url('index', compression = None)) - self.assertEqual('https://collector.torproject.org/index/index.json', url('index', compression = Compression.PLAINTEXT)) - self.assertEqual('https://collector.torproject.org/index/index.json.gz', url('index', compression = Compression.GZIP)) - self.assertEqual('https://collector.torproject.org/index/index.json.bz2', url('index', compression = Compression.BZ2)) - self.assertEqual('https://collector.torproject.org/index/index.json.xz', url('index', compression = Compression.LZMA)) + @patch(URL_OPEN) + def test_download_plaintext(self, urlopen_mock): + urlopen_mock.return_value = io.BytesIO(MINIMAL_INDEX_JSON) + + collector = CollecTor(compression = Compression.PLAINTEXT) + self.assertEqual(MINIMAL_INDEX, collector.index()) + urlopen_mock.assert_called_with('https://collector.torproject.org/index/index.json', timeout = None) @patch(URL_OPEN) - def test_retries(self, urlopen_mock): - collector = CollecTor(retries = 4) + def test_download_gzip(self, urlopen_mock): + if not Compression.GZIP.available: + self.skipTest('(gzip compression unavailable)') + return + + import zlib + urlopen_mock.return_value = io.BytesIO(zlib.compress(MINIMAL_INDEX_JSON)) + + collector = CollecTor(compression = Compression.GZIP) + self.assertEqual(MINIMAL_INDEX, collector.index()) + urlopen_mock.assert_called_with('https://collector.torproject.org/index/index.json.gz', timeout = None) + + @patch(URL_OPEN) + def test_download_bz2(self, urlopen_mock): + if not Compression.BZ2.available: + self.skipTest('(bz2 compression unavailable)') + return + + import bz2 + urlopen_mock.return_value = io.BytesIO(bz2.compress(MINIMAL_INDEX_JSON)) + + collector = CollecTor(compression = Compression.BZ2) + self.assertEqual(MINIMAL_INDEX, collector.index()) + urlopen_mock.assert_called_with('https://collector.torproject.org/index/index.json.bz2', timeout = None) + + @patch(URL_OPEN) + def test_download_lzma(self, urlopen_mock): + if not Compression.LZMA.available: + self.skipTest('(lzma compression unavailable)') + return + + import lzma + urlopen_mock.return_value = io.BytesIO(lzma.compress(MINIMAL_INDEX_JSON)) + + collector = CollecTor(compression = Compression.LZMA) + self.assertEqual(MINIMAL_INDEX, collector.index()) + urlopen_mock.assert_called_with('https://collector.torproject.org/index/index.json.lzma', timeout = None) + + @patch(URL_OPEN) + def test_download_zstd(self, urlopen_mock): + if not Compression.ZSTD.available: + self.skipTest('(zstd compression unavailable)') + return + + import zstd + compressor = zstd.ZstdCompressor() + urlopen_mock.return_value = io.BytesIO(compressor.compress(MINIMAL_INDEX_JSON)) + + collector = CollecTor(compression = Compression.ZSTD) + self.assertEqual(MINIMAL_INDEX, collector.index()) + urlopen_mock.assert_called_with('https://collector.torproject.org/index/index.json.zst', timeout = None) + + @patch(URL_OPEN) + def test_download_retries(self, urlopen_mock): urlopen_mock.side_effect = IOError('boom') + collector = CollecTor(retries = 0) + self.assertRaisesRegexp(IOError, 'boom', collector.index) + self.assertEqual(1, urlopen_mock.call_count) + + urlopen_mock.reset_mock() + + collector = CollecTor(retries = 4) self.assertRaisesRegexp(IOError, 'boom', collector.index) self.assertEqual(5, urlopen_mock.call_count) - @patch(URL_OPEN, Mock(return_value = io.BytesIO(b'{"index_created":"2017-12-25 21:06","build_revision":"56a303e","path":"https://collector.torproject.org"}'))) + @patch(URL_OPEN, Mock(return_value = io.BytesIO(MINIMAL_INDEX_JSON))) def test_index(self): - expected = { - 'index_created': '2017-12-25 21:06', - 'build_revision': '56a303e', - 'path': 'https://collector.torproject.org' - } - collector = CollecTor(compression = Compression.PLAINTEXT) - self.assertEqual(expected, collector.index()) + self.assertEqual(MINIMAL_INDEX, collector.index()) @patch(URL_OPEN, Mock(return_value = io.BytesIO(b'not json'))) def test_index_malformed_json(self): @@ -57,7 +118,10 @@ class TestCollector(unittest.TestCase): self.assertRaisesRegexp(ValueError, 'No JSON object could be decoded', collector.index) def test_index_malformed_compression(self): - for compression in (Compression.GZIP, Compression.BZ2, Compression.LZMA): + for compression in (Compression.GZIP, Compression.BZ2, Compression.LZMA, Compression.ZSTD): + if not compression.available: + next + with patch(URL_OPEN, Mock(return_value = io.BytesIO(b'not compressed'))): collector = CollecTor(compression = compression) self.assertRaisesRegexp(IOError, 'Unable to decompress %s response' % compression, collector.index)

1 0

[stem/master] Test real CollecTor index
by atagar＠torproject.org 17 Aug '19

17 Aug '19

commit e0b4e2d8bf6c0c31e64a167c2e96e845185ddff5 Author: Damian Johnson <atagar(a)torproject.org> Date: Sat Jul 6 15:38:04 2019 -0700 Test real CollecTor index Adding a CollecTor index, truncated to three files per directory. Even with this truncation this is *way* too big for this unit test assertion, but starting with the whole shebang as I make sense of how I'm gonna work with it. --- test/unit/descriptor/collector.py | 539 +++++++++++++++++++++++++ test/unit/descriptor/data/collector_index.json | 1 + 2 files changed, 540 insertions(+) diff --git a/test/unit/descriptor/collector.py b/test/unit/descriptor/collector.py index 260e246b..e1ff4bdb 100644 --- a/test/unit/descriptor/collector.py +++ b/test/unit/descriptor/collector.py @@ -9,6 +9,7 @@ import stem.prereq from stem.descriptor import Compression from stem.descriptor.collector import CollecTor +from test.unit.descriptor import get_resource try: # added in python 3.3 @@ -26,6 +27,535 @@ MINIMAL_INDEX = { MINIMAL_INDEX_JSON = b'{"index_created":"2017-12-25 21:06","build_revision":"56a303e","path":"https://collector.torproject.org"}' +EXAMPLE_INDEX = { + 'index_created': '2019-07-06 01:54', + 'build_revision': 'df98ac79', + 'path': 'https://collector.torproject.org', + 'directories': [ + { + 'path': 'archive', + 'directories': [ + { + 'path': 'bridge-descriptors', + 'directories': [ + { + 'path': 'extra-infos', + 'files': [ + { + 'path': 'bridge-extra-infos-2008-05.tar.xz', + 'size': 377644, + 'last_modified': '2016-09-04 09:21' + }, { + 'path': 'bridge-extra-infos-2008-06.tar.xz', + 'size': 600484, + 'last_modified': '2016-09-04 09:21' + }, { + 'path': 'bridge-extra-infos-2008-07.tar.xz', + 'size': 716320, + 'last_modified': '2016-09-04 09:21' + } + ] + }, { + 'path': 'server-descriptors', + 'files': [ + { + 'path': 'bridge-server-descriptors-2008-05.tar.xz', + 'size': 205348, + 'last_modified': '2016-09-09 14:13' + }, { + 'path': 'bridge-server-descriptors-2008-06.tar.xz', + 'size': 342828, + 'last_modified': '2016-09-09 14:13' + }, { + 'path': 'bridge-server-descriptors-2008-07.tar.xz', + 'size': 374848, + 'last_modified': '2016-09-09 14:13' + } + ] + }, { + 'path': 'statuses', + 'files': [ + { + 'path': 'bridge-statuses-2008-05.tar.xz', + 'size': 74792, + 'last_modified': '2016-09-14 21:11' + }, { + 'path': 'bridge-statuses-2008-06.tar.xz', + 'size': 123488, + 'last_modified': '2016-09-14 21:11' + }, { + 'path': 'bridge-statuses-2008-07.tar.xz', + 'size': 143836, + 'last_modified': '2016-09-14 21:11' + } + ] + } + ] + }, { + 'path': 'bridge-pool-assignments', + 'files': [ + { + 'path': 'bridge-pool-assignments-2010-09.tar.xz', + 'size': 32804, + 'last_modified': '2012-05-31 10:21' + }, { + 'path': 'bridge-pool-assignments-2010-10.tar.xz', + 'size': 304684, + 'last_modified': '2012-05-31 10:21' + }, { + 'path': 'bridge-pool-assignments-2010-11.tar.xz', + 'size': 292228, + 'last_modified': '2012-05-31 10:21' + } + ] + }, { + 'path': 'exit-lists', + 'files': [ + { + 'path': 'exit-list-2010-02.tar.xz', + 'size': 272008, + 'last_modified': '2012-05-31 18:57' + }, { + 'path': 'exit-list-2010-03.tar.xz', + 'size': 1247484, + 'last_modified': '2012-05-31 18:57' + }, { + 'path': 'exit-list-2010-04.tar.xz', + 'size': 1139896, + 'last_modified': '2012-05-31 18:57' + } + ] + }, { + 'path': 'relay-descriptors', + 'files': [ + { + 'path': 'certs.tar.xz', + 'size': 144696, + 'last_modified': '2019-07-03 03:29' + } + ], + 'directories': [ + { + 'path': 'bandwidths', + 'files': [ + { + 'path': 'bandwidths-2019-05.tar.xz', + 'size': 50385816, + 'last_modified': '2019-06-07 08:05' + }, { + 'path': 'bandwidths-2019-06.tar.xz', + 'size': 105881156, + 'last_modified': '2019-07-03 07:30' + }, { + 'path': 'bandwidths-2019-07.tar.xz', + 'size': 11374436, + 'last_modified': '2019-07-03 07:12' + } + ] + }, { + 'path': 'consensuses', + 'files': [ + { + 'path': 'consensuses-2007-10.tar.xz', + 'size': 1061648, + 'last_modified': '2012-05-15 14:35' + }, { + 'path': 'consensuses-2007-11.tar.xz', + 'size': 6810308, + 'last_modified': '2012-05-15 14:35' + }, { + 'path': 'consensuses-2007-12.tar.xz', + 'size': 8106968, + 'last_modified': '2012-05-15 14:35' + } + ] + }, { + 'path': 'extra-infos', + 'files': [ + { + 'path': 'extra-infos-2007-08.tar.xz', + 'size': 3016916, + 'last_modified': '2016-06-23 09:53' + }, { + 'path': 'extra-infos-2007-09.tar.xz', + 'size': 6459884, + 'last_modified': '2016-06-23 09:54' + }, { + 'path': 'extra-infos-2007-10.tar.xz', + 'size': 7326564, + 'last_modified': '2016-06-23 09:54' + } + ] + }, { + 'path': 'microdescs', + 'files': [ + { + 'path': 'microdescs-2014-01.tar.xz', + 'size': 7515396, + 'last_modified': '2014-02-07 03:59' + }, { + 'path': 'microdescs-2014-02.tar.xz', + 'size': 21822944, + 'last_modified': '2014-03-07 04:54' + }, { + 'path': 'microdescs-2014-03.tar.xz', + 'size': 24201436, + 'last_modified': '2014-04-07 03:54' + } + ] + }, { + 'path': 'server-descriptors', + 'files': [ + { + 'path': 'server-descriptors-2005-12.tar.xz', + 'size': 1348620, + 'last_modified': '2016-06-24 08:12' + }, { + 'path': 'server-descriptors-2006-02.tar.xz', + 'size': 28625408, + 'last_modified': '2016-06-24 08:14' + }, { + 'path': 'server-descriptors-2006-03.tar.xz', + 'size': 49548736, + 'last_modified': '2016-06-24 08:17' + } + ] + }, { + 'path': 'statuses', + 'files': [ + { + 'path': 'statuses-2005-12.tar.xz', + 'size': 1468844, + 'last_modified': '2016-06-25 11:50' + }, { + 'path': 'statuses-2006-01.tar.xz', + 'size': 3344280, + 'last_modified': '2016-06-25 11:52' + }, { + 'path': 'statuses-2006-02.tar.xz', + 'size': 4006336, + 'last_modified': '2016-06-25 11:54' + } + ] + }, { + 'path': 'tor', + 'files': [ + { + 'path': 'tor-2004-05.tar.xz', + 'size': 386672, + 'last_modified': '2012-05-18 14:26' + }, { + 'path': 'tor-2004-06.tar.xz', + 'size': 1087980, + 'last_modified': '2012-05-18 14:26' + }, { + 'path': 'tor-2004-07.tar.xz', + 'size': 1366568, + 'last_modified': '2012-05-18 14:26' + } + ] + }, { + 'path': 'votes', + 'files': [ + { + 'path': 'votes-2007-10.tar.xz', + 'size': 1356504, + 'last_modified': '2012-05-15 14:51' + }, { + 'path': 'votes-2007-11.tar.xz', + 'size': 10641492, + 'last_modified': '2012-05-15 14:51' + }, { + 'path': 'votes-2007-12.tar.xz', + 'size': 14712136, + 'last_modified': '2012-05-15 14:52' + } + ] + } + ] + }, { + 'path': 'torperf', + 'files': [ + { + 'path': 'torperf-2009-07.tar.xz', + 'size': 182712, + 'last_modified': '2012-05-30 07:23' + }, { + 'path': 'torperf-2009-08.tar.xz', + 'size': 203236, + 'last_modified': '2012-05-30 07:23' + }, { + 'path': 'torperf-2009-09.tar.xz', + 'size': 193832, + 'last_modified': '2012-05-30 07:23' + } + ] + }, { + 'path': 'webstats', + 'files': [ + { + 'path': 'webstats-2015-01.tar', + 'size': 30720, + 'last_modified': '2018-03-19 16:07' + }, { + 'path': 'webstats-2015-02.tar', + 'size': 20480, + 'last_modified': '2018-03-19 16:07' + }, { + 'path': 'webstats-2015-03.tar', + 'size': 20480, + 'last_modified': '2018-03-19 16:07' + } + ] + } + ] + }, { + 'path': 'contrib' + }, { + 'path': 'recent', + 'directories': [ + { + 'path': 'bridge-descriptors', + 'directories': [ + { + 'path': 'extra-infos', + 'files': [ + { + 'path': '2019-07-03-02-09-00-extra-infos', + 'size': 507816, + 'last_modified': '2019-07-03 02:09' + }, { + 'path': '2019-07-03-03-09-00-extra-infos', + 'size': 558790, + 'last_modified': '2019-07-03 03:09' + }, { + 'path': '2019-07-03-04-09-05-extra-infos', + 'size': 592279, + 'last_modified': '2019-07-03 04:09' + } + ] + }, { + 'path': 'server-descriptors', + 'files': [ + { + 'path': '2019-07-03-02-09-00-server-descriptors', + 'size': 274355, + 'last_modified': '2019-07-03 02:09' + }, { + 'path': '2019-07-03-03-09-00-server-descriptors', + 'size': 295671, + 'last_modified': '2019-07-03 03:09' + }, { + 'path': '2019-07-03-04-09-05-server-descriptors', + 'size': 312822, + 'last_modified': '2019-07-03 04:09' + } + ] + }, { + 'path': 'statuses', + 'files': [ + { + 'path': '20190703-011231-BA44A889E64B93FAA2B114E02C2A279A8555C533', + 'size': 238340, + 'last_modified': '2019-07-03 02:09' + }, { + 'path': '20190703-014231-BA44A889E64B93FAA2B114E02C2A279A8555C533', + 'size': 238147, + 'last_modified': '2019-07-03 02:09' + }, { + 'path': '20190703-021231-BA44A889E64B93FAA2B114E02C2A279A8555C533', + 'size': 238231, + 'last_modified': '2019-07-03 03:09' + } + ] + } + ] + }, { + 'path': 'exit-lists', + 'files': [ + { + 'path': '2019-07-03-02-02-00', + 'size': 158516, + 'last_modified': '2019-07-03 02:02' + }, { + 'path': '2019-07-03-03-02-00', + 'size': 158830, + 'last_modified': '2019-07-03 03:02' + }, { + 'path': '2019-07-03-04-02-00', + 'size': 158831, + 'last_modified': '2019-07-03 04:02' + } + ] + }, { + 'path': 'relay-descriptors', + 'directories': [ + { + 'path': 'bandwidths', + 'files': [ + { + 'path': '2019-07-03-00-30-57-bandwidth-616DF2278EE2C90F475E4EA2562E2C00EB9F10E517FB901229F331AEB70B8AD7', + 'size': 2580044, + 'last_modified': '2019-07-03 02:35' + }, { + 'path': '2019-07-03-01-35-02-bandwidth-2A5B679E9AA2D5C903CD16C00E16FEFA3E21E38AB3DE338941AE4AC13FD505DE', + 'size': 4210146, + 'last_modified': '2019-07-03 02:35' + }, { + 'path': '2019-07-03-01-35-03-bandwidth-3B06DAD9EDE8E1D08CC494E856AFD9D49256ECC2D68456799DD1EC7ED3436875', + 'size': 4249699, + 'last_modified': '2019-07-03 02:35' + } + ] + }, { + 'path': 'consensuses', + 'files': [ + { + 'path': '2019-07-03-02-00-00-consensus', + 'size': 2211265, + 'last_modified': '2019-07-03 02:05' + }, { + 'path': '2019-07-03-03-00-00-consensus', + 'size': 2204478, + 'last_modified': '2019-07-03 03:05' + }, { + 'path': '2019-07-03-04-00-00-consensus', + 'size': 2202554, + 'last_modified': '2019-07-03 04:05' + } + ] + }, { + 'path': 'extra-infos', + 'files': [ + { + 'path': '2019-07-03-02-05-00-extra-infos', + 'size': 1162899, + 'last_modified': '2019-07-03 02:05' + }, { + 'path': '2019-07-03-03-05-00-extra-infos', + 'size': 1133425, + 'last_modified': '2019-07-03 03:05' + }, { + 'path': '2019-07-03-04-05-00-extra-infos', + 'size': 1153793, + 'last_modified': '2019-07-03 04:05' + } + ] + }, { + 'path': 'microdescs', + 'directories': [ + { + 'path': 'consensus-microdesc', + 'files': [ + { + 'path': '2019-07-03-02-00-00-consensus-microdesc', + 'size': 2028994, + 'last_modified': '2019-07-03 02:05' + }, { + 'path': '2019-07-03-03-00-00-consensus-microdesc', + 'size': 2022808, + 'last_modified': '2019-07-03 03:05' + }, { + 'path': '2019-07-03-04-00-00-consensus-microdesc', + 'size': 2020751, + 'last_modified': '2019-07-03 04:05' + } + ] + }, { + 'path': 'micro', + 'files': [ + { + 'path': '2019-07-02-23-05-00-micro', + 'size': 19934, + 'last_modified': '2019-07-02 23:05' + }, { + 'path': '2019-07-03-00-05-00-micro', + 'size': 12030, + 'last_modified': '2019-07-03 00:05' + }, { + 'path': '2019-07-03-01-05-00-micro', + 'size': 14850, + 'last_modified': '2019-07-03 01:05' + } + ] + } + ] + }, { + 'path': 'server-descriptors', + 'files': [ + { + 'path': '2019-07-03-02-05-00-server-descriptors', + 'size': 1374587, + 'last_modified': '2019-07-03 02:05' + }, { + 'path': '2019-07-03-03-05-00-server-descriptors', + 'size': 1353286, + 'last_modified': '2019-07-03 03:05' + }, { + 'path': '2019-07-03-04-05-00-server-descriptors', + 'size': 1336125, + 'last_modified': '2019-07-03 04:05' + } + ] + }, { + 'path': 'votes', + 'files': [ + { + 'path': '2019-07-03-02-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-D9B6923A3AF9FB5DB3CD30191BCBD29A7A936D86', + 'size': 3449707, + 'last_modified': '2019-07-03 02:05' + }, { + 'path': '2019-07-03-02-00-00-vote-14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4-39429F0789A9EC07A2A8754C4C449CCABAB787CC', + 'size': 3435482, + 'last_modified': '2019-07-03 02:05' + }, { + 'path': '2019-07-03-02-00-00-vote-23D15D965BC35114467363C165C4F724B64B4F66-4337842B697467123B5351E1E8EB64ED497C0EDA', + 'size': 3571106, + 'last_modified': '2019-07-03 02:05' + } + ] + } + ] + }, { + 'path': 'torperf', + 'files': [ + { + 'path': 'op-ab-1048576-2019-07-03.tpf', + 'size': 39152, + 'last_modified': '2019-07-04 00:01' + }, { + 'path': 'op-ab-51200-2019-07-03.tpf', + 'size': 248173, + 'last_modified': '2019-07-04 00:01' + }, { + 'path': 'op-ab-5242880-2019-07-03.tpf', + 'size': 15900, + 'last_modified': '2019-07-04 00:01' + } + ] + }, { + 'path': 'webstats', + 'files': [ + { + 'path': '2019.www.torproject.org_hetzner-hel1-03.torproject.org_access.log_20190620.xz', + 'size': 8872, + 'last_modified': '2019-07-03 04:21' + }, { + 'path': '2019.www.torproject.org_hetzner-hel1-03.torproject.org_access.log_20190621.xz', + 'size': 9312, + 'last_modified': '2019-07-04 04:21' + }, { + 'path': '2019.www.torproject.org_hetzner-hel1-03.torproject.org_access.log_20190622.xz', + 'size': 9836, + 'last_modified': '2019-07-05 04:21' + } + ] + } + ] + } + ] +} + class TestCollector(unittest.TestCase): @patch(URL_OPEN) @@ -111,3 +641,12 @@ class TestCollector(unittest.TestCase): with patch(URL_OPEN, Mock(return_value = io.BytesIO(b'not compressed'))): collector = CollecTor(compression = compression) self.assertRaisesRegexp(IOError, 'Unable to decompress %s response' % compression, collector.index) + + @patch(URL_OPEN) + def test_real_index(self, urlopen_mock): + with open(get_resource('collector_index.json'), 'rb') as index_file: + urlopen_mock.return_value = io.BytesIO(index_file.read()) + + self.maxDiff = None + collector = CollecTor(compression = Compression.PLAINTEXT) + self.assertEqual(EXAMPLE_INDEX, collector.index()) diff --git a/test/unit/descriptor/data/collector_index.json b/test/unit/descriptor/data/collector_index.json new file mode 100644 index 00000000..c6528f23 --- /dev/null +++ b/test/unit/descriptor/data/collector_index.json @@ -0,0 +1 @@ +{"index_created":"2019-07-06 01:54","build_revision":"df98ac79","path":"https://collector.torproject.org","directories":[{"path":"archive","directories":[{"path":"bridge-descriptors","directories":[{"path":"extra-infos","files":[{"path":"bridge-extra-infos-2008-05.tar.xz","size":377644,"last_modified":"2016-09-04 09:21"},{"path":"bridge-extra-infos-2008-06.tar.xz","size":600484,"last_modified":"2016-09-04 09:21"},{"path":"bridge-extra-infos-2008-07.tar.xz","size":716320,"last_modified":"2016-09-04 09:21"}]},{"path":"server-descriptors","files":[{"path":"bridge-server-descriptors-2008-05.tar.xz","size":205348,"last_modified":"2016-09-09 14:13"},{"path":"bridge-server-descriptors-2008-06.tar.xz","size":342828,"last_modified":"2016-09-09 14:13"},{"path":"bridge-server-descriptors-2008-07.tar.xz","size":374848,"last_modified":"2016-09-09 14:13"}]},{"path":"statuses","files":[{"path":"bridge-statuses-2008-05.tar.xz","size":74792,"last_modified":"2016-09-14 21:11"},{"path":"bridge-statuse s-2008-06.tar.xz","size":123488,"last_modified":"2016-09-14 21:11"},{"path":"bridge-statuses-2008-07.tar.xz","size":143836,"last_modified":"2016-09-14 21:11"}]}]},{"path":"bridge-pool-assignments","files":[{"path":"bridge-pool-assignments-2010-09.tar.xz","size":32804,"last_modified":"2012-05-31 10:21"},{"path":"bridge-pool-assignments-2010-10.tar.xz","size":304684,"last_modified":"2012-05-31 10:21"},{"path":"bridge-pool-assignments-2010-11.tar.xz","size":292228,"last_modified":"2012-05-31 10:21"}]},{"path":"exit-lists","files":[{"path":"exit-list-2010-02.tar.xz","size":272008,"last_modified":"2012-05-31 18:57"},{"path":"exit-list-2010-03.tar.xz","size":1247484,"last_modified":"2012-05-31 18:57"},{"path":"exit-list-2010-04.tar.xz","size":1139896,"last_modified":"2012-05-31 18:57"}]},{"path":"relay-descriptors","files":[{"path":"certs.tar.xz","size":144696,"last_modified":"2019-07-03 03:29"}],"directories":[{"path":"bandwidths","files":[{"path":"bandwidths-2019-05.tar.xz","size":50385 816,"last_modified":"2019-06-07 08:05"},{"path":"bandwidths-2019-06.tar.xz","size":105881156,"last_modified":"2019-07-03 07:30"},{"path":"bandwidths-2019-07.tar.xz","size":11374436,"last_modified":"2019-07-03 07:12"}]},{"path":"consensuses","files":[{"path":"consensuses-2007-10.tar.xz","size":1061648,"last_modified":"2012-05-15 14:35"},{"path":"consensuses-2007-11.tar.xz","size":6810308,"last_modified":"2012-05-15 14:35"},{"path":"consensuses-2007-12.tar.xz","size":8106968,"last_modified":"2012-05-15 14:35"}]},{"path":"extra-infos","files":[{"path":"extra-infos-2007-08.tar.xz","size":3016916,"last_modified":"2016-06-23 09:53"},{"path":"extra-infos-2007-09.tar.xz","size":6459884,"last_modified":"2016-06-23 09:54"},{"path":"extra-infos-2007-10.tar.xz","size":7326564,"last_modified":"2016-06-23 09:54"}]},{"path":"microdescs","files":[{"path":"microdescs-2014-01.tar.xz","size":7515396,"last_modified":"2014-02-07 03:59"},{"path":"microdescs-2014-02.tar.xz","size":21822944,"last_modified" :"2014-03-07 04:54"},{"path":"microdescs-2014-03.tar.xz","size":24201436,"last_modified":"2014-04-07 03:54"}]},{"path":"server-descriptors","files":[{"path":"server-descriptors-2005-12.tar.xz","size":1348620,"last_modified":"2016-06-24 08:12"},{"path":"server-descriptors-2006-02.tar.xz","size":28625408,"last_modified":"2016-06-24 08:14"},{"path":"server-descriptors-2006-03.tar.xz","size":49548736,"last_modified":"2016-06-24 08:17"}]},{"path":"statuses","files":[{"path":"statuses-2005-12.tar.xz","size":1468844,"last_modified":"2016-06-25 11:50"},{"path":"statuses-2006-01.tar.xz","size":3344280,"last_modified":"2016-06-25 11:52"},{"path":"statuses-2006-02.tar.xz","size":4006336,"last_modified":"2016-06-25 11:54"}]},{"path":"tor","files":[{"path":"tor-2004-05.tar.xz","size":386672,"last_modified":"2012-05-18 14:26"},{"path":"tor-2004-06.tar.xz","size":1087980,"last_modified":"2012-05-18 14:26"},{"path":"tor-2004-07.tar.xz","size":1366568,"last_modified":"2012-05-18 14:26"}]},{"path":"v otes","files":[{"path":"votes-2007-10.tar.xz","size":1356504,"last_modified":"2012-05-15 14:51"},{"path":"votes-2007-11.tar.xz","size":10641492,"last_modified":"2012-05-15 14:51"},{"path":"votes-2007-12.tar.xz","size":14712136,"last_modified":"2012-05-15 14:52"}]}]},{"path":"torperf","files":[{"path":"torperf-2009-07.tar.xz","size":182712,"last_modified":"2012-05-30 07:23"},{"path":"torperf-2009-08.tar.xz","size":203236,"last_modified":"2012-05-30 07:23"},{"path":"torperf-2009-09.tar.xz","size":193832,"last_modified":"2012-05-30 07:23"}]},{"path":"webstats","files":[{"path":"webstats-2015-01.tar","size":30720,"last_modified":"2018-03-19 16:07"},{"path":"webstats-2015-02.tar","size":20480,"last_modified":"2018-03-19 16:07"},{"path":"webstats-2015-03.tar","size":20480,"last_modified":"2018-03-19 16:07"}]}]},{"path":"contrib"},{"path":"recent","directories":[{"path":"bridge-descriptors","directories":[{"path":"extra-infos","files":[{"path":"2019-07-03-02-09-00-extra-infos","size":50781 6,"last_modified":"2019-07-03 02:09"},{"path":"2019-07-03-03-09-00-extra-infos","size":558790,"last_modified":"2019-07-03 03:09"},{"path":"2019-07-03-04-09-05-extra-infos","size":592279,"last_modified":"2019-07-03 04:09"}]},{"path":"server-descriptors","files":[{"path":"2019-07-03-02-09-00-server-descriptors","size":274355,"last_modified":"2019-07-03 02:09"},{"path":"2019-07-03-03-09-00-server-descriptors","size":295671,"last_modified":"2019-07-03 03:09"},{"path":"2019-07-03-04-09-05-server-descriptors","size":312822,"last_modified":"2019-07-03 04:09"}]},{"path":"statuses","files":[{"path":"20190703-011231-BA44A889E64B93FAA2B114E02C2A279A8555C533","size":238340,"last_modified":"2019-07-03 02:09"},{"path":"20190703-014231-BA44A889E64B93FAA2B114E02C2A279A8555C533","size":238147,"last_modified":"2019-07-03 02:09"},{"path":"20190703-021231-BA44A889E64B93FAA2B114E02C2A279A8555C533","size":238231,"last_modified":"2019-07-03 03:09"}]}]},{"path":"exit-lists","files":[{"path":"2019-07-03-02- 02-00","size":158516,"last_modified":"2019-07-03 02:02"},{"path":"2019-07-03-03-02-00","size":158830,"last_modified":"2019-07-03 03:02"},{"path":"2019-07-03-04-02-00","size":158831,"last_modified":"2019-07-03 04:02"}]},{"path":"relay-descriptors","directories":[{"path":"bandwidths","files":[{"path":"2019-07-03-00-30-57-bandwidth-616DF2278EE2C90F475E4EA2562E2C00EB9F10E517FB901229F331AEB70B8AD7","size":2580044,"last_modified":"2019-07-03 02:35"},{"path":"2019-07-03-01-35-02-bandwidth-2A5B679E9AA2D5C903CD16C00E16FEFA3E21E38AB3DE338941AE4AC13FD505DE","size":4210146,"last_modified":"2019-07-03 02:35"},{"path":"2019-07-03-01-35-03-bandwidth-3B06DAD9EDE8E1D08CC494E856AFD9D49256ECC2D68456799DD1EC7ED3436875","size":4249699,"last_modified":"2019-07-03 02:35"}]},{"path":"consensuses","files":[{"path":"2019-07-03-02-00-00-consensus","size":2211265,"last_modified":"2019-07-03 02:05"},{"path":"2019-07-03-03-00-00-consensus","size":2204478,"last_modified":"2019-07-03 03:05"},{"path":"2019-07-03-04 -00-00-consensus","size":2202554,"last_modified":"2019-07-03 04:05"}]},{"path":"extra-infos","files":[{"path":"2019-07-03-02-05-00-extra-infos","size":1162899,"last_modified":"2019-07-03 02:05"},{"path":"2019-07-03-03-05-00-extra-infos","size":1133425,"last_modified":"2019-07-03 03:05"},{"path":"2019-07-03-04-05-00-extra-infos","size":1153793,"last_modified":"2019-07-03 04:05"}]},{"path":"microdescs","directories":[{"path":"consensus-microdesc","files":[{"path":"2019-07-03-02-00-00-consensus-microdesc","size":2028994,"last_modified":"2019-07-03 02:05"},{"path":"2019-07-03-03-00-00-consensus-microdesc","size":2022808,"last_modified":"2019-07-03 03:05"},{"path":"2019-07-03-04-00-00-consensus-microdesc","size":2020751,"last_modified":"2019-07-03 04:05"}]},{"path":"micro","files":[{"path":"2019-07-02-23-05-00-micro","size":19934,"last_modified":"2019-07-02 23:05"},{"path":"2019-07-03-00-05-00-micro","size":12030,"last_modified":"2019-07-03 00:05"},{"path":"2019-07-03-01-05-00-micro","si ze":14850,"last_modified":"2019-07-03 01:05"}]}]},{"path":"server-descriptors","files":[{"path":"2019-07-03-02-05-00-server-descriptors","size":1374587,"last_modified":"2019-07-03 02:05"},{"path":"2019-07-03-03-05-00-server-descriptors","size":1353286,"last_modified":"2019-07-03 03:05"},{"path":"2019-07-03-04-05-00-server-descriptors","size":1336125,"last_modified":"2019-07-03 04:05"}]},{"path":"votes","files":[{"path":"2019-07-03-02-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-D9B6923A3AF9FB5DB3CD30191BCBD29A7A936D86","size":3449707,"last_modified":"2019-07-03 02:05"},{"path":"2019-07-03-02-00-00-vote-14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4-39429F0789A9EC07A2A8754C4C449CCABAB787CC","size":3435482,"last_modified":"2019-07-03 02:05"},{"path":"2019-07-03-02-00-00-vote-23D15D965BC35114467363C165C4F724B64B4F66-4337842B697467123B5351E1E8EB64ED497C0EDA","size":3571106,"last_modified":"2019-07-03 02:05"}]}]},{"path":"torperf","files":[{"path":"op-ab-1048576-2019-07-03.tpf","size":3 9152,"last_modified":"2019-07-04 00:01"},{"path":"op-ab-51200-2019-07-03.tpf","size":248173,"last_modified":"2019-07-04 00:01"},{"path":"op-ab-5242880-2019-07-03.tpf","size":15900,"last_modified":"2019-07-04 00:01"}]},{"path":"webstats","files":[{"path":"2019.www.torproject.org_hetzner-hel1-03.torproject.org_access.log_20190620.xz","size":8872,"last_modified":"2019-07-03 04:21"},{"path":"2019.www.torproject.org_hetzner-hel1-03.torproject.org_access.log_20190621.xz","size":9312,"last_modified":"2019-07-04 04:21"},{"path":"2019.www.torproject.org_hetzner-hel1-03.torproject.org_access.log_20190622.xz","size":9836,"last_modified":"2019-07-05 04:21"}]}]}]}

1 0