April 2018 - tor-commits - lists.torproject.org

[translation/support-tormobile] Update translations for support-tormobile
by translation＠torproject.org 24 Apr '18

24 Apr '18

commit 80b49e1f6aa65c58f2515e298208b2a194ca29d3 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Apr 24 19:51:00 2018 +0000 Update translations for support-tormobile --- sk_SK.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sk_SK.json b/sk_SK.json index d832318fe..a20309d2d 100644 --- a/sk_SK.json +++ b/sk_SK.json @@ -2,7 +2,7 @@ "tormobile-1": { "id": "#tormobile-1", "control": "tormobile-1", - "title": "Can I run Tor on an Android device?", + "title": "Môžem použiť Tor na Android zariadení?", "description": "Tor on Android is provided by The Guardian Project. More information can be found on the <a href=\"https://guardianproject.info/apps/orbot/\">Orbot</a> and <a href=\"https://guardianproject.info/apps/orfox/\">Orfox</a> web pages. Orbot is for web browsing, and Orfox can route other apps on your Android phone over the Tor network." }, "tormobile-2": { @@ -14,7 +14,7 @@ "tormobile-3": { "id": "#tormobile-3", "control": "tormobile-3", - "title": "Can I run Tor on an iOS device?", + "title": "Môžem použiť Tor na iOS zariadení?", "description": " We recommend an iOS app called Onion Browser, which is open source, uses Tor routing, and is developed by someone who works closely with the Tor Project. However, Apple requires browsers on iOS to use something called Webkit, which prevents Onion Browser from having the same privacy protections as Tor Browser. <a href=\"https://blog.torproject.org/tor-heart-onion-browser-and-more-ios-tor\">Learn more about Onion Browser</a>." }, "tormobile-4": {

1 0

[translation/support-tormessenger] Update translations for support-tormessenger
by translation＠torproject.org 24 Apr '18

24 Apr '18

commit 9f1aa8af855fa6402481cf89ce56adb93d20579f Author: Translation commit bot <translation(a)torproject.org> Date: Tue Apr 24 19:50:52 2018 +0000 Update translations for support-tormessenger --- sk_SK.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sk_SK.json b/sk_SK.json index bbad42d5c..cf306f47a 100644 --- a/sk_SK.json +++ b/sk_SK.json @@ -8,13 +8,13 @@ "messenger-2": { "id": "#messenger-2", "control": "messenger-2", - "title": "What platforms is Tor Messenger available on?", - "description": "Tor Messenger (beta) is currently available on Linux, Windows, and OSX." + "title": "Na akých platformách je Tor Messenger dostupný?", + "description": "Tor Messenger (beta) je momentálne dostupný na Linuxe, Windowse, and OSX." }, "messenger-3": { "id": "#messenger-3", "control": "messenger-3", - "title": "Where can I download Tor Messenger?", + "title": "Kde môžem stiahnuť Tor Messenger?", "description": "While the Tor Messenger remains in beta, the latest downloads can be found on our wiki." } }

1 0

[translation/tor-browser-manual] Update translations for tor-browser-manual
by translation＠torproject.org 24 Apr '18

24 Apr '18

commit cd980709cf550fb035fa545a165a80471481ddd9 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Apr 24 19:49:48 2018 +0000 Update translations for tor-browser-manual --- sk_SK/sk_SK.po | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sk_SK/sk_SK.po b/sk_SK/sk_SK.po index 650eeb239..97040c44f 100644 --- a/sk_SK/sk_SK.po +++ b/sk_SK/sk_SK.po @@ -280,7 +280,7 @@ msgstr "" #: downloading.page:7 msgid "How to download Tor Browser" -msgstr "" +msgstr "Ako stiahuť Tor Browser" #: downloading.page:10 msgid "Downloading" @@ -795,7 +795,7 @@ msgstr "" #: onionsites.page:58 troubleshooting.page:10 msgid "Troubleshooting" -msgstr "" +msgstr "Riešenie problémov" #: onionsites.page:59 msgid "" @@ -1215,7 +1215,7 @@ msgstr "" #: troubleshooting.page:49 msgid "Temporarily disable your firewall." -msgstr "" +msgstr "Dočasne vypnúť firewall." #: troubleshooting.page:54 msgid ""

1 0

[translation/tor-launcher-network-settings] Update translations for tor-launcher-network-settings
by translation＠torproject.org 24 Apr '18

24 Apr '18

commit 60d22e53671bc4c9cb875b3068006ca407e73601 Author: Translation commit bot <translation(a)torproject.org> Date: Tue Apr 24 19:46:39 2018 +0000 Update translations for tor-launcher-network-settings --- sk_SK/network-settings.dtd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sk_SK/network-settings.dtd b/sk_SK/network-settings.dtd index c704f6ff7..ce4cb7e8c 100644 --- a/sk_SK/network-settings.dtd +++ b/sk_SK/network-settings.dtd @@ -25,7 +25,7 @@ <!ENTITY torsettings.optional "Voliteľné"> -<!ENTITY torsettings.useProxy.checkbox "I use a proxy to connect to the Internet"> +<!ENTITY torsettings.useProxy.checkbox "Na pripojenie k internetu používam proxy"> <!ENTITY torsettings.useProxy.type "Typ proxy:"> <!ENTITY torsettings.useProxy.type.placeholder "select a proxy type"> <!ENTITY torsettings.useProxy.address "Adresa:">

1 0

[translation/liveusb-creator] Update translations for liveusb-creator
by translation＠torproject.org 24 Apr '18

24 Apr '18

commit 4755db372b9da2e84111b4f22dd1b2fc0178af7f Author: Translation commit bot <translation(a)torproject.org> Date: Tue Apr 24 19:45:43 2018 +0000 Update translations for liveusb-creator --- sk_SK/sk_SK.po | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/sk_SK/sk_SK.po b/sk_SK/sk_SK.po index 0b632531f..43865ca00 100644 --- a/sk_SK/sk_SK.po +++ b/sk_SK/sk_SK.po @@ -6,13 +6,14 @@ # Daša Matúšová, 2016 # Marián Mižik <inactive+marian.mizik(a)transifex.com>, 2015 # once, 2014 +# Vlado Jendroľ, 2018 msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2017-11-10 15:57+0100\n" -"PO-Revision-Date: 2017-11-11 03:09+0000\n" -"Last-Translator: carolyn <carolyn(a)anhalt.org>\n" +"PO-Revision-Date: 2018-04-24 19:30+0000\n" +"Last-Translator: Vlado Jendroľ\n" "Language-Team: Slovak (Slovakia) (http://www.transifex.com/otf/torproject/language/sk_SK/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -318,7 +319,7 @@ msgstr "" #: ../tails_installer/gui.py:462 ../data/tails-installer.ui.h:7 msgid "Install" -msgstr "" +msgstr "Inštalovať" #: ../tails_installer/gui.py:469 #, python-format @@ -331,7 +332,7 @@ msgstr "" #: ../tails_installer/gui.py:482 msgid "Please select a Tails ISO image." -msgstr "" +msgstr "Vyberte prosím Tails ISO obraz." #: ../tails_installer/gui.py:521 msgid "No device suitable to install Tails could be found" @@ -359,7 +360,7 @@ msgstr "Zariadenie \"%(pretty_name)s\" je príliš malé na inštaláciu Tails ( #: ../tails_installer/gui.py:589 msgid "An error happened while installing Tails" -msgstr "" +msgstr "Pri inštalácii Tails nastala chyba" #: ../tails_installer/gui.py:601 msgid "Refreshing releases..." @@ -375,7 +376,7 @@ msgstr "Inštalácia kompletná!" #: ../tails_installer/gui.py:649 msgid "Installation was completed." -msgstr "" +msgstr "Inštalácia bola dokončená." #: ../tails_installer/gui.py:698 msgid "Unable to mount device" @@ -396,7 +397,7 @@ msgstr "" #: ../tails_installer/gui.py:722 #, python-format msgid "%(parent_size)s %(vendor)s %(model)s device (%(device)s)" -msgstr "" +msgstr "%(parent_size)s %(vendor)s %(model)s zariadenie (%(device)s)" #: ../tails_installer/gui.py:730 msgid "" @@ -408,7 +409,7 @@ msgstr "" #: ../tails_installer/gui.py:731 #, python-format msgid "%(description)s%(persistence_message)s" -msgstr "" +msgstr "%(description)s%(persistence_message)s" #: ../tails_installer/gui.py:781 msgid "Download complete!" @@ -459,17 +460,17 @@ msgstr "" #: ../tails_installer/source.py:63 #, python-format msgid "'%s' does not exist" -msgstr "" +msgstr "'%s' neexistuje" #: ../tails_installer/source.py:65 #, python-format msgid "'%s' is not a directory" -msgstr "" +msgstr "'%s' nie je priečinok" #: ../tails_installer/source.py:75 #, python-format msgid "Skipping '%(filename)s'" -msgstr "" +msgstr "Preskakuje sa '%(filename)s'" #: ../tails_installer/utils.py:44 #, python-format @@ -480,7 +481,7 @@ msgstr "" #: ../tails_installer/utils.py:124 msgid "Could not open device for writing." -msgstr "" +msgstr "Nepodarilo sa otvoriť zariadenia na zápis." #: ../data/tails-installer.ui.h:1 msgid "Installation Instructions" @@ -504,4 +505,4 @@ msgstr "" #: ../data/tails-installer.ui.h:6 msgid "Reinstall (delete all data)" -msgstr "" +msgstr "Preinštalovať (zmazať všetky dáta)"

1 0

[stem/master] Add a stem.Endpoint class
by atagar＠torproject.org 24 Apr '18

24 Apr '18

commit dac2c5092c8f95a14bacc657297fc80a0207fd93 Author: Damian Johnson <atagar(a)torproject.org> Date: Fri Apr 20 11:58:22 2018 -0700 Add a stem.Endpoint class Not yet used anywhere, but adding a little endpoint class so we can better differentiate ORPort vs DirPort endpoint arguments. --- stem/__init__.py | 42 ++++++++++++++++++++++++++++++++++++++++++ stem/util/connection.py | 2 ++ test/settings.cfg | 3 ++- test/unit/endpoint.py | 32 ++++++++++++++++++++++++++++++++ 4 files changed, 78 insertions(+), 1 deletion(-) diff --git a/stem/__init__.py b/stem/__init__.py index 853d704c..c8c43f65 100644 --- a/stem/__init__.py +++ b/stem/__init__.py @@ -618,6 +618,45 @@ class SocketClosed(SocketError): 'Control socket was closed before completing the message.' +class Endpoint(object): + """ + Tor endpint that can be connected to. + + .. versionadded:: 1.7.0 + + :var str address: ip address of the endpoint + :var int port: port of the endpoint + """ + + def __init__(self, address, port): + if not stem.util.connection.is_valid_ipv4_address(address) and not stem.util.connection.is_valid_ipv6_address(address): + raise ValueError("'%s' isn't a valid IPv4 or IPv6 address" % address) + elif not stem.util.connection.is_valid_port(port): + raise ValueError("'%s' isn't a valid port" % port) + + self.address = address + self.port = int(port) + + +class ORPort(Endpoint): + """ + Tor relay's ORPort. The endpoint on which Tor accepts relay traffic. + + :var list link_protocols: link protocol version we're willing to establish + """ + + def __init__(self, address, port, link_protocols = None): + super(ORPort, self).__init__(address, port) + self.link_protocols = link_protocols + + +class DirPort(Endpoint): + """ + Tor relay's DirPort. The endpoint on which Tor provides http access for + downloading descriptors. + """ + + Runlevel = stem.util.enum.UppercaseEnum( 'DEBUG', 'INFO', @@ -863,3 +902,6 @@ HSAuth = stem.util.enum.UppercaseEnum( 'STEALTH_AUTH', 'UNKNOWN', ) + + +import stem.util.connection # importing afterward to avoid circular dependency diff --git a/stem/util/connection.py b/stem/util/connection.py index 3395a3a5..74fbdde2 100644 --- a/stem/util/connection.py +++ b/stem/util/connection.py @@ -439,6 +439,8 @@ def is_valid_ipv6_address(address, allow_brackets = False): if isinstance(address, bytes): address = str_tools._to_unicode(address) + elif not isinstance(address, str_type): + return False if allow_brackets: if address.startswith('[') and address.endswith(']'): diff --git a/test/settings.cfg b/test/settings.cfg index aaab0790..f1cbd381 100644 --- a/test/settings.cfg +++ b/test/settings.cfg @@ -132,7 +132,7 @@ pycodestyle.ignore E127 pycodestyle.ignore E131 pycodestyle.ignore E722 -pycodestyle.ignore stem/__init__.py => E402: import stem.util.enum +pycodestyle.ignore stem/__init__.py => E402: import stem.util.connection pycodestyle.ignore stem/descriptor/__init__.py => E402: import stem.descriptor.server_descriptor pycodestyle.ignore stem/descriptor/__init__.py => E402: import stem.descriptor.extrainfo_descriptor pycodestyle.ignore stem/descriptor/__init__.py => E402: import stem.descriptor.networkstatus @@ -216,6 +216,7 @@ test.unit_tests |test.unit.descriptor.certificate.TestEd25519Certificate |test.unit.exit_policy.rule.TestExitPolicyRule |test.unit.exit_policy.policy.TestExitPolicy +|test.unit.endpoint.TestEndpoint |test.unit.version.TestVersion |test.unit.manual.TestManual |test.unit.tutorial.TestTutorial diff --git a/test/unit/endpoint.py b/test/unit/endpoint.py new file mode 100644 index 00000000..ad710a22 --- /dev/null +++ b/test/unit/endpoint.py @@ -0,0 +1,32 @@ +""" +Unit tests for the stem.Endpoint class. +""" + +import unittest + +import stem + + +class TestEndpoint(unittest.TestCase): + def test_constructor(self): + endpoint = stem.ORPort('12.34.56.78', 80) + self.assertEqual('12.34.56.78', endpoint.address) + self.assertEqual(80, endpoint.port) + self.assertEqual(None, endpoint.link_protocols) + + endpoint = stem.ORPort('12.34.56.78', 80, [3]) + self.assertEqual('12.34.56.78', endpoint.address) + self.assertEqual(80, endpoint.port) + self.assertEqual([3], endpoint.link_protocols) + + endpoint = stem.DirPort('12.34.56.78', 80) + self.assertEqual('12.34.56.78', endpoint.address) + self.assertEqual(80, endpoint.port) + + def test_validation(self): + self.assertRaises(ValueError, stem.DirPort, '12.34.56.78', 'hello') + self.assertRaises(ValueError, stem.DirPort, '12.34.56.78', -5) + self.assertRaises(ValueError, stem.DirPort, '12.34.56.78', None) + self.assertRaises(ValueError, stem.DirPort, 'hello', 80) + self.assertRaises(ValueError, stem.DirPort, -5, 80) + self.assertRaises(ValueError, stem.DirPort, None, 80)

1 0

[stem/master] Helper for downloading from dirports
by atagar＠torproject.org 24 Apr '18

24 Apr '18

commit 3b5343e8c35044a1c05af09d7153b7f259e6d979 Author: Damian Johnson <atagar(a)torproject.org> Date: Sat Apr 21 12:40:39 2018 -0700 Helper for downloading from dirports Our _download_descriptors() has grown pretty big. Downloading and decompressing DirPort responses are a good candidate for a little helper. --- stem/__init__.py | 13 +++++ stem/descriptor/remote.py | 107 +++++++++++++++++++++++------------------ test/unit/descriptor/remote.py | 3 +- test/unit/endpoint.py | 14 ++++++ 4 files changed, 89 insertions(+), 48 deletions(-) diff --git a/stem/__init__.py b/stem/__init__.py index c8c43f65..c7432d86 100644 --- a/stem/__init__.py +++ b/stem/__init__.py @@ -476,6 +476,7 @@ Library for working with the tor process. ================= =========== """ +import stem.util import stem.util.enum __version__ = '1.6.0-dev' @@ -637,6 +638,15 @@ class Endpoint(object): self.address = address self.port = int(port) + def __hash__(self): + return stem.util._hash_attr(self, 'address', 'port') + + def __eq__(self, other): + return hash(self) == hash(other) if isinstance(other, Endpoint) else False + + def __ne__(self, other): + return not self == other + class ORPort(Endpoint): """ @@ -649,6 +659,9 @@ class ORPort(Endpoint): super(ORPort, self).__init__(address, port) self.link_protocols = link_protocols + def __hash__(self): + return super(ORPort, self).__hash__() + stem.util._hash_attr(self, 'link_protocols', 'port') * 10 + class DirPort(Endpoint): """ diff --git a/stem/descriptor/remote.py b/stem/descriptor/remote.py index 5b942787..8835e8d9 100644 --- a/stem/descriptor/remote.py +++ b/stem/descriptor/remote.py @@ -236,6 +236,57 @@ def get_consensus(authority_v3ident = None, microdescriptor = False, **query_arg return get_instance().get_consensus(authority_v3ident, microdescriptor, **query_args) +def _download_from_dirport(url, compression, timeout): + """ + Downloads descriptors from the given url. + + :param str url: dirport url from which to download from + :param list compression: compression methods for the request + :param float timeout: duration before we'll time out our request + + :returns: two value tuple of the form (data, reply_headers) + + :raises: + * **socket.timeout** if our request timed out + * **urllib2.URLError** for most request failures + """ + + response = urllib.urlopen( + urllib.Request( + url, + headers = { + 'Accept-Encoding': ', '.join(compression), + 'User-Agent': 'Stem/%s' % stem.__version__, + } + ), + timeout = timeout, + ) + + data = response.read() + encoding = response.headers.get('Content-Encoding') + + # Tor doesn't include compression headers. As such when using gzip we + # need to include '32' for automatic header detection... + # + # https://stackoverflow.com/questions/3122145/zlib-error-error-3-while-decomp… + # + # ... and with zstd we need to use the streaming API. + + if encoding in (Compression.GZIP, 'deflate'): + data = zlib.decompress(data, zlib.MAX_WBITS | 32) + elif encoding == Compression.ZSTD and ZSTD_SUPPORTED: + output_buffer = io.BytesIO() + + with zstd.ZstdDecompressor().write_to(output_buffer) as decompressor: + decompressor.write(data) + + data = output_buffer.getvalue() + elif encoding == Compression.LZMA and LZMA_SUPPORTED: + data = lzma.decompress(data) + + return data.strip(), response.headers + + def _guess_descriptor_type(resource): # Attempts to determine the descriptor type based on the resource url. This # raises a ValueError if the resource isn't recognized. @@ -411,7 +462,7 @@ class Query(object): if endpoints: for endpoint in endpoints: if isinstance(endpoint, tuple) and len(endpoint) == 2: - self.endpoints.append(stem.DirPort(endpoint[0], endpoint[1])) + self.endpoints.append(stem.DirPort(endpoint[0], endpoint[1])) # TODO: remove this in stem 2.0 elif isinstance(endpoint, (stem.ORPort, stem.DirPort)): self.endpoints.append(endpoint) else: @@ -524,10 +575,10 @@ class Query(object): for desc in self._run(True): yield desc - def _pick_url(self, use_authority = False): + def _pick_endpoint(self, use_authority = False): """ - Provides a url that can be queried. If we have multiple endpoints then one - will be picked randomly. + Provides an endpoint to query. If we have multiple endpoints then one + is picked at random. :param bool use_authority: ignores our endpoints and uses a directory authority instead @@ -539,54 +590,18 @@ class Query(object): directories = get_authorities().values() picked = random.choice(list(directories)) - address, dirport = picked.address, picked.dir_port + return stem.DirPort(picked.address, picked.dir_port) else: - picked = random.choice(self.endpoints) - address, dirport = picked.address, picked.port - - return 'http://%s:%i/%s' % (address, dirport, self.resource.lstrip('/')) + return random.choice(self.endpoints) def _download_descriptors(self, retries, timeout): try: use_authority = retries == 0 and self.fall_back_to_authority - self.download_url = self._pick_url(use_authority) - self.start_time = time.time() - - response = urllib.urlopen( - urllib.Request( - self.download_url, - headers = { - 'Accept-Encoding': ', '.join(self.compression), - 'User-Agent': 'Stem/%s' % stem.__version__, - } - ), - timeout = timeout, - ) + endpoint = self._pick_endpoint(use_authority) + self.download_url = 'http://%s:%i/%s' % (endpoint.address, endpoint.port, self.resource.lstrip('/')) - data = response.read() - encoding = response.headers.get('Content-Encoding') - - # Tor doesn't include compression headers. As such when using gzip we - # need to include '32' for automatic header detection... - # - # https://stackoverflow.com/questions/3122145/zlib-error-error-3-while-decomp… - # - # ... and with zstd we need to use the streaming API. - - if encoding in (Compression.GZIP, 'deflate'): - data = zlib.decompress(data, zlib.MAX_WBITS | 32) - elif encoding == Compression.ZSTD and ZSTD_SUPPORTED: - output_buffer = io.BytesIO() - - with zstd.ZstdDecompressor().write_to(output_buffer) as decompressor: - decompressor.write(data) - - data = output_buffer.getvalue() - elif encoding == Compression.LZMA and LZMA_SUPPORTED: - data = lzma.decompress(data) - - self.content = data.strip() - self.reply_headers = response.headers + self.start_time = time.time() + self.content, self.reply_headers = _download_from_dirport(self.download_url, self.compression, timeout) self.runtime = time.time() - self.start_time log.trace("Descriptors retrieved from '%s' in %0.2fs" % (self.download_url, self.runtime)) except: diff --git a/test/unit/descriptor/remote.py b/test/unit/descriptor/remote.py index deb10060..b2cfa7c1 100644 --- a/test/unit/descriptor/remote.py +++ b/test/unit/descriptor/remote.py @@ -282,8 +282,7 @@ class TestDescriptorDownloader(unittest.TestCase): validate = True, ) - expeced_url = 'http://128.31.0.39:9131' + TEST_RESOURCE - self.assertEqual(expeced_url, query._pick_url()) + self.assertEqual(stem.DirPort('128.31.0.39', 9131), query._pick_endpoint()) descriptors = list(query) self.assertEqual(1, len(descriptors)) diff --git a/test/unit/endpoint.py b/test/unit/endpoint.py index ad710a22..397323a7 100644 --- a/test/unit/endpoint.py +++ b/test/unit/endpoint.py @@ -30,3 +30,17 @@ class TestEndpoint(unittest.TestCase): self.assertRaises(ValueError, stem.DirPort, 'hello', 80) self.assertRaises(ValueError, stem.DirPort, -5, 80) self.assertRaises(ValueError, stem.DirPort, None, 80) + + def test_equality(self): + self.assertTrue(stem.ORPort('12.34.56.78', 80) == stem.ORPort('12.34.56.78', 80)) + self.assertTrue(stem.ORPort('12.34.56.78', 80, [1, 2, 3]) == stem.ORPort('12.34.56.78', 80, [1, 2, 3])) + self.assertFalse(stem.ORPort('12.34.56.78', 80) == stem.ORPort('12.34.56.88', 80)) + self.assertFalse(stem.ORPort('12.34.56.78', 80) == stem.ORPort('12.34.56.78', 443)) + self.assertFalse(stem.ORPort('12.34.56.78', 80, [2, 3]) == stem.ORPort('12.34.56.78', 80, [1, 2, 3])) + + self.assertTrue(stem.DirPort('12.34.56.78', 80) == stem.DirPort('12.34.56.78', 80)) + self.assertFalse(stem.DirPort('12.34.56.78', 80) == stem.DirPort('12.34.56.88', 80)) + self.assertFalse(stem.DirPort('12.34.56.78', 80) == stem.DirPort('12.34.56.78', 443)) + + self.assertFalse(stem.ORPort('12.34.56.78', 80) == stem.DirPort('12.34.56.78', 80)) + self.assertFalse(stem.DirPort('12.34.56.78', 80) == stem.ORPort('12.34.56.78', 80))

1 0

[stem/master] Accepting ORPort and DirPort endpoints
by atagar＠torproject.org 24 Apr '18

24 Apr '18

commit 60f99f2b2056b8784d80599350c2be9158d2abac Author: Damian Johnson <atagar(a)torproject.org> Date: Fri Apr 20 12:15:20 2018 -0700 Accepting ORPort and DirPort endpoints We do *not* yet support actually using ORPort endpoints, but our constructor now takes them. ORPorts are treated like DirPorts (clearly wrong), but this will be followed up by actual support. --- stem/descriptor/remote.py | 24 ++++++++++++++++++++---- test/unit/descriptor/remote.py | 13 +++++++++++++ 2 files changed, 33 insertions(+), 4 deletions(-) diff --git a/stem/descriptor/remote.py b/stem/descriptor/remote.py index 89019c2d..5b942787 100644 --- a/stem/descriptor/remote.py +++ b/stem/descriptor/remote.py @@ -333,13 +333,18 @@ class Query(object): this was called httplib.HTTPMessage, whereas in python3 the class was renamed to http.client.HTTPMessage. + .. versionchanged:: 1.7.0 + Endpoints are now expected to be :class:`~stem.DirPort` or + :class:`~stem.ORPort` instances. Usage of tuples for this + argument is deprecated and will be removed in the future. + :var str resource: resource being fetched, such as '/tor/server/all' :var str descriptor_type: type of descriptors being fetched (for options see :func:`~stem.descriptor.__init__.parse_file`), this is guessed from the resource if **None** - :var list endpoints: (address, dirport) tuples of the authority or mirror - we're querying, this uses authorities if undefined + :var list endpoints: :class:`~stem.DirPort` or :class:`~stem.ORPort` of the + authority or mirror we're querying, this uses authorities if undefined :var list compression: list of :data:`stem.descriptor.remote.Compression` we're willing to accept, when none are mutually supported downloads fall back to Compression.PLAINTEXT @@ -401,9 +406,19 @@ class Query(object): else: self.descriptor_type = _guess_descriptor_type(resource) + self.endpoints = [] + + if endpoints: + for endpoint in endpoints: + if isinstance(endpoint, tuple) and len(endpoint) == 2: + self.endpoints.append(stem.DirPort(endpoint[0], endpoint[1])) + elif isinstance(endpoint, (stem.ORPort, stem.DirPort)): + self.endpoints.append(endpoint) + else: + raise ValueError("Endpoints must be an stem.ORPort, stem.DirPort, or two value tuple. '%s' is a %s." % (endpoint, type(endpoint).__name__)) + self.resource = resource self.compression = compression - self.endpoints = endpoints if endpoints else [] self.retries = retries self.fall_back_to_authority = fall_back_to_authority @@ -526,7 +541,8 @@ class Query(object): picked = random.choice(list(directories)) address, dirport = picked.address, picked.dir_port else: - address, dirport = random.choice(self.endpoints) + picked = random.choice(self.endpoints) + address, dirport = picked.address, picked.port return 'http://%s:%i/%s' % (address, dirport, self.resource.lstrip('/')) diff --git a/test/unit/descriptor/remote.py b/test/unit/descriptor/remote.py index a033d1c1..deb10060 100644 --- a/test/unit/descriptor/remote.py +++ b/test/unit/descriptor/remote.py @@ -3,6 +3,7 @@ Unit tests for stem.descriptor.remote. """ import io +import re import socket import tempfile import time @@ -343,6 +344,18 @@ class TestDescriptorDownloader(unittest.TestCase): self.assertRaises(socket.timeout, query.run) self.assertEqual(2, urlopen_mock.call_count) + def test_query_with_invalid_endpoints(self): + invalid_endpoints = { + 'hello': "'h' is a str.", + ('hello',): "'hello' is a str.", + (15,): "'15' is a int.", + (('12.34.56.78', 15, 'third arg'),): "'('12.34.56.78', 15, 'third arg')' is a tuple.", + } + + for endpoints, error_suffix in invalid_endpoints.items(): + expected_error = re.escape('Endpoints must be an stem.ORPort, stem.DirPort, or two value tuple. ' + error_suffix) + self.assertRaisesRegexp(ValueError, expected_error, stem.descriptor.remote.Query, TEST_RESOURCE, 'server-descriptor 1.0', endpoints = endpoints) + @patch(URL_OPEN, _urlopen_mock(TEST_DESCRIPTOR)) def test_can_iterate_multiple_times(self): query = stem.descriptor.remote.Query(

1 0

[stem/master] Download descriptors from ORPorts
by atagar＠torproject.org 24 Apr '18

24 Apr '18

commit 526f9bc4a9ff71acef9e7e521f1bd2f8e7e06589 Author: Damian Johnson <atagar(a)torproject.org> Date: Sat Apr 21 15:04:05 2018 -0700 Download descriptors from ORPorts Does it work? Haven't a clue, gotta write tests next. --- stem/descriptor/remote.py | 38 ++++++++++++++++++++++++++++++-------- 1 file changed, 30 insertions(+), 8 deletions(-) diff --git a/stem/descriptor/remote.py b/stem/descriptor/remote.py index 8835e8d9..eb9c7b62 100644 --- a/stem/descriptor/remote.py +++ b/stem/descriptor/remote.py @@ -107,6 +107,7 @@ import time import zlib import stem +import stem.client import stem.descriptor import stem.prereq import stem.util.enum @@ -236,6 +237,24 @@ def get_consensus(authority_v3ident = None, microdescriptor = False, **query_arg return get_instance().get_consensus(authority_v3ident, microdescriptor, **query_args) +def _download_from_orport(endpoint, resource): + """ + Downloads descriptors from the given orport. + + :param stem.ORPort endpoint: endpoint to download from + :param str resource: descriptor resource to download + + :returns: **str** with the descirptor data + + :raises: + """ + + with stem.client.Relay.connect(endpoint.address, endpoint.port, endpoint.link_protocols) as relay: + with relay.create_circuit() as circ: + circ.send('RELAY_BEGIN_DIR', stream_id = 1) + return circ.send('RELAY_DATA', resource, stream_id = 1).data + + def _download_from_dirport(url, compression, timeout): """ Downloads descriptors from the given url. @@ -587,21 +606,24 @@ class Query(object): """ if use_authority or not self.endpoints: - directories = get_authorities().values() - - picked = random.choice(list(directories)) + picked = random.choice(list(get_authorities().values())) return stem.DirPort(picked.address, picked.dir_port) else: return random.choice(self.endpoints) def _download_descriptors(self, retries, timeout): try: - use_authority = retries == 0 and self.fall_back_to_authority - endpoint = self._pick_endpoint(use_authority) - self.download_url = 'http://%s:%i/%s' % (endpoint.address, endpoint.port, self.resource.lstrip('/')) - self.start_time = time.time() - self.content, self.reply_headers = _download_from_dirport(self.download_url, self.compression, timeout) + endpoint = self._pick_endpoint(use_authority = retries == 0 and self.fall_back_to_authority) + + if isinstance(endpoint, stem.ORPort): + self.content = _download_from_orport(endpoint, self.resource) + elif isinstance(endpoint, stem.DirPort): + self.download_url = 'http://%s:%i/%s' % (endpoint.address, endpoint.port, self.resource.lstrip('/')) + self.content, self.reply_headers = _download_from_dirport(self.download_url, self.compression, timeout) + else: + raise ValueError("BUG: endpoints can only be ORPorts or DirPorts, '%s' was a %s" % (endpoint, type(endpoint).__name__)) + self.runtime = time.time() - self.start_time log.trace("Descriptors retrieved from '%s' in %0.2fs" % (self.download_url, self.runtime)) except:

1 0

[stem/master] Indicate attributes only applicable to DirPorts
by atagar＠torproject.org 24 Apr '18

24 Apr '18

commit 66a116face59321e2a92d6475be2e09810080ca7 Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Apr 23 10:24:48 2018 -0700 Indicate attributes only applicable to DirPorts Several attributes are only applicable when downloading from a DirPort. Especially in the case of compression and timeouts this isn't obvious (and in the later case I'd really like it if we *did* support timeouts at some point...). --- stem/descriptor/remote.py | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/stem/descriptor/remote.py b/stem/descriptor/remote.py index c191b02a..ebf9e786 100644 --- a/stem/descriptor/remote.py +++ b/stem/descriptor/remote.py @@ -435,9 +435,6 @@ class Query(object): :var list endpoints: :class:`~stem.DirPort` or :class:`~stem.ORPort` of the authority or mirror we're querying, this uses authorities if undefined - :var list compression: list of :data:`stem.descriptor.remote.Compression` - we're willing to accept, when none are mutually supported downloads fall - back to Compression.PLAINTEXT :var int retries: number of times to attempt the request if downloading it fails :var bool fall_back_to_authority: when retrying request issues the last @@ -446,11 +443,8 @@ class Query(object): :var str content: downloaded descriptor content :var Exception error: exception if a problem occured :var bool is_done: flag that indicates if our request has finished - :var str download_url: last url used to download the descriptor, this is - unset until we've actually made a download attempt :var float start_time: unix timestamp when we first started running - :var float timeout: duration before we'll time out our request :var float runtime: time our query took, this is **None** if it's not yet finished @@ -458,9 +452,20 @@ class Query(object): **True**, skips these checks otherwise :var stem.descriptor.__init__.DocumentHandler document_handler: method in which to parse a :class:`~stem.descriptor.networkstatus.NetworkStatusDocument` + :var dict kwargs: additional arguments for the descriptor constructor + + Following are only applicable when downloading from a + :class:`~stem.DirPort`... + + :var list compression: list of :data:`stem.descriptor.remote.Compression` + we're willing to accept, when none are mutually supported downloads fall + back to Compression.PLAINTEXT + :var float timeout: duration before we'll time out our request + + :var str download_url: last url used to download the descriptor, this is + unset until we've actually made a download attempt :var http.client.HTTPMessage reply_headers: headers provided in the response, **None** if we haven't yet made our request - :var dict kwargs: additional arguments for the descriptor constructor :param bool start: start making the request when constructed (default is **True**) :param bool block: only return after the request has been completed, this is

1 0