- tor-commits - lists.torproject.org

[translation/vidalia_help_completed] Update translations for vidalia_help_completed
by translation＠torproject.org 31 Dec '12

31 Dec '12

commit 89ad99d1797f2b1db84782ac14f5cc1f55879215 Author: Translation commit bot <translation(a)torproject.org> Date: Mon Dec 31 11:15:37 2012 +0000 Update translations for vidalia_help_completed --- es/services.po | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/es/services.po b/es/services.po index b282567..315d237 100644 --- a/es/services.po +++ b/es/services.po @@ -7,7 +7,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2010-06-26 17:00+0200\n" -"PO-Revision-Date: 2012-12-31 02:37+0000\n" +"PO-Revision-Date: 2012-12-31 11:09+0000\n" "Last-Translator: strel <strelnic(a)gmail.com>\n" "Language-Team: Spanish (http://www.transifex.com/projects/p/torproject/language/es/)\n" "MIME-Version: 1.0\n" @@ -126,7 +126,7 @@ msgid "" "service-specific port here, e.g. port 80 for HTTP. Note that the virtual " "port usually has nothing to do with firewall settings, because it is only " "used Tor-internally." -msgstr "Puerto virtual (requerido): Este es el puerto TCP que los clientes necesitarán conocer para acceder a su servicio oculto desde la red Tor. Es típico que quiera utilizar el puerto específico del servicio, por ejemplo, el puerto 80 para HTTP; pero observe que el puerto virtual, habitualmente, no tiene nada que ver con la configuración del cortafuegos (firewall), Tor sólo lo usa internamente para escuchar las peticiones a su servicio oculto en la red Tor, pero el tráfico que intercambia su servicio con la red Tor atraviesa su firewall por el puerto que usa su repetidor Tor." +msgstr "Puerto virtual (requerido): Este es el puerto TCP que los clientes necesitarán conocer para acceder a su servicio oculto desde la red Tor. Es típico que quiera utilizar el puerto específico del servicio, por ejemplo, el puerto 80 para HTTP; pero observe que el puerto virtual, habitualmente, no tiene nada que ver con la configuración del cortafuegos (firewall), Tor sólo lo usa internamente para escuchar las peticiones a su servicio oculto en la red Tor; el tráfico que intercambia su servicio con la red Tor atraviesa su firewall por el puerto que usa su repetidor Tor." #. type: Content of: <html><body><p><ul><li> #: en/services.html:73

1 0

[translation/vidalia_help] Update translations for vidalia_help
by translation＠torproject.org 31 Dec '12

31 Dec '12

commit 4f0e3478ffa0ca4b9b37a9571d74fedd085700bf Author: Translation commit bot <translation(a)torproject.org> Date: Mon Dec 31 11:15:27 2012 +0000 Update translations for vidalia_help --- es/services.po | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/es/services.po b/es/services.po index b282567..315d237 100644 --- a/es/services.po +++ b/es/services.po @@ -7,7 +7,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2010-06-26 17:00+0200\n" -"PO-Revision-Date: 2012-12-31 02:37+0000\n" +"PO-Revision-Date: 2012-12-31 11:09+0000\n" "Last-Translator: strel <strelnic(a)gmail.com>\n" "Language-Team: Spanish (http://www.transifex.com/projects/p/torproject/language/es/)\n" "MIME-Version: 1.0\n" @@ -126,7 +126,7 @@ msgid "" "service-specific port here, e.g. port 80 for HTTP. Note that the virtual " "port usually has nothing to do with firewall settings, because it is only " "used Tor-internally." -msgstr "Puerto virtual (requerido): Este es el puerto TCP que los clientes necesitarán conocer para acceder a su servicio oculto desde la red Tor. Es típico que quiera utilizar el puerto específico del servicio, por ejemplo, el puerto 80 para HTTP; pero observe que el puerto virtual, habitualmente, no tiene nada que ver con la configuración del cortafuegos (firewall), Tor sólo lo usa internamente para escuchar las peticiones a su servicio oculto en la red Tor, pero el tráfico que intercambia su servicio con la red Tor atraviesa su firewall por el puerto que usa su repetidor Tor." +msgstr "Puerto virtual (requerido): Este es el puerto TCP que los clientes necesitarán conocer para acceder a su servicio oculto desde la red Tor. Es típico que quiera utilizar el puerto específico del servicio, por ejemplo, el puerto 80 para HTTP; pero observe que el puerto virtual, habitualmente, no tiene nada que ver con la configuración del cortafuegos (firewall), Tor sólo lo usa internamente para escuchar las peticiones a su servicio oculto en la red Tor; el tráfico que intercambia su servicio con la red Tor atraviesa su firewall por el puerto que usa su repetidor Tor." #. type: Content of: <html><body><p><ul><li> #: en/services.html:73

1 0

[stem/master] Server descriptor parsing for 'ipv6-policy' lines
by atagar＠torproject.org 31 Dec '12

31 Dec '12

commit cdf2c13bbe5db5a85670760572f425dffb6ca897 Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Dec 31 00:33:30 2012 -0800 Server descriptor parsing for 'ipv6-policy' lines Adding support for IPv6 exit policies in server descriptors. Descriptors with these are already in the wild. https://trac.torproject.org/7826 --- stem/descriptor/server_descriptor.py | 4 ++++ test/unit/descriptor/server_descriptor.py | 12 +++++++++++- 2 files changed, 15 insertions(+), 1 deletions(-) diff --git a/stem/descriptor/server_descriptor.py b/stem/descriptor/server_descriptor.py index aad5e5e..634a705 100644 --- a/stem/descriptor/server_descriptor.py +++ b/stem/descriptor/server_descriptor.py @@ -146,6 +146,7 @@ class ServerDescriptor(stem.descriptor.Descriptor): :var int uptime: uptime when published in seconds :var str contact: contact information :var stem.exit_policy.ExitPolicy exit_policy: **\*** stated exit policy + :var stem.exit_policy.MicrodescriptorExitPolicy exit_policy_v6: **\*** exit policy for IPv6 :var list family: **\*** nicknames or fingerprints of declared family :var int average_bandwidth: **\*** average rate it's willing to relay in bytes/s @@ -212,6 +213,7 @@ class ServerDescriptor(stem.descriptor.Descriptor): self.uptime = None self.contact = None self.exit_policy = None + self.exit_policy_v6 = None self.family = [] self.average_bandwidth = None @@ -472,6 +474,8 @@ class ServerDescriptor(stem.descriptor.Descriptor): self.family = value.split(" ") elif keyword == "eventdns": self.eventdns = value == "1" + elif keyword == "ipv6-policy": + self.exit_policy_v6 = stem.exit_policy.MicrodescriptorExitPolicy(value) elif keyword == "or-address": or_address_entries = [value for (value, _) in values] diff --git a/test/unit/descriptor/server_descriptor.py b/test/unit/descriptor/server_descriptor.py index 9b0528a..e0a86fb 100644 --- a/test/unit/descriptor/server_descriptor.py +++ b/test/unit/descriptor/server_descriptor.py @@ -6,8 +6,9 @@ import datetime import StringIO import unittest -import stem.prereq import stem.descriptor.server_descriptor +import stem.exit_policy +import stem.prereq import test.runner from stem.descriptor.server_descriptor import RelayDescriptor, BridgeDescriptor @@ -258,6 +259,15 @@ class TestServerDescriptor(unittest.TestCase): desc_text = get_relay_server_descriptor({"opt fingerprint": fingerprint}, content = True) self._expect_invalid_attr(desc_text, "fingerprint", fingerprint.replace(" ", "")) + def test_ipv6_policy(self): + """ + Checks a 'ipv6-policy' line. + """ + + expected = stem.exit_policy.MicrodescriptorExitPolicy("accept 22-23,53,80,110") + desc = get_relay_server_descriptor({"ipv6-policy": "accept 22-23,53,80,110"}) + self.assertEquals(expected, desc.exit_policy_v6) + def test_minimal_bridge_descriptor(self): """ Basic sanity check that we can parse a descriptor with minimal attributes.

1 0

[stem/master] Descriptor reader callbacks for when reading a file
by atagar＠torproject.org 31 Dec '12

31 Dec '12

commit 974b71c65a3e67da25254418856c4a65c5b0993a Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Dec 31 00:07:34 2012 -0800 Descriptor reader callbacks for when reading a file While writing a script for 'https://trac.torproject.org/7828' I realized that it would be nice to be able to add hooks for when reading a file. --- stem/descriptor/reader.py | 22 ++++++++++++++++++++++ 1 files changed, 22 insertions(+), 0 deletions(-) diff --git a/stem/descriptor/reader.py b/stem/descriptor/reader.py index 95a1f8b..c1c32ed 100644 --- a/stem/descriptor/reader.py +++ b/stem/descriptor/reader.py @@ -59,6 +59,7 @@ and picks up where it left off if ran again... DescriptorReader - Iterator for descriptor data on the local file system |- get_processed_files - provides the listing of files that we've processed |- set_processed_files - sets our tracking of the files we have processed + |- register_read_listener - adds a listener for when files are read |- register_skip_listener - adds a listener that's notified of skipped files |- start - begins reading descriptor data |- stop - stops reading descriptor data @@ -247,6 +248,7 @@ class DescriptorReader(object): self._follow_links = follow_links self._persistence_path = persistence_path + self._read_listeners = [] self._skip_listeners = [] self._processed_files = {} @@ -303,6 +305,20 @@ class DescriptorReader(object): self._processed_files = dict(processed_files) + def register_read_listener(self, listener): + """ + Registers a listener for when files are read. This is executed prior to + processing files. Listeners are expected to be of the form... + + :: + + my_listener(path) + + :param functor listener: functor to be notified when files are read + """ + + self._read_listeners.append(listener) + def register_skip_listener(self, listener): """ Registers a listener for files that are skipped. This listener is expected @@ -464,6 +480,7 @@ class DescriptorReader(object): def _handle_descriptor_file(self, target, mime_type): try: + self._notify_read_listeners(target) with open(target) as target_file: for desc in stem.descriptor.parse_file(target, target_file): if self._is_stopped.isSet(): return @@ -484,6 +501,7 @@ class DescriptorReader(object): tar_file = None try: + self._notify_read_listeners(target) tar_file = tarfile.open(target) for tar_entry in tar_file: @@ -505,6 +523,10 @@ class DescriptorReader(object): finally: if tar_file: tar_file.close() + def _notify_read_listeners(self, path): + for listener in self._read_listeners: + listener(path) + def _notify_skip_listeners(self, path, exception): for listener in self._skip_listeners: listener(path, exception)

1 0

[stem/master] Network status integ tests should check for unrecognized lines
by atagar＠torproject.org 31 Dec '12

31 Dec '12

commit 87b05342286546f603ea4bca2aee68e2b95d875e Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Dec 31 00:12:40 2012 -0800 Network status integ tests should check for unrecognized lines Probably the most useful integration test that we have is the check for unrecognized lines in the server descriptors. That one test has found dozens of upstream tor bugs (mostly new additions without spec updates). It turns out that we were missing this check for network statuses, and it's already tripping on an issue. Oops. --- test/integ/descriptor/networkstatus.py | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/test/integ/descriptor/networkstatus.py b/test/integ/descriptor/networkstatus.py index 6355220..aa20169 100644 --- a/test/integ/descriptor/networkstatus.py +++ b/test/integ/descriptor/networkstatus.py @@ -50,6 +50,11 @@ class TestNetworkStatus(unittest.TestCase): for flag in router.flags: if not flag in stem.descriptor.Flag: raise ValueError("Unrecognized flag type: %s, found on relay %s (%s)" % (flag, router.fingerprint, router.nickname)) + + unrecognized_lines = router.get_unrecognized_lines() + + if unrecognized_lines: + self.fail("Unrecognized descriptor content: %s" % unrecognized_lines) # Sanity test that there's at least a hundred relays. If that's not the # case then this probably isn't a real, complete tor consensus. @@ -87,6 +92,11 @@ class TestNetworkStatus(unittest.TestCase): for flag in router.flags: if not flag in stem.descriptor.Flag: raise ValueError("Unrecognized flag type: %s, found on microdescriptor relay %s (%s)" % (flag, router.fingerprint, router.nickname)) + + unrecognized_lines = router.get_unrecognized_lines() + + if unrecognized_lines: + self.fail("Unrecognized descriptor content: %s" % unrecognized_lines) self.assertTrue(count > 100)

1 0

[stem/master] Support for consensus 'a' lines
by atagar＠torproject.org 31 Dec '12

31 Dec '12

commit 683b1ba479f2aff3c277e0ff53bdc8b1f81664af Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Dec 31 01:43:00 2012 -0800 Support for consensus 'a' lines Adding support for the 'a' lines in version 3 router status entries. These contain IPv6 addresses and port lists. The port lists will make this unweildy for users but that's out of our hands. --- stem/descriptor/router_status_entry.py | 46 ++++++++++++++++++++++--- test/unit/descriptor/router_status_entry.py | 49 +++++++++++++++++++++++++- 2 files changed, 87 insertions(+), 8 deletions(-) diff --git a/stem/descriptor/router_status_entry.py b/stem/descriptor/router_status_entry.py index 31458c1..ed01132 100644 --- a/stem/descriptor/router_status_entry.py +++ b/stem/descriptor/router_status_entry.py @@ -270,6 +270,8 @@ class RouterStatusEntryV3(RouterStatusEntry): Information about an individual router stored within a version 3 network status document. + :var dict addresses_v6: **\*** relay's IPv6 OR addresses, this is a mapping + of IPv6 addresses to a listing of [(min port, max port)...] it accepts :var str digest: **\*** router's digest :var int bandwidth: bandwidth claimed by the relay (in kb/s) @@ -279,14 +281,16 @@ class RouterStatusEntryV3(RouterStatusEntry): :var stem.exit_policy.MicrodescriptorExitPolicy exit_policy: router's exit policy - :var list microdescriptor_hashes: tuples of two values, the list of consensus - methods for generating a set of digests and the 'algorithm => digest' mappings + :var list microdescriptor_hashes: **\*** tuples of two values, the list of + consensus methods for generating a set of digests and the 'algorithm => + digest' mappings **\*** attribute is either required when we're parsed with validation or has a default value, others are left as **None** if undefined """ def __init__(self, content, validate = True, document = None): + self.addresses_v6 = {} self.digest = None self.bandwidth = None @@ -294,7 +298,7 @@ class RouterStatusEntryV3(RouterStatusEntry): self.unrecognized_bandwidth_entries = [] self.exit_policy = None - self.microdescriptor_hashes = None + self.microdescriptor_hashes = [] super(RouterStatusEntryV3, self).__init__(content, validate, document) @@ -305,6 +309,11 @@ class RouterStatusEntryV3(RouterStatusEntry): if keyword == 'r': _parse_r_line(self, value, validate, True) del entries['r'] + elif keyword == 'a': + for entry, _ in values: + _parse_a_line(self, entry, validate) + + del entries['a'] elif keyword == 'w': _parse_w_line(self, value, validate) del entries['w'] @@ -449,6 +458,34 @@ def _parse_r_line(desc, value, validate, include_digest = True): if validate: raise ValueError("Publication time time wasn't parsable: r %s" % value) +def _parse_a_line(desc, value, validate): + # "a" SP address ":" portlist + # example: a [2001:888:2133:0:82:94:251:204]:9001 + + if not ':' in value: + if not validate: return + raise ValueError("%s 'a' line must be of the form '[address]:[ports]': a %s" % (desc._name(), value)) + + address, ports = value.rsplit(':', 1) + + if validate and not stem.util.connection.is_valid_ipv6_address(address, allow_brackets = True): + raise ValueError("%s 'a' line must start with an IPv6 address: a %s" % (desc._name(), value)) + + address = address.lstrip('[').rstrip(']') + + for port_entry in ports.split(','): + if '-' in port_entry: + min_port, max_port = port_entry.split('-', 1) + else: + min_port = max_port = port_entry + + if not stem.util.connection.is_valid_port(min_port) or \ + not stem.util.connection.is_valid_port(max_port): + if not validate: continue + raise ValueError("%s 'a' line had an invalid port range (%s): a %s" % (desc._name(), port_entry, value)) + + desc.addresses_v6.setdefault(address, []).append((int(min_port), int(max_port))) + def _parse_s_line(desc, value, validate): # "s" Flags # example: s Named Running Stable Valid @@ -556,9 +593,6 @@ def _parse_m_line(desc, value, validate): hash_name, digest = entry.split('=', 1) hashes[hash_name] = digest - if desc.microdescriptor_hashes is None: - desc.microdescriptor_hashes = [] - desc.microdescriptor_hashes.append((methods, hashes)) def _decode_fingerprint(identity, validate): diff --git a/test/unit/descriptor/router_status_entry.py b/test/unit/descriptor/router_status_entry.py index b0cd3b8..ef03baa 100644 --- a/test/unit/descriptor/router_status_entry.py +++ b/test/unit/descriptor/router_status_entry.py @@ -82,7 +82,7 @@ class TestRouterStatusEntry(unittest.TestCase): self.assertEqual(None, entry.measured) self.assertEqual([], entry.unrecognized_bandwidth_entries) self.assertEqual(None, entry.exit_policy) - self.assertEqual(None, entry.microdescriptor_hashes) + self.assertEqual([], entry.microdescriptor_hashes) self.assertEqual([], entry.get_unrecognized_lines()) def test_minimal_micro_v3(self): @@ -304,6 +304,51 @@ class TestRouterStatusEntry(unittest.TestCase): content = get_router_status_entry_v3({'r': r_line}, content = True) self._expect_invalid_attr(content, attr, expected) + def test_ipv6_addresses(self): + """ + Handles a variety of 'a' lines. + """ + + test_values = { + "[2607:fcd0:daaa:101::602c:bd62]:443": { + '2607:fcd0:daaa:101::602c:bd62': [(443, 443)]}, + "[2607:fcd0:daaa:101::602c:bd62]:80,443": { + '2607:fcd0:daaa:101::602c:bd62': [(80, 80), (443, 443)]}, + "[2607:fcd0:daaa:101::602c:bd62]:443-512": { + '2607:fcd0:daaa:101::602c:bd62': [(443, 512)]}, + } + + for a_line, expected in test_values.items(): + entry = get_router_status_entry_v3({'a': a_line}) + self.assertEquals(expected, entry.addresses_v6) + + # includes multiple 'a' lines + + content = get_router_status_entry_v3(content = True) + content += "\na [2607:fcd0:daaa:101::602c:bd62]:80,443" + content += "\na [2607:fcd0:daaa:101::602c:bd62]:512-600" + content += "\na [1148:fcd0:daaa:101::602c:bd62]:80" + + expected = { + '2607:fcd0:daaa:101::602c:bd62': [(80, 80), (443, 443), (512, 600)], + '1148:fcd0:daaa:101::602c:bd62': [(80, 80)], + } + + entry = RouterStatusEntryV3(content) + self.assertEquals(expected, entry.addresses_v6) + + # tries some invalid inputs + + test_values = ( + "", + "127.0.0.1:80", + "[1148:fcd0:daaa:101::602c:bd62]:80000", + ) + + for a_line in test_values: + content = get_router_status_entry_v3({'a': a_line}, content = True) + self._expect_invalid_attr(content, expected_value = {}) + def test_flags(self): """ Handles a variety of flag inputs. @@ -454,7 +499,7 @@ class TestRouterStatusEntry(unittest.TestCase): # try without a document content = get_router_status_entry_v3({'m': "8,9,10,11,12"}, content = True) - self._expect_invalid_attr(content, "microdescriptor_hashes") + self._expect_invalid_attr(content, "microdescriptor_hashes", expected_value = []) # tries some invalid inputs test_values = (

1 0

[stem/master] Duplicate network status 'm' lines were ignored
by atagar＠torproject.org 31 Dec '12

31 Dec '12

commit 7df236509dca6d32d5cebb8aa0b634d3b3d962b1 Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Dec 31 00:47:58 2012 -0800 Duplicate network status 'm' lines were ignored The 'm' lines in version 3 router status entries can appear multiple times, but we were only parsing the first instance. --- stem/descriptor/router_status_entry.py | 4 +++- test/unit/descriptor/router_status_entry.py | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 1 deletions(-) diff --git a/stem/descriptor/router_status_entry.py b/stem/descriptor/router_status_entry.py index 2d02330..31458c1 100644 --- a/stem/descriptor/router_status_entry.py +++ b/stem/descriptor/router_status_entry.py @@ -312,7 +312,9 @@ class RouterStatusEntryV3(RouterStatusEntry): _parse_p_line(self, value, validate) del entries['p'] elif keyword == 'm': - _parse_m_line(self, value, validate) + for entry, _ in values: + _parse_m_line(self, entry, validate) + del entries['m'] RouterStatusEntry._parse(self, entries, validate) diff --git a/test/unit/descriptor/router_status_entry.py b/test/unit/descriptor/router_status_entry.py index 8a7e711..b0cd3b8 100644 --- a/test/unit/descriptor/router_status_entry.py +++ b/test/unit/descriptor/router_status_entry.py @@ -438,6 +438,20 @@ class TestRouterStatusEntry(unittest.TestCase): entry = RouterStatusEntryV3(content, document = mock_document) self.assertEquals(expected, entry.microdescriptor_hashes) + # try with multiple 'm' lines + + content = get_router_status_entry_v3(content = True) + content += "\nm 11,12 sha256=g1vx9si329muxV3tquWIXXySNOIwRGMeAESKs/v4DWs" + content += "\nm 31,32 sha512=g1vx9si329muxV3tquWIXXySNOIwRGMeAESKs/v4DWs" + + expected = [ + ([11, 12], {"sha256": "g1vx9si329muxV3tquWIXXySNOIwRGMeAESKs/v4DWs"}), + ([31, 32], {"sha512": "g1vx9si329muxV3tquWIXXySNOIwRGMeAESKs/v4DWs"}), + ] + + entry = RouterStatusEntryV3(content, document = mock_document) + self.assertEquals(expected, entry.microdescriptor_hashes) + # try without a document content = get_router_status_entry_v3({'m': "8,9,10,11,12"}, content = True) self._expect_invalid_attr(content, "microdescriptor_hashes")

1 0

[stem/master] Lazily parsing ExitPolicy
by atagar＠torproject.org 31 Dec '12

31 Dec '12

commit da65f282699d3e8e763e1f3ba3c7612cc970178a Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Dec 30 21:00:50 2012 -0800 Lazily parsing ExitPolicy Why parse exit policies that we never use? ExitPolicy instances exist numerous places, note notably in the descriptors. Users may never use or care about the exit policy, so there's no point in putting effort into parsing them until they're needed. The advantages is that this means faster untime and less memory usage if they go unused. Down side is that we won't pick up on malformed content until they're used (handy for tests, but not somethign that users care about). This dropped the runtime for the server descriptor integ tests on my system from 10.1 to 7.7 seconds. --- stem/exit_policy.py | 40 ++++++++++++++++++++++++++-------------- 1 files changed, 26 insertions(+), 14 deletions(-) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index e1f68b7..462a0b5 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -86,16 +86,13 @@ class ExitPolicy(object): """ def __init__(self, *rules): - self._rules = [] - + # sanity check the types for rule in rules: - if isinstance(rule, str): - self._rules.append(ExitPolicyRule(rule.strip())) - elif isinstance(rule, ExitPolicyRule): - self._rules.append(rule) - else: + if not (isinstance(rule, str) or isinstance(rule, ExitPolicyRule)): raise TypeError("Exit policy rules can only contain strings or ExitPolicyRules, got a %s (%s)" % (type(rule), rules)) + self._rules = None # lazily loaded series of ExitPolicyRule + self._input_rules = rules # input rules, only kept until self._rules is set self._is_allowed_default = True self._summary_representation = None @@ -111,7 +108,7 @@ class ExitPolicy(object): :returns: **True** if exiting to this destination is allowed, **False** otherwise """ - for rule in self._rules: + for rule in self._get_rules(): if rule.is_match(address, port): return rule.is_accept @@ -124,7 +121,7 @@ class ExitPolicy(object): """ rejected_ports = set() - for rule in self._rules: + for rule in self._get_rules(): if rule.is_accept: for port in xrange(rule.min_port, rule.max_port + 1): if not port in rejected_ports: @@ -161,7 +158,7 @@ class ExitPolicy(object): # determines if we're a white-list or blacklist is_whitelist = not self._is_allowed_default - for rule in self._rules: + for rule in self._get_rules(): if rule.is_address_wildcard() and rule.is_port_wildcard(): is_whitelist = not rule.is_accept break @@ -173,7 +170,7 @@ class ExitPolicy(object): display_ports, skip_ports = [], set() - for rule in self._rules: + for rule in self._get_rules(): if not rule.is_address_wildcard(): continue elif rule.is_port_wildcard(): break @@ -231,16 +228,31 @@ class ExitPolicy(object): self._is_allowed_default = is_allowed_default + def _get_rules(self): + if self._rules is None: + rules = [] + + for rule in self._input_rules: + if isinstance(rule, str): + rules.append(ExitPolicyRule(rule.strip())) + elif isinstance(rule, ExitPolicyRule): + rules.append(rule) + + self._rules = rules + self._input_rules = None + + return self._rules + def __iter__(self): - for rule in self._rules: + for rule in self._get_rules(): yield rule def __str__(self): - return ', '.join([str(rule) for rule in self._rules]) + return ', '.join([str(rule) for rule in self._get_rules()]) def __eq__(self, other): if isinstance(other, ExitPolicy): - return self._rules == list(other) + return self._get_rules() == list(other) else: return False

1 0

[stem/master] Using a lightweight rule counterpart for microdescriptor policies
by atagar＠torproject.org 31 Dec '12

31 Dec '12

commit d35b9c737e320500fa6cfdfa874534cfbe6f1b3a Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Dec 30 21:26:34 2012 -0800 Using a lightweight rule counterpart for microdescriptor policies The exit policies used in microdescriptors are wonderfully simple. There's no point in storing full ExitPolicyRule instances for them, so making an alternate lightweight subclass. Advantages are faster for parsing and less memory usage. Disadvantage is that this doesn't use the ExitPolicyRule constructor, so we'll need to take care that ExitPolicyRule changes are reflected in this subclass. This dropped the runtime of the network status integ tests on my system from 9.4 to 8.3 seconds. --- stem/exit_policy.py | 43 ++++++++++++++++++++++++++++++++++--------- 1 files changed, 34 insertions(+), 9 deletions(-) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index 462a0b5..d7e091f 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -308,19 +308,20 @@ class MicrodescriptorExitPolicy(ExitPolicy): policy = policy[1:] - # convert our port list into ExitPolicyRules + # convert our port list into MicroExitPolicyRule rules = [] - rule_format = "accept *:%s" if self.is_accept else "reject *:%s" for port_entry in policy.split(","): - rule_str = rule_format % port_entry + if '-' in port_entry: + min_port, max_port = port_entry.split('-', 1) + else: + min_port = max_port = port_entry + + if not stem.util.connection.is_valid_port(min_port) or \ + not stem.util.connection.is_valid_port(max_port): + raise ValueError("'%s' is an invalid port range" % port_entry) - try: - rule = ExitPolicyRule(rule_str) - rules.append(rule) - except ValueError, exc: - exc_msg = "Policy '%s' is malformed. %s" % (self._policy, str(exc).replace(rule_str, port_entry)) - raise ValueError(exc_msg) + rules.append(MicroExitPolicyRule(self.is_accept, int(min_port), int(max_port))) super(MicrodescriptorExitPolicy, self).__init__(*rules) self._set_default_allowed(not self.is_accept) @@ -694,3 +695,27 @@ def _address_type_to_int(address_type): def _int_to_address_type(address_type_int): return AddressType[AddressType.keys()[address_type_int]] +class MicroExitPolicyRule(ExitPolicyRule): + """ + Lighter weight ExitPolicyRule derivative for microdescriptors. + """ + + def __init__(self, is_accept, min_port, max_port): + self.is_accept = is_accept + self.address = None # wildcard address + self.min_port = min_port + self.max_port = max_port + self._str_representation = None + + def is_address_wildcard(self): + return True + + def get_address_type(self): + return AddressType.WILDCARD + + def get_mask(self, cache = True): + return None + + def get_masked_bits(self): + return None +

1 0

[stem/master] Tor 0.2.2.9 required for tests with new/extend_circuit()
by atagar＠torproject.org 31 Dec '12

31 Dec '12

commit c34e90b55d744b0e19922d61b638c8987b089233 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Dec 30 19:39:54 2012 -0800 Tor 0.2.2.9 required for tests with new/extend_circuit() Our tests utilizing new_circuit() and extend_circuit() do not provide an explicit path. This breaks the tests if tor isn't version 0.2.2.9 or later. https://trac.torproject.org/7833 --- test/integ/control/controller.py | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-) diff --git a/test/integ/control/controller.py b/test/integ/control/controller.py index c8055ab..534bba2 100644 --- a/test/integ/control/controller.py +++ b/test/integ/control/controller.py @@ -24,6 +24,7 @@ import test.runner import test.util from stem.control import EventType +from stem.version import Requirement class TestController(unittest.TestCase): def test_from_port(self): @@ -515,6 +516,7 @@ class TestController(unittest.TestCase): def test_extendcircuit(self): if test.runner.require_control(self): return elif test.runner.require_online(self): return + elif test.runner.require_version(self, Requirement.EXTENDCIRCUIT_PATH_OPTIONAL): return with test.runner.get_runner().get_tor_controller() as controller: circuit_id = controller.extend_circuit('0') @@ -535,6 +537,7 @@ class TestController(unittest.TestCase): if test.runner.require_control(self): return elif test.runner.require_online(self): return + elif test.runner.require_version(self, Requirement.EXTENDCIRCUIT_PATH_OPTIONAL): return runner = test.runner.get_runner() @@ -558,6 +561,7 @@ class TestController(unittest.TestCase): if test.runner.require_control(self): return elif test.runner.require_online(self): return + elif test.runner.require_version(self, Requirement.EXTENDCIRCUIT_PATH_OPTIONAL): return runner = test.runner.get_runner() @@ -722,6 +726,7 @@ class TestController(unittest.TestCase): def test_attachstream(self): if test.runner.require_control(self): return elif test.runner.require_online(self): return + elif test.runner.require_version(self, Requirement.EXTENDCIRCUIT_PATH_OPTIONAL): return circuit_id = None @@ -753,7 +758,8 @@ class TestController(unittest.TestCase): """ if test.runner.require_control(self): return - if test.runner.require_online(self): return + elif test.runner.require_online(self): return + elif test.runner.require_version(self, Requirement.EXTENDCIRCUIT_PATH_OPTIONAL): return runner = test.runner.get_runner() with runner.get_tor_controller() as controller:

1 0