January 2013 - tor-commits - lists.torproject.org

[tor/master] Merge branch 'bug7935'
by nickm＠torproject.org 14 Jan '13

14 Jan '13

commit dab25eb37dd7d5245a03a5380ba5f80c64e4ac3d Merge: dffc8e3 a5ee383 Author: Nick Mathewson <nickm(a)torproject.org> Date: Sun Jan 13 21:43:53 2013 -0500 Merge branch 'bug7935' changes/bug7935 | 4 ++++ src/common/compat.h | 7 ++++++- src/ext/eventdns.c | 4 ++++ 3 files changed, 14 insertions(+), 1 deletions(-)

1 0

[tor/master] Handle EWOULDBLOCK as EAGAIN if they happen to be different.
by nickm＠torproject.org 14 Jan '13

14 Jan '13

commit a5ee3834bf060c8a238be84ea7287bdfa8303a27 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jan 11 16:27:45 2013 -0800 Handle EWOULDBLOCK as EAGAIN if they happen to be different. Fixes bug 7935. Reported by 'oftc_must_be_destroyed'. --- changes/bug7935 | 4 ++++ src/common/compat.h | 7 ++++++- src/ext/eventdns.c | 4 ++++ 3 files changed, 14 insertions(+), 1 deletions(-) diff --git a/changes/bug7935 b/changes/bug7935 new file mode 100644 … [View More]

1 0

[stem/master] Providing ExitPolicy via get_config_policy()
by atagar＠torproject.org 14 Jan '13

14 Jan '13

commit f264a626d682f6b8a4cdea8b58a260b1c1069524 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 13 16:33:03 2013 -0800 Providing ExitPolicy via get_config_policy() On reflection it desn't make sense to provide a list of ExitPolicyRules. Usually we want a policy itself, and if we do want a rule list then that's easily done via 'list(get_config_policy())'. --- stem/exit_policy.py | 4 +- test/unit/exit_policy/policy.py | 42 +++++++++++… [View More]++++++--------------------- 2 files changed, 21 insertions(+), 25 deletions(-) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index 3bccc27..b5d0573 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -97,7 +97,7 @@ def get_config_policy(rules): :param str,list rules: comma separated rules or list to be converted - :returns: **list** of :class:`~stem.exit_policy.ExitPolicyRule` + :returns: :class:`~stem.exit_policy.ExitPolicy` reflected by the rules :raises: **ValueError** if input isn't a valid tor exit policy """ @@ -125,7 +125,7 @@ def get_config_policy(rules): else: result.append(ExitPolicyRule(rule)) - return result + return ExitPolicy(*result) class ExitPolicy(object): diff --git a/test/unit/exit_policy/policy.py b/test/unit/exit_policy/policy.py index b8ac286..af4ceba 100644 --- a/test/unit/exit_policy/policy.py +++ b/test/unit/exit_policy/policy.py @@ -199,29 +199,25 @@ class TestExitPolicy(unittest.TestCase): def test_get_config_policy(self): test_inputs = { - "": [], - "reject *": [ - ExitPolicyRule('reject *:*'), - ], - "reject *:*": [ - ExitPolicyRule('reject *:*'), - ], - "reject private": [ - ExitPolicyRule('reject 0.0.0.0/8:*'), - ExitPolicyRule('reject 169.254.0.0/16:*'), - ExitPolicyRule('reject 127.0.0.0/8:*'), - ExitPolicyRule('reject 192.168.0.0/16:*'), - ExitPolicyRule('reject 10.0.0.0/8:*'), - ExitPolicyRule('reject 172.16.0.0/12:*'), - ], - "accept *:80, reject *": [ - ExitPolicyRule('accept *:80'), - ExitPolicyRule('reject *:*'), - ], - " accept *:80, reject * ": [ - ExitPolicyRule('accept *:80'), - ExitPolicyRule('reject *:*'), - ], + "": ExitPolicy(), + "reject *": ExitPolicy('reject *:*'), + "reject *:*": ExitPolicy('reject *:*'), + "reject private": ExitPolicy( + 'reject 0.0.0.0/8:*', + 'reject 169.254.0.0/16:*', + 'reject 127.0.0.0/8:*', + 'reject 192.168.0.0/16:*', + 'reject 10.0.0.0/8:*', + 'reject 172.16.0.0/12:*', + ), + "accept *:80, reject *": ExitPolicy( + 'accept *:80', + 'reject *:*', + ), + " accept *:80, reject * ": ExitPolicy( + 'accept *:80', + 'reject *:*', + ), } for test_input, expected in test_inputs.items(): [View Less]

1 0

[stem/master] Truncating exit policies after a catch-all
by atagar＠torproject.org 14 Jan '13

14 Jan '13

commit 67c00e3c89b0cebd00ce5f6f19ce194b200eb8f4 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 13 16:19:17 2013 -0800 Truncating exit policies after a catch-all It's pointless for an exit policy to contain rules after an 'accept *:*' or 'reject *:*'. Cropping policies if they go on after that. --- stem/exit_policy.py | 9 ++++++--- test/unit/exit_policy/policy.py | 5 +++++ 2 files changed, 11 insertions(+), 3 deletions(-) diff --git … [View More]

1 0

[stem/master] Function to parse configuration exit policies
by atagar＠torproject.org 14 Jan '13

14 Jan '13

commit 54d2c103940b31195b3c91ac440d7f07cb6b9489 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 13 12:57:02 2013 -0800 Function to parse configuration exit policies Tor exit policies found in the torrc (and 'GETCONF ExitPolicy') differ slightly from the exitpattern definition found in the spec. Adding a get_config_policy() function that converts these into proper exit policies. --- stem/exit_policy.py | 58 ++++++++++++++++++++++++++++++… [View More]+++++++-- test/unit/exit_policy/policy.py | 44 +++++++++++++++++++++++++++++- 2 files changed, 98 insertions(+), 4 deletions(-) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index 923b700..45e0a98 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -56,9 +56,21 @@ import stem.util.enum AddressType = stem.util.enum.Enum(("WILDCARD", "Wildcard"), ("IPv4", "IPv4"), ("IPv6", "IPv6")) -# TODO: The ExitPolicyRule's exitpatterns are used everywhere except the torrc. -# This is fine for now, but we should add a subclass to handle those slight -# differences later if we want to provide the ability to parse torrcs. +# Addresses aliased by the 'private' policy. From the tor man page... +# +# To specify all internal and link-local networks (including 0.0.0.0/8, +# 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, and 172.16.0.0/12), +# you can use the "private" alias instead of an address. + +PRIVATE_ADDRESSES = ( + "0.0.0.0/8", + "169.254.0.0/16", + "127.0.0.0/8", + "192.168.0.0/16", + "10.0.0.0/8", + "172.16.0.0/12", +) + # TODO: The ExitPolicyRule could easily be a mutable class if we did the # following... @@ -75,6 +87,46 @@ AddressType = stem.util.enum.Enum(("WILDCARD", "Wildcard"), ("IPv4", "IPv4"), (" # some use cases where we might want to construct custom policies. Maybe make # it a CustomExitPolicyRule subclass? +def get_config_policy(rules): + """ + Converts an ExitPolicy found in a torrc to a proper exit pattern. This + accounts for... + + * ports being optional + * the 'private' keyword + + :param str,list rules: comma separated rules or list to be converted + + :returns: **list** of :class:`~stem.exit_policy.ExitPolicyRule` + + :raises: **ValueError** if input isn't a valid tor exit policy + """ + + if isinstance(rules, str): + rules = rules.split(',') + + result = [] + + for rule in rules: + rule = rule.strip() + + if not rule: + continue + + if not ':' in rule: + rule = "%s:*" % rule + + if 'private' in rule: + acceptance = rule.split(' ', 1)[0] + port = rule.split(':', 1)[1] + + for private_addr in PRIVATE_ADDRESSES: + result.append(ExitPolicyRule("%s %s:%s" % (acceptance, private_addr, port))) + else: + result.append(ExitPolicyRule(rule)) + + return result + class ExitPolicy(object): """ diff --git a/test/unit/exit_policy/policy.py b/test/unit/exit_policy/policy.py index c22eab0..4da21b0 100644 --- a/test/unit/exit_policy/policy.py +++ b/test/unit/exit_policy/policy.py @@ -4,7 +4,8 @@ Unit tests for the stem.exit_policy.ExitPolicy class. import unittest -from stem.exit_policy import ExitPolicy, \ +from stem.exit_policy import get_config_policy, \ + ExitPolicy, \ MicroExitPolicy, \ ExitPolicyRule @@ -190,3 +191,44 @@ class TestExitPolicy(unittest.TestCase): self.assertFalse(policy.can_exit_to('127.0.0.1', 79)) self.assertTrue(policy.can_exit_to('127.0.0.1', 80)) + + def test_get_config_policy(self): + test_inputs = { + "": [], + "reject *": [ + ExitPolicyRule('reject *:*'), + ], + "reject *:*": [ + ExitPolicyRule('reject *:*'), + ], + "reject private": [ + ExitPolicyRule('reject 0.0.0.0/8:*'), + ExitPolicyRule('reject 169.254.0.0/16:*'), + ExitPolicyRule('reject 127.0.0.0/8:*'), + ExitPolicyRule('reject 192.168.0.0/16:*'), + ExitPolicyRule('reject 10.0.0.0/8:*'), + ExitPolicyRule('reject 172.16.0.0/12:*'), + ], + "accept *:80, reject *": [ + ExitPolicyRule('accept *:80'), + ExitPolicyRule('reject *:*'), + ], + " accept *:80, reject * ": [ + ExitPolicyRule('accept *:80'), + ExitPolicyRule('reject *:*'), + ], + } + + for test_input, expected in test_inputs.items(): + self.assertEqual(expected, get_config_policy(test_input)) + + test_inputs = ( + "blarg", + "accept *:*:*", + "acceptt *:80", + "accept 257.0.0.1:80", + "accept *:999999", + ) + + for test_input in test_inputs: + self.assertRaises(ValueError, get_config_policy, test_input) [View Less]

1 0

[stem/master] Controller method to get the exit policy
by atagar＠torproject.org 14 Jan '13

14 Jan '13

commit 716f8a693e9b814da5e2c9df551dbe6768f4f324 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 13 17:19:33 2013 -0800 Controller method to get the exit policy What started out with 'we should have a method to get our exit policy' turned out to not be as simple as I thought. A relay's exit policy is really the combination of three things: * 'reject private:*, reject [public address]:*' if ExitPolicyRejectPrivate is set. * Our … [View More]ExitPolicy. Tor provides the torrc value rather than a proper policy spec, so I added a get_config_policy() function to account for the differences. * The default exit policy (as per 'GETINFO exit-policy/default'). --- stem/control.py | 51 +++++++++++++++++++++++++++++++++++++- stem/exit_policy.py | 2 + test/integ/control/controller.py | 35 ++++++++++++++++++++++++++ test/unit/control/controller.py | 43 ++++++++++++++++++++++++++++++++ 4 files changed, 130 insertions(+), 1 deletions(-) diff --git a/stem/control.py b/stem/control.py index 4d3bb52..2090983 100644 --- a/stem/control.py +++ b/stem/control.py @@ -16,7 +16,8 @@ providing its own for interacting at a higher level. |- authenticate - authenticates this controller with tor | |- get_info - issues a GETINFO query for a parameter - |- get_version - convenience method to get tor version + |- get_version - provides our tor version + |- get_exit_policy - provides our exit policy |- get_socks_listeners - provides where tor is listening for SOCKS connections |- get_protocolinfo - information about the controller interface | @@ -135,6 +136,7 @@ import time import stem.descriptor.router_status_entry import stem.descriptor.server_descriptor +import stem.exit_policy import stem.response import stem.socket import stem.util.connection @@ -813,6 +815,50 @@ class Controller(BaseController): else: return default + def get_exit_policy(self, default = UNDEFINED): + """ + Effective ExitPolicy for our relay. This accounts for + ExitPolicyRejectPrivate and default policies. + + :param object default: response if the query fails + + :returns: :class:`~stem.exit_policy.ExitPolicy` of the tor instance that + we're connected to + + :raises: + * :class:`stem.ControllerError` if unable to query the policy + * **ValueError** if unable to parse the policy + + An exception is only raised if we weren't provided a default response. + """ + + with self._msg_lock: + try: + if not "exit_policy" in self._request_cache: + policy = [] + + if self.get_conf("ExitPolicyRejectPrivate") == "1": + policy.append("reject private:*") + + public_addr = self.get_info("address", None) + + if public_addr: + policy.append("reject %s:*" % public_addr) + + for policy_line in self.get_conf("ExitPolicy", multiple = True): + policy += policy_line.split(",") + + policy += self.get_info("exit-policy/default").split(",") + + self._request_cache["exit_policy"] = stem.exit_policy.get_config_policy(policy) + + return self._request_cache["exit_policy"] + except Exception, exc: + if default == UNDEFINED: + raise exc + else: + return default + def get_socks_listeners(self, default = UNDEFINED): """ Provides the SOCKS **(address, port)** tuples that tor has open. @@ -1312,6 +1358,9 @@ class Controller(BaseController): self._request_cache[cache_key] = [value] else: self._request_cache[cache_key] = value + + if param.lower() == "exitpolicy" and "exit_policy" in self._request_cache: + del self._request_cache["exit_policy"] else: log.debug("%s (failed, code: %s, message: %s)" % (query, response.code, response.message)) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index 2809e5e..e568deb 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -38,6 +38,8 @@ exiting to a destination is permissible or not. For instance... |- get_masked_bits - provides the bit representation of our mask +- __str__ - string representation for this rule + get_config_policy - provides the ExitPolicy based on torrc rules + .. data:: AddressType (enum) Enumerations for IP address types that can be in an exit policy. diff --git a/test/integ/control/controller.py b/test/integ/control/controller.py index bf6b229..8b75c6f 100644 --- a/test/integ/control/controller.py +++ b/test/integ/control/controller.py @@ -23,6 +23,7 @@ import test.runner import test.util from stem.control import EventType +from stem.exit_policy import ExitPolicy from stem.version import Requirement @@ -247,6 +248,40 @@ class TestController(unittest.TestCase): self.assertTrue(isinstance(version, stem.version.Version)) self.assertEqual(version, runner.get_tor_version()) + def test_get_exit_policy(self): + """ + Sanity test for get_exit_policy(). We have the default policy (no + ExitPolicy set) which is a little... long due to the boilerplate. + """ + + if test.runner.require_control(self): + return + + expected = ExitPolicy( + 'reject 0.0.0.0/8:*', + 'reject 169.254.0.0/16:*', + 'reject 127.0.0.0/8:*', + 'reject 192.168.0.0/16:*', + 'reject 10.0.0.0/8:*', + 'reject 172.16.0.0/12:*', + 'reject *:25', + 'reject *:119', + 'reject *:135-139', + 'reject *:445', + 'reject *:563', + 'reject *:1214', + 'reject *:4661-4666', + 'reject *:6346-6429', + 'reject *:6699', + 'reject *:6881-6999', + 'accept *:*', + ) + + runner = test.runner.get_runner() + + with runner.get_tor_controller() as controller: + self.assertEqual(expected, controller.get_exit_policy()) + def test_authenticate(self): """ Test that the convenient method authenticate() works. diff --git a/test/unit/control/controller.py b/test/unit/control/controller.py index 09b0684..77c8f84 100644 --- a/test/unit/control/controller.py +++ b/test/unit/control/controller.py @@ -11,6 +11,7 @@ import stem.version from stem import InvalidArguments, InvalidRequest, ProtocolError from stem.control import _parse_circ_path, Controller, EventType +from stem.exit_policy import ExitPolicy from test import mocking @@ -69,6 +70,48 @@ class TestControl(unittest.TestCase): # Turn caching back on before we leave. self.controller._is_caching_enabled = True + def test_get_exit_policy(self): + """ + Exercises the get_exit_policy() method. + """ + + mocking.mock_method(Controller, "get_conf", mocking.return_for_args({ + ("ExitPolicyRejectPrivate",): "1", + ("ExitPolicy", "multiple=True"): ["accept *:80, accept *:443", "accept 43.5.5.5,reject *:22"] + }, is_method = True)) + + mocking.mock_method(Controller, "get_info", mocking.return_for_args({ + ("address", None): "123.45.67.89", + ("exit-policy/default",): "reject *:25,reject *:119,reject *:135-139,reject *:445,reject *:563,reject *:1214,reject *:4661-4666,reject *:6346-6429,reject *:6699,reject *:6881-6999,accept *:*" + }, is_method = True)) + + expected = ExitPolicy( + 'reject 0.0.0.0/8:*', # private entries + 'reject 169.254.0.0/16:*', + 'reject 127.0.0.0/8:*', + 'reject 192.168.0.0/16:*', + 'reject 10.0.0.0/8:*', + 'reject 172.16.0.0/12:*', + 'reject 123.45.67.89:*', # relay's public address + 'accept *:80', # finally we get to our ExitPolicy + 'accept *:443', + 'accept 43.5.5.5:*', + 'reject *:22', + 'reject *:25', # default policy + 'reject *:119', + 'reject *:135-139', + 'reject *:445', + 'reject *:563', + 'reject *:1214', + 'reject *:4661-4666', + 'reject *:6346-6429', + 'reject *:6699', + 'reject *:6881-6999', + 'accept *:*', + ) + + self.assertEqual(expected, self.controller.get_exit_policy()) + def test_get_socks_listeners_old(self): """ Exercises the get_socks_listeners() method as though talking to an old tor [View Less]

1 0

[stem/master] Dropping redundant policy rules
by atagar＠torproject.org 14 Jan '13

14 Jan '13

commit f0ae1eaeec229e51ce8ce223dd1b862d769c1a0c Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 13 17:14:55 2013 -0800 Dropping redundant policy rules While working with 'reject private:*' entries I realized that ExitPolicyRejectPrivate makes for really long, ugly policies. In general this is just life - exit policies are more complicated than just the ExitPolicy torrc option. Hoever, in the case of 'reject *:*' we can safely boil things down.… [View More] --- stem/exit_policy.py | 25 +++++++++++++++++++++++++ test/unit/exit_policy/policy.py | 5 +++++ 2 files changed, 30 insertions(+), 0 deletions(-) diff --git a/stem/exit_policy.py b/stem/exit_policy.py index b5d0573..2809e5e 100644 --- a/stem/exit_policy.py +++ b/stem/exit_policy.py @@ -287,16 +287,41 @@ class ExitPolicy(object): def _get_rules(self): if self._rules is None: rules = [] + is_all_accept, is_all_reject = True, True for rule in self._input_rules: if isinstance(rule, str): rule = ExitPolicyRule(rule.strip()) + if rule.is_accept: + is_all_reject = False + else: + is_all_accept = False + rules.append(rule) if rule.is_address_wildcard() and rule.is_port_wildcard(): break # this is a catch-all, no reason to include more + # If we only have one kind of entry *and* end with a wildcard then + # we might as well use the simpler version. For instance... + # + # reject *:80, reject *:443, reject *:* + # + # ... could also be represented as simply... + # + # reject *:* + # + # This mostly comes up with reject-all policies because the + # 'reject private:*' appends an extra seven rules that have no + # effect. + + if rules and (rules[-1].is_address_wildcard() and rules[-1].is_port_wildcard()): + if is_all_accept: + rules = [ExitPolicyRule("accept *:*")] + elif is_all_reject: + rules = [ExitPolicyRule("reject *:*")] + self._rules = rules self._input_rules = None diff --git a/test/unit/exit_policy/policy.py b/test/unit/exit_policy/policy.py index af4ceba..8659b68 100644 --- a/test/unit/exit_policy/policy.py +++ b/test/unit/exit_policy/policy.py @@ -42,6 +42,11 @@ class TestExitPolicy(unittest.TestCase): policy = ExitPolicy(*"accept *:80, accept *:443, reject *:*, accept *:20-50".split(",")) self.assertEquals(expected_policy, policy) + # checks that we compress redundant policies + + policy = ExitPolicy(*"reject *:80, reject *:443, reject *:*".split(",")) + self.assertEquals(ExitPolicy("reject *:*"), policy) + def test_set_default_allowed(self): policy = ExitPolicy('reject *:80', 'accept *:443') [View Less]

1 0

[stem/master] Controller get_exit_policy() method
by atagar＠torproject.org 14 Jan '13

14 Jan '13

commit b03613cc62d7ae19faa1bbf63e95cffb3547eeae Merge: 427fce8 716f8a6 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 13 17:38:12 2013 -0800 Controller get_exit_policy() method stem/control.py | 51 ++++++++++++++++++++- stem/exit_policy.py | 94 +++++++++++++++++++++++++++++++++++-- test/integ/control/controller.py | 35 ++++++++++++++ test/unit/control/controller.py | 43 +++++++++++++++++ test/unit/exit_policy/policy.py | … [View More]

1 0

[translation/tsum_completed] Update translations for tsum_completed
by translation＠torproject.org 14 Jan '13

14 Jan '13

commit 669608885151130558b85252f651170d35d8e6fc Author: Translation commit bot <translation(a)torproject.org> Date: Mon Jan 14 00:15:13 2013 +0000 Update translations for tsum_completed --- es/short-user-manual_es_noimg.xhtml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/es/short-user-manual_es_noimg.xhtml b/es/short-user-manual_es_noimg.xhtml index 1665ae3..290fd0e 100644 --- a/es/short-user-manual_es_noimg.xhtml +++ b/es/short-user-manual_es_noimg.… [View More]

1 0

[translation/tsum] Update translations for tsum
by translation＠torproject.org 14 Jan '13

14 Jan '13

commit 5eeee6b20447a994cc651d54c360ee4e41281620 Author: Translation commit bot <translation(a)torproject.org> Date: Mon Jan 14 00:15:10 2013 +0000 Update translations for tsum --- es/short-user-manual_es_noimg.xhtml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/es/short-user-manual_es_noimg.xhtml b/es/short-user-manual_es_noimg.xhtml index 1665ae3..290fd0e 100644 --- a/es/short-user-manual_es_noimg.xhtml +++ b/es/short-user-manual_es_noimg.xhtml @@ -124,… [View More]

1 0