tor-commits February 2015

tor-commits@lists.torproject.org

15 participants
2121 discussions

[stem/master] Strip tabs that slipped into a comment
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit 64785ca5ccdfa032e681f598caf4f4e598c55d67 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 08:46:18 2015 -0800 Strip tabs that slipped into a comment Our pep8 checks evidently don't catch this one. --- stem/util/connection.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/stem/util/connection.py b/stem/util/connection.py index c863d97..7e9d43b 100644 --- a/stem/util/connection.py +++ b/stem/util/connection.py @@ -27,18 +27,18 @@ Connection and networking based utility functions. .. versionadded:: 1.1.0 - ================= =========== - Resolver Description - ================= =========== - **PROC** /proc contents - **NETSTAT** netstat - **NETSTAT_WINDOWS** netstat command under Windows - **SS** ss command - **LSOF** lsof command - **SOCKSTAT** sockstat command under *nix - **BSD_SOCKSTAT** sockstat command under FreeBSD - **BSD_PROCSTAT** procstat command under FreeBSD - ================= =========== + ================= =========== + Resolver Description + ================= =========== + **PROC** /proc contents + **NETSTAT** netstat + **NETSTAT_WINDOWS** netstat command under Windows + **SS** ss command + **LSOF** lsof command + **SOCKSTAT** sockstat command under *nix + **BSD_SOCKSTAT** sockstat command under FreeBSD + **BSD_PROCSTAT** procstat command under FreeBSD + ================= =========== """ import collections

1 0

[stem/master] Give a useful error if our pid lookup fails
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit e53d1bfbfa3e11f11d247610c518bc8fd8c64730 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 08:56:00 2015 -0800 Give a useful error if our pid lookup fails pid_by_name() returns None if the lookup fails, so moving it earlier so we raise an exception when that happens. Also a misspelling. --- stem/util/connection.py | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/stem/util/connection.py b/stem/util/connection.py index e5abaa8..fa2a3d5 100644 --- a/stem/util/connection.py +++ b/stem/util/connection.py @@ -119,7 +119,6 @@ RESOLVER_FILTER = { Resolver.NETSTAT: '^{protocol}\s+.*\s+{local_address}:{local_port}\s+{remote_address}:{remote_port}\s+ESTABLISHED\s+{pid}/{name}\s*$', # tcp 586 192.168.0.1:44284 38.229.79.2:443 ESTABLISHED 15843 - Resolver.NETSTAT_WINDOWS: '^\s*{protocol}\s+{local_address}:{local_port}\s+{remote_address}:{remote_port}\s+ESTABLISHED\s+{pid}\s*$', # tcp ESTAB 0 0 192.168.0.20:44415 38.229.79.2:443 users:(("tor",15843,9)) @@ -180,7 +179,10 @@ def get_connections(resolver, process_pid = None, process_name = None): except ValueError: raise ValueError('Process pid was non-numeric: %s' % process_pid) - if process_pid is None and resolver in (Resolver.PROC, Resolver.BSD_PROCSTAT): + if process_pid is None and process_name and resolver == Resolver.NETSTAT_WINDOWS: + process_pid = stem.util.system.pid_by_name(process_name) + + if process_pid is None and resolver in (Resolver.NETSTAT_WINDOWS, Resolver.PROC, Resolver.BSD_PROCSTAT): raise ValueError('%s resolution requires a pid' % resolver) if resolver == Resolver.PROC: @@ -188,11 +190,6 @@ def get_connections(resolver, process_pid = None, process_name = None): resolver_command = RESOLVER_COMMAND[resolver].format(pid = process_pid) - # In case, process_name is only specified - if resolver == Resolver.NETSTAT_WINDOWS: - if not process_pid and process_name: - process_pid = stem.util.system.pid_by_name(process_name) - try: results = stem.util.system.call(resolver_command) except OSError as exc: @@ -273,7 +270,7 @@ def system_resolvers(system = None): elif system in ('Darwin', 'OpenBSD'): resolvers = [Resolver.LSOF] elif system == 'FreeBSD': - # Netstat is available, but lacks a '-p' equivilant so we can't associate + # Netstat is available, but lacks a '-p' equivalent so we can't associate # the results to processes. The platform also has a ss command, but it # belongs to a spreadsheet application.

1 0

[stem/master] Explain what the flags mean
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit 52d6b36cbf71e9bdc198df532354ae57cad79cad Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 08:50:53 2015 -0800 Explain what the flags mean Explaining what all the netstat args mean as we do for the other commands... http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en… --- stem/util/connection.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stem/util/connection.py b/stem/util/connection.py index 7e9d43b..e5abaa8 100644 --- a/stem/util/connection.py +++ b/stem/util/connection.py @@ -93,7 +93,7 @@ RESOLVER_COMMAND = { # -n = prevents dns lookups, -p = include process Resolver.NETSTAT: 'netstat -np', - # -ano is a Windows variant for netstat including pid + # -a = show all TCP/UDP connections, -n = numeric addresses and ports, -o = include pid Resolver.NETSTAT_WINDOWS: 'netstat -ano', # -n = numeric ports, -p = include process, -t = tcp sockets, -u = udp sockets

1 0

[stem/master] Correcting issues with pid_by_name()
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit 738d4f54fb2559236b810d2ca3ca6b1e0eb5edc1 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 09:34:58 2015 -0800 Correcting issues with pid_by_name() This has quite a few bits of weirdness... * Our comment says call() doesn't work but then uses it. * Our comment says the command might or might not end with '.exe' but then we always check for that. Sounds wrong but opting for what the code does and just dropping the mismatching comment. * We checked for '.exe' anywhere in the string rather than just the end. * We checked to see if the netstat command existed to see if we could run tasklist. * Unlike the other checks we raised an exception here rather than passing through when unable to determine the pid. * Regex handling was a bit more verbose than it needed to be. * We didn't actually use GET_PID_BY_NAME_TASKLIST. --- stem/util/system.py | 31 ++++++++----------------------- test/integ/util/system.py | 6 ++---- 2 files changed, 10 insertions(+), 27 deletions(-) diff --git a/stem/util/system.py b/stem/util/system.py index efe8663..9ee8ad6 100644 --- a/stem/util/system.py +++ b/stem/util/system.py @@ -47,10 +47,10 @@ import ctypes.util import mimetypes import os import platform +import re import subprocess import tarfile import time -import re import stem.util.proc import stem.util.str_tools @@ -79,7 +79,6 @@ GET_PID_BY_NAME_PIDOF = 'pidof %s' GET_PID_BY_NAME_PS_LINUX = 'ps -o pid -C %s' GET_PID_BY_NAME_PS_BSD = 'ps axc' GET_PID_BY_NAME_LSOF = 'lsof -tc %s' -GET_PID_BY_NAME_TASKLIST = 'tasklist | findstr %s' GET_PID_BY_PORT_NETSTAT = 'netstat -npltu' GET_PID_BY_PORT_SOCKSTAT = 'sockstat -4l -P tcp -p %s' GET_PID_BY_PORT_LSOF = 'lsof -wnP -iTCP -sTCP:LISTEN' @@ -438,48 +437,34 @@ def pid_by_name(process_name, multiple = False): except ValueError: pass - # Calling and Parsing tasklist command on Windows (Because call method doesn't work properly with it) - # Process name may or may not include .exe - if is_available('netstat -ano') and is_windows(): - - if process_name.find(".exe") == -1: + if is_available('tasklist') and is_windows(): + if not process_name.endswith('.exe'): process_name = process_name + '.exe' - command = GET_PID_BY_NAME_TASKLIST % process_name process_ids = [] - try: - results = stem.util.system.call('tasklist') - tasklist_regex_str = '^\s*' + process_name + '\s+(?P<pid>[0-9]*)' - tasklist_regex = re.compile(tasklist_regex_str) + results = stem.util.system.call('tasklist', None) - if not results: - raise IOError("No results found for tasklist") + if results: + tasklist_regex = re.compile('^\s*%s\s+(?P<pid>[0-9]*)' % process_name) for line in results: match = tasklist_regex.search(line) + if match: - attr = match.groupdict() - id = int(attr['pid']) - process_ids.append(id) + process_ids.append(int(match.group('pid'))) if multiple: return process_ids elif len(process_ids) > 0: return process_ids[0] - except OSError as exc: - log.debug("failed to query '%s': %s" % (command, exc)) - raise IOError("Unable to query '%s': %s" % (command, exc)) - log.debug("failed to resolve a pid for '%s'" % process_name) return [] if multiple else None def pid_by_port(port): - """ - Attempts to determine the process id for a process with the given port, using... diff --git a/test/integ/util/system.py b/test/integ/util/system.py index aa18f45..e06b569 100644 --- a/test/integ/util/system.py +++ b/test/integ/util/system.py @@ -221,7 +221,6 @@ class TestSystem(unittest.TestCase): Tests the pid_by_name function with a tasklist response. """ - runner = test.runner.get_runner() if self._is_extra_tor_running(): test.runner.skip(self, '(multiple tor instances)') return @@ -229,9 +228,8 @@ class TestSystem(unittest.TestCase): test.runner.skip(self, '(tasklist unavailable)') return - tor_pid = test.runner.get_runner().get_pid() - tor_cmd = test.runner.get_runner().get_tor_command(True) - self.assertEqual(tor_pid, stem.util.system.pid_by_name(tor_cmd)) + runner = test.runner.get_runner() + self.assertEqual(runner.get_pid(), stem.util.system.pid_by_name(runner.get_tor_command(True))) def test_pid_by_port(self): """

1 0

[stem/master] Mocking issues with unit test for tasklist
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit ff8de9372ec6d4fa47949f0b171a0c58b0aa1440 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 09:51:14 2015 -0800 Mocking issues with unit test for tasklist We didn't pretend to be on windows so this was skipped on other platforms, and returned the tasklist results for *any* call() invocation (not just the one for tasklist). Unit tests now pass for me. --- test/unit/util/system.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/test/unit/util/system.py b/test/unit/util/system.py index b733769..f6e9f35 100644 --- a/test/unit/util/system.py +++ b/test/unit/util/system.py @@ -267,12 +267,13 @@ class TestSystem(unittest.TestCase): @patch('stem.util.system.call') @patch('stem.util.system.is_available', Mock(return_value = True)) + @patch('stem.util.system.is_windows', Mock(return_value = True)) def test_pid_by_name_tasklist(self, call_mock): """ Tests the pid_by_name function with tasklist responses. """ - call_mock.return_value = GET_PID_BY_NAME_TASKLIST_RESULTS + call_mock.side_effect = mock_call('tasklist', GET_PID_BY_NAME_TASKLIST_RESULTS) self.assertEqual(3712, system.pid_by_name('tor')) self.assertEqual(None, system.pid_by_name('DirectoryService')) self.assertEqual(None, system.pid_by_name('blarg'))

1 0

[stem/master] Dropped a necessary permission mock
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit 42e251ed24cf5d85c7ea56cc87b101f8952c8122 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 09:46:01 2015 -0800 Dropped a necessary permission mock Not sure why this was dropped. Necessary since otherwise we check proc read permissions. --- test/unit/util/connection.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/test/unit/util/connection.py b/test/unit/util/connection.py index eba0f03..c63f97f 100644 --- a/test/unit/util/connection.py +++ b/test/unit/util/connection.py @@ -117,15 +117,17 @@ BSD_PROCSTAT_OUTPUT = """\ class TestConnection(unittest.TestCase): + @patch('os.access') @patch('stem.util.system.is_available') @patch('stem.util.proc.is_available') - def test_system_resolvers(self, proc_mock, is_available_mock): + def test_system_resolvers(self, proc_mock, is_available_mock, os_mock): """ Checks the system_resolvers function. """ is_available_mock.return_value = True proc_mock.return_value = False + os_mock.return_value = True self.assertEqual([Resolver.NETSTAT_WINDOWS], stem.util.connection.system_resolvers('Windows')) self.assertEqual([Resolver.LSOF], stem.util.connection.system_resolvers('Darwin')) @@ -136,7 +138,7 @@ class TestConnection(unittest.TestCase): proc_mock.return_value = True self.assertEqual([Resolver.PROC, Resolver.NETSTAT, Resolver.SOCKSTAT, Resolver.LSOF, Resolver.SS], stem.util.connection.system_resolvers('Linux')) - # check that calling without an argument is equivilant to calling for this + # check that calling without an argument is equivalent to calling for this # platform self.assertEqual(stem.util.connection.system_resolvers(platform.system()), stem.util.connection.system_resolvers())

1 0

[stem/master] Changelog entry
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit b085b2397c5693156942d0f4df6001684360f705 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 10:04:34 2015 -0800 Changelog entry --- docs/change_log.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/change_log.rst b/docs/change_log.rst index d48be58..945a52f 100644 --- a/docs/change_log.rst +++ b/docs/change_log.rst @@ -59,6 +59,7 @@ conversion (:trac:`14075`). * **Utilities** + * Windows support for connection resolution (:trac:`14844`) * :func:`stem.util.connection.port_usage` always returned None (:trac:`14046`) * Proc connection resolution could fail on especially busy systems (:trac:`14048`)

1 0

[stem/master] Fixing python3 support
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit 7f356311909a12c9ad4ad3c0021daf5e5fa73d33 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 09:59:38 2015 -0800 Fixing python3 support Earlier changes rolled back support for python3. Not sure why. --- stem/util/system.py | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/stem/util/system.py b/stem/util/system.py index 9ee8ad6..2c15d23 100644 --- a/stem/util/system.py +++ b/stem/util/system.py @@ -55,7 +55,7 @@ import time import stem.util.proc import stem.util.str_tools -from stem import UNDEFINED +from stem import UNDEFINED, str_type from stem.util import log # Mapping of commands to if they're available or not. @@ -243,7 +243,7 @@ def is_running(command): command_listing = call(secondary_resolver, None) if command_listing: - command_listing = map(unicode.strip, command_listing) + command_listing = map(str_type.strip, command_listing) return command in command_listing return None @@ -329,7 +329,7 @@ def pid_by_name(process_name, multiple = False): if results: try: - pids = map(int, results) + pids = list(map(int, results)) if multiple: return pids @@ -350,7 +350,7 @@ def pid_by_name(process_name, multiple = False): if results and len(results) == 1: try: - pids = map(int, results[0].split()) + pids = list(map(int, results[0].split())) if multiple: return pids @@ -382,7 +382,7 @@ def pid_by_name(process_name, multiple = False): if results: try: - pids = map(int, results[1:]) + pids = list(map(int, results[1:])) if multiple: return pids @@ -400,7 +400,7 @@ def pid_by_name(process_name, multiple = False): results = [r.split()[0] for r in results if r.endswith(' %s' % process_name)] try: - pids = map(int, results) + pids = list(map(int, results)) if multiple: return pids @@ -428,7 +428,7 @@ def pid_by_name(process_name, multiple = False): if results: try: - pids = map(int, results) + pids = list(map(int, results)) if multiple: return pids @@ -990,7 +990,7 @@ def get_process_name(): args, argc = [], argc_t() - for i in xrange(100): + for i in range(100): # The ending index can be either None or raise a ValueError when # accessed... #

1 0

[stem/master] Windows support for connecion resolution
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit 9944206580b2765b9a6da7484915be3eaf4c3593 Merge: dbb0660 7fe7711 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 10:08:20 2015 -0800 Windows support for connecion resolution Support for Window's netstat and tasklist commands. Patch by icodemachine... https://trac.torproject.org/projects/tor/ticket/14844 docs/change_log.rst | 1 + stem/util/connection.py | 38 ++++++++++++++++++++++++-------------- stem/util/system.py | 24 ++++++++++++++++++++++++ test/integ/util/connection.py | 4 ++++ test/integ/util/system.py | 21 ++++++++++++++++----- test/unit/util/connection.py | 29 ++++++++++++++++++++++++++++- test/unit/util/system.py | 31 +++++++++++++++++++++++++++++-- 7 files changed, 126 insertions(+), 22 deletions(-)

1 0

[stem/master] Dropping a symlink that slipped in
by atagar＠torproject.org 17 Feb '15

17 Feb '15

commit 7fe77116d5031d77de5305352ef970e8827f997c Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Feb 17 10:06:02 2015 -0800 Dropping a symlink that slipped in Windows symlink that slipped in somehow. --- test/integ/test - Shortcut.lnk | Bin 1155 -> 0 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/test/integ/test - Shortcut.lnk b/test/integ/test - Shortcut.lnk deleted file mode 100644 index f8568cd..0000000 Binary files a/test/integ/test - Shortcut.lnk and /dev/null differ

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits February 2015