January 2014 - tor-commits - lists.torproject.org

[torspec/master] Revise proposal 185 to make DirCache flag unconditional
by nickm＠torproject.org 06 Jan '14

06 Jan '14

commit 8ef1e83ed9695603d4a692ba80a185ebe3202154 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Jan 6 12:47:41 2014 -0500 Revise proposal 185 to make DirCache flag unconditional Based on review by Karsten, who says: """I'm not sure why a) relays with a working DirPort shouldn't include "dir-cache 1" in their router descriptor and b) authorities shouldn't assign the "DirCache" flag to relays with a working DirPort that don't have a "dir-cache 1" line in their router descriptor. I understand that neither of the two actions are necessary to make the proposal work. But this could be a chance to get rid of the DirPort concept entirely and only rely on "dir-cache 1" and "DirCache" flags in the future.""" --- proposals/185-dir-without-dirport.txt | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/185-dir-without-dirport.txt b/proposals/185-dir-without-dirport.txt index 62d5460..7302e75 100644 --- a/proposals/185-dir-without-dirport.txt +++ b/proposals/185-dir-without-dirport.txt @@ -30,11 +30,11 @@ Configuration: Advertising cache status: - Nodes which are running as a directory cache but which do not have - a DirPort set, or which do not have a working DirPort, should set - the entry "dir-cache 1" in their router descriptors and give their - directory port as 0 in their router lines. (Nodes that have a - working directory port advertise it as usual, and do not include a + Nodes that are running as a directory cache should set the entry + "dir-cache 1" in their router descriptors. If they do not have a + DirPort set, or do not have a working DirPort, they should give + their directory port as 0 in their router lines. (Nodes that have + a working directory port advertise it as usual, and also include a "dir-cache" line. Nodes that do not serve directory information should set their directory port to 0, and not include any dir-cache line. Implementations should accept and ignore @@ -43,6 +43,6 @@ Advertising cache status: Consensus: Authorities should assign a "DirCache" flag to all nodes running - as a directory cache that do not set a DirPort. + as a directory cache. This does not require a new version of the consensus algorithm.

1 0

[torspec/master] Fix typos and unclear phrasing in proposal 165.
by nickm＠torproject.org 06 Jan '14

06 Jan '14

commit 0db4bd55d3b4002b8f239dd001b3fcc78e819036 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Sun Dec 22 15:57:44 2013 +0100 Fix typos and unclear phrasing in proposal 165. --- proposals/165-simple-robust-voting.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/165-simple-robust-voting.txt b/proposals/165-simple-robust-voting.txt index f813285..9d920bc 100644 --- a/proposals/165-simple-robust-voting.txt +++ b/proposals/165-simple-robust-voting.txt @@ -6,7 +6,7 @@ Status: Open Overview: - This proposal describes any easy-to-implement, easy-to-verify way to + This proposal describes an easy-to-implement, easy-to-verify way to change the set of authorities without creating a "flag day" situation. Motivation: @@ -58,7 +58,7 @@ Proposed protocol design: Authorities exchange votes with every other authority in any of their voting sets. - When it is time to calculate a consensus, an authority votes with + When it is time to calculate a consensus, an authority picks votes from whichever voting set it lists that is listed by the most members of that set. In other words, given two sets S1 and S2 that an authority lists, that authority will prefer to vote with S1 over S2 whenever @@ -124,7 +124,7 @@ Data format changes: Migration issues: - We should keep track somewhere of which Tor client versions + We should keep track somewhere which Tor client versions recognized which authorities. Acknowledgments:

1 0

[torspec/master] Merge remote-tracking branch 'karsten/prop165tweaks'
by nickm＠torproject.org 06 Jan '14

06 Jan '14

commit 26fb675bcabb89e75dd6ca76fec876e35ddb8370 Merge: f0aa17a 0db4bd5 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Jan 6 12:26:38 2014 -0500 Merge remote-tracking branch 'karsten/prop165tweaks' proposals/165-simple-robust-voting.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)

1 0

[torspec/master] Make the rendezvous-cookie passing more explicit.
by nickm＠torproject.org 06 Jan '14

06 Jan '14

commit f0aa17a275d4b5cabf5d0e6531560940b6f205f8 Author: George Kadianakis <desnacked(a)riseup.net> Date: Sat Jan 4 20:19:49 2014 +0200 Make the rendezvous-cookie passing more explicit. It's not clear that REND_TOKEN refers to the rendezvous cookie. --- proposals/224-rend-spec-ng.txt | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/224-rend-spec-ng.txt b/proposals/224-rend-spec-ng.txt index 0f94745..055f171 100644 --- a/proposals/224-rend-spec-ng.txt +++ b/proposals/224-rend-spec-ng.txt @@ -1235,7 +1235,7 @@ Status: Draft The decrypted plaintext must have the form: - REND_TOKEN [20 bytes] + RENDEZVOUS_COOKIE [20 bytes] Any number of times: EXT_FIELD_TYPE [1 byte] EXT_FIELD_LEN [1 byte] @@ -1507,6 +1507,9 @@ Status: Draft HANDSHAKE_INFO [variable; depends on handshake type used.] + where RENDEZVOUS_COOKIE is the cookie suggested by the client + during the introduction (see [PROCESS_INTRO2]). + If the cookie matches the rendezvous cookie set on any not-yet-connected circuit on the rendezvous point, the rendezvous point connects the two circuits, and sends a RENDEZVOUS2 cell to the

1 0

[tor/master] Fix some seccomp2 issues
by nickm＠torproject.org 06 Jan '14

06 Jan '14

commit 682c2252a564be67fd4fa817d535df0ddc1c758a Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Jan 6 04:27:58 2014 -0500 Fix some seccomp2 issues Fix for #10563. This is a compatibility issue with libseccomp-2.1. I guess you could call it a bugfix on 0.2.5.1? --- changes/seccomp2-fixes | 3 +++ src/common/sandbox.c | 10 ++++++++++ 2 files changed, 13 insertions(+) diff --git a/changes/seccomp2-fixes b/changes/seccomp2-fixes new file mode 100644 index 0000000..600feec --- /dev/null +++ b/changes/seccomp2-fixes @@ -0,0 +1,3 @@ + o Minor bugfixes: + - Fix compilation warnings and startup issues when running with + libseccomp-2.1.0. Fixes bug 10563. diff --git a/src/common/sandbox.c b/src/common/sandbox.c index 7ef577d..0b67b18 100644 --- a/src/common/sandbox.c +++ b/src/common/sandbox.c @@ -250,6 +250,7 @@ static int sb_mmap2(scmp_filter_ctx ctx, sandbox_cfg_t *filter) { int rc = 0; + (void)filter; rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(mmap2), 2, SCMP_CMP(2, SCMP_CMP_EQ, PROT_READ), @@ -405,6 +406,14 @@ sb_socket(scmp_filter_ctx ctx, sandbox_cfg_t *filter) rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(socket), 3, SCMP_CMP(0, SCMP_CMP_EQ, PF_INET), + SCMP_CMP(1, SCMP_CMP_EQ, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK), + SCMP_CMP(2, SCMP_CMP_EQ, IPPROTO_TCP)); + if (rc) + return rc; + + + rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(socket), 3, + SCMP_CMP(0, SCMP_CMP_EQ, PF_INET), SCMP_CMP(1, SCMP_CMP_EQ, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK), SCMP_CMP(2, SCMP_CMP_EQ, IPPROTO_IP)); if (rc) @@ -504,6 +513,7 @@ static int sb_fcntl64(scmp_filter_ctx ctx, sandbox_cfg_t *filter) { int rc = 0; + (void) filter; rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(fcntl64), 1, SCMP_CMP(1, SCMP_CMP_EQ, F_GETFL));

1 0

[stem/master] Increase --test scope to be any module substring
by atagar＠torproject.org 06 Jan '14

06 Jan '14

commit b1081d4abb56439a7d4b8dfd254d079d32d9b976 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 5 21:44:19 2014 -0800 Increase --test scope to be any module substring Our --test argument only ran tests matching a given prefix but this was silly for a couple reasons... 1. That's unintuitive. If I give 'util.conf' I expect it to run tests with 'util.conf', not starting with that. There's no advantage to a prefix match. 2. Now that we're displaying less verbose test output we strip off the 'test.unit.' or 'test.integ.' prefix that all the module names have. With this change the user can copy-and-paste what we show them to just run that test. --- run_tests.py | 10 +++++----- test/settings.cfg | 2 +- test/util.py | 16 ++++++++-------- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/run_tests.py b/run_tests.py index 00cf381..5450dea 100755 --- a/run_tests.py +++ b/run_tests.py @@ -45,7 +45,7 @@ ARGS = { 'run_integ': False, 'run_python3': False, 'run_python3_clean': False, - 'test_prefix': None, + 'specific_test': None, 'logging_runlevel': None, 'tor_path': 'tor', 'run_targets': [Target.RUN_OPEN], @@ -153,7 +153,7 @@ def main(): pyflakes_task, pep8_task = None, None - if not stem.prereq.is_python_3() and not args.test_prefix: + if not stem.prereq.is_python_3() and not args.specific_test: if test.util.is_pyflakes_available(): pyflakes_task = PYFLAKES_TASK @@ -212,7 +212,7 @@ def main(): test.output.print_divider("UNIT TESTS", True) error_tracker.set_category("UNIT TEST") - for test_class in test.util.get_unit_tests(args.test_prefix): + for test_class in test.util.get_unit_tests(args.specific_test): run_result = _run_test(args, test_class, output_filters, logging_buffer) skipped_tests += len(getattr(run_result, 'skipped', [])) @@ -248,7 +248,7 @@ def main(): if integ_runner.is_accessible(): owner = integ_runner.get_tor_controller(True) # controller to own our main Tor process - for test_class in test.util.get_integ_tests(args.test_prefix): + for test_class in test.util.get_integ_tests(args.specific_test): run_result = _run_test(args, test_class, output_filters, logging_buffer) skipped_tests += len(getattr(run_result, 'skipped', [])) @@ -379,7 +379,7 @@ def _get_args(argv): args['run_targets'] = run_targets args['attribute_targets'] = attribute_targets elif opt in ("-l", "--test"): - args['test_prefix'] = arg + args['specific_test'] = arg elif opt in ("-l", "--log"): arg = arg.upper() diff --git a/test/settings.cfg b/test/settings.cfg index e78ddb4..fd960a1 100644 --- a/test/settings.cfg +++ b/test/settings.cfg @@ -36,7 +36,7 @@ msg.help | -u, --unit runs unit tests | -i, --integ runs integration tests | -t, --target TARGET comma separated list of extra targets for integ tests -| --test TEST_NAME only run tests modules matching the given name prefix +| --test TEST_NAME only run tests modules containing the given name | --python3 runs tests against a 2to3 conversion of the codebase | -l, --log RUNLEVEL includes logging output with test results, runlevels: | TRACE, DEBUG, INFO, NOTICE, WARN, ERROR diff --git a/test/util.py b/test/util.py index cfa0b95..f4fb9fb 100644 --- a/test/util.py +++ b/test/util.py @@ -89,34 +89,34 @@ Target = stem.util.enum.UppercaseEnum( STEM_BASE = os.path.sep.join(__file__.split(os.path.sep)[:-2]) -def get_unit_tests(prefix = None): +def get_unit_tests(module_substring = None): """ Provides the classes for our unit tests. - :param str prefix: only provide the test if the module starts with this prefix + :param str module_substring: only provide the test if the module includes this substring :returns: an **iterator** for our unit tests """ - return _get_tests(CONFIG["test.unit_tests"].splitlines(), prefix) + return _get_tests(CONFIG["test.unit_tests"].splitlines(), module_substring) -def get_integ_tests(prefix = None): +def get_integ_tests(module_substring = None): """ Provides the classes for our integration tests. - :param str prefix: only provide the test if the module starts with this prefix + :param str module_substring: only provide the test if the module includes this substring :returns: an **iterator** for our integration tests """ - return _get_tests(CONFIG["test.integ_tests"].splitlines(), prefix) + return _get_tests(CONFIG["test.integ_tests"].splitlines(), module_substring) -def _get_tests(modules, prefix): +def _get_tests(modules, module_substring): for import_name in modules: if import_name: - if prefix and not import_name.startswith(prefix): + if module_substring and module_substring not in import_name: continue # Dynamically imports test modules. The __import__() call has a couple

1 0

[stem/master] Don't run static tests with the --test argument
by atagar＠torproject.org 06 Jan '14

06 Jan '14

commit 4cf84706db2416d11bc2ccd82a83a2397384cf0d Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 5 21:39:00 2014 -0800 Don't run static tests with the --test argument When we're running a single test it sucks to wait for static checks. They're now quick, but not that quick. --- run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/run_tests.py b/run_tests.py index 90493da..00cf381 100755 --- a/run_tests.py +++ b/run_tests.py @@ -153,7 +153,7 @@ def main(): pyflakes_task, pep8_task = None, None - if not stem.prereq.is_python_3(): + if not stem.prereq.is_python_3() and not args.test_prefix: if test.util.is_pyflakes_available(): pyflakes_task = PYFLAKES_TASK

1 0

[stem/master] Utility function to provide port usage information
by atagar＠torproject.org 06 Jan '14

06 Jan '14

commit 42bdd8307705ff33b762a60b2ba9d958650f4b59 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 5 21:55:37 2014 -0800 Utility function to provide port usage information Adding a stem.util.connection.port_usage() function that provides a brief description of a port's common use. This is a capability arm has long hand, and nicely complements our ability to query connection information. --- docs/change_log.rst | 1 + stem/util/connection.py | 51 ++++++- stem/util/ports.cfg | 312 ++++++++++++++++++++++++++++++++++++++++++ test/unit/util/connection.py | 11 ++ 4 files changed, 374 insertions(+), 1 deletion(-) diff --git a/docs/change_log.rst b/docs/change_log.rst index 352d4a8..fa8b638 100644 --- a/docs/change_log.rst +++ b/docs/change_log.rst @@ -49,6 +49,7 @@ The following are only available within stem's `git repository * **Utilities** + * Added :func:`stem.util.connection.port_usage` * Added :func:`stem.util.system.files_with_suffix` .. _version_1.1: diff --git a/stem/util/connection.py b/stem/util/connection.py index 893230e..cd4fa70 100644 --- a/stem/util/connection.py +++ b/stem/util/connection.py @@ -8,6 +8,7 @@ Connection and networking based utility functions. get_connections - quieries the connections belonging to a given process get_system_resolvers - provides connection resolution methods that are likely to be available + port_usage - brief description of the common usage for a port is_valid_ipv4_address - checks if a string is a valid IPv4 address is_valid_ipv6_address - checks if a string is a valid IPv6 address @@ -47,7 +48,7 @@ import re import stem.util.proc import stem.util.system -from stem.util import enum, log +from stem.util import conf, enum, log # Connection resolution is risky to log about since it's highly likely to # contain sensitive information. That said, it's also difficult to get right in @@ -79,6 +80,8 @@ FULL_IPv6_MASK = "FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF" CRYPTOVARIABLE_EQUALITY_COMPARISON_NONCE = os.urandom(32) +PORT_USES = None # port number => description + RESOLVER_COMMAND = { Resolver.PROC: '', @@ -269,6 +272,52 @@ def get_system_resolvers(system = None): return resolvers +def port_usage(port): + """ + Provides the common use of a given port. For example, 'HTTP' for port 80 or + 'SSH' for 22. + + .. versionadded:: 1.2.0 + + :param int port: port number to look up + + :returns: **str** with a description for the port, **None** if none is known + """ + + global PORT_USES + + if PORT_USES is None: + config = conf.Config() + config_path = os.path.join(os.path.dirname(__file__), 'ports.cfg') + + try: + config.load(config_path) + port_uses = {} + + for key, value in config.get('port', {}).items(): + if key.isdigit(): + port_uses[int(key)] = value + elif '-' in key: + min_port, max_port = key.split('-', 1) + + for port_entry in range(int(min_port), int(max_port) + 1): + port_uses[port_entry] = value + else: + raise ValueError("'%s' is an invalid key" % key) + + PORT_USES = port_uses + except Exception as exc: + log.warn("BUG: stem failed to load its internal port descriptions from '%s': %s" % (config_path, exc)) + + if not PORT_USES: + return None + + if isinstance(port, str) and port.isdigit(): + port = int(port) + + return PORT_USES.get(port) + + def is_valid_ipv4_address(address): """ Checks if a string is a valid IPv4 address. diff --git a/stem/util/ports.cfg b/stem/util/ports.cfg new file mode 100644 index 0000000..90c5e90 --- /dev/null +++ b/stem/util/ports.cfg @@ -0,0 +1,312 @@ +################################################################################ +# +# Common usages for port . This is based on... +# +# https://secure.wikimedia.org/wikipedia/en/wiki/List_of_TCP_and_UDP_port numbers +# http://isc.sans.edu/services.html +# +################################################################################ + +port 1 => TCPMUX +port 2 => CompressNET +port 3 => CompressNET +port 5 => RJE +port 7 => Echo +port 9 => Discard +port 11 => SYSTAT +port 13 => Daytime +port 15 => netstat +port 17 => QOTD +port 18 => MSP +port 19 => CHARGEN +port 20 => FTP +port 21 => FTP +port 22 => SSH +port 23 => Telnet +port 24 => Priv-mail +port 25 => SMTP +port 34 => RF +port 35 => Printer +port 37 => TIME +port 39 => RLP +port 41 => Graphics +port 42 => WINS +port 43 => WHOIS +port 47 => NI FTP +port 49 => TACACS +port 50 => Remote Mail +port 51 => IMP +port 52 => XNS +port 53 => DNS +port 54 => XNS +port 55 => ISI-GL +port 56 => RAP +port 57 => MTP +port 58 => XNS +port 67 => BOOTP +port 68 => BOOTP +port 69 => TFTP +port 70 => Gopher +port 79 => Finger +port 80 => HTTP +port 81 => HTTP Alternate +port 82 => Torpark +port 83 => MIT ML +port 88 => Kerberos +port 90 => dnsix +port 99 => WIP +port 101 => NIC +port 102 => ISO-TSAP +port 104 => ACR/NEMA +port 105 => CCSO +port 107 => Telnet +port 108 => SNA +port 109 => POP2 +port 110 => POP3 +port 111 => ONC RPC +port 113 => ident +port 115 => SFTP +port 117 => UUCP +port 118 => SQL +port 119 => NNTP +port 123 => NTP +port 135 => DCE +port 137 => NetBIOS +port 138 => NetBIOS +port 139 => NetBIOS +port 143 => IMAP +port 152 => BFTP +port 153 => SGMP +port 156 => SQL +port 158 => DMSP +port 161 => SNMP +port 162 => SNMPTRAP +port 170 => Print-srv +port 177 => XDMCP +port 179 => BGP +port 194 => IRC +port 199 => SMUX +port 201 => AppleTalk +port 209 => QMTP +port 210 => ANSI +port 213 => IPX +port 218 => MPP +port 220 => IMAP +port 256 => 2DEV +port 259 => ESRO +port 264 => BGMP +port 308 => Novastor +port 311 => OSX Admin +port 318 => PKIX TSP +port 319 => PTP +port 320 => PTP +port 323 => IMMP +port 350 => MATIP +port 351 => MATIP +port 366 => ODMR +port 369 => Rpc2port ap +port 370 => codaauth2 +port 371 => ClearCase +port 383 => HP Alarm Mgr +port 384 => ARNS +port 387 => AURP +port 389 => LDAP +port 401 => UPS +port 402 => Altiris +port 427 => SLP +port 443 => HTTPS +port 444 => SNPP +port 445 => SMB +port 464 => Kerberos (kpasswd) +port 465 => SMTP +port 475 => tcpnethaspsrv +port 497 => Retrospect +port 500 => ISAKMP +port 501 => STMF +port 502 => Modbus +port 504 => Citadel +port 510 => FirstClass +port 512 => Rexec +port 513 => rlogin +port 514 => rsh +port 515 => LPD +port 517 => Talk +port 518 => NTalk +port 520 => efs +port 524 => NCP +port 530 => RPC +port 531 => AIM/IRC +port 532 => netnews +port 533 => netwall +port 540 => UUCP +port 542 => commerce +port 543 => Kerberos (klogin) +port 544 => Kerberos (kshell) +port 545 => OSISoft PI +port 546 => DHCPv6 +port 547 => DHCPv6 +port 548 => AFP +port 550 => new-who +port 554 => RTSP +port 556 => RFS +port 560 => rmonitor +port 561 => monitor +port 563 => NNTPS +port 587 => SMTP +port 591 => FileMaker +port 593 => HTTP RPC +port 604 => TUNNEL +port 623 => ASF-RMCP +port 631 => CUPS +port 635 => RLZ DBase +port 636 => LDAPS +port 639 => MSDP +port 641 => Support oft +port 646 => LDP +port 647 => DHCP +port 648 => RRP +port 651 => IEEE-MMS +port 652 => DTCP +port 653 => Support oft +port 654 => MMS/MMP +port 657 => RMC +port 660 => OSX Admin +port 665 => sun-dr +port 666 => Doom +port 674 => ACAP +port 691 => MS Exchange +port 692 => Hyperwave-ISP +port 694 => Linux-HA +port 695 => IEEE-MMS-SSL +port 698 => OLSR +port 699 => Access Network +port 700 => EPP +port 701 => LMP +port 702 => IRIS +port 706 => SILC +port 711 => MPLS +port 712 => TBRPF +port 720 => SMQP +port 749 => Kerberos (admin) +port 750 => rfile +port 751 => pump +port 752 => qrh +port 753 => rrh +port 754 => tell send +port 760 => ns +port 782 => Conserver +port 783 => spamd +port 829 => CMP +port 843 => Flash +port 847 => DHCP +port 860 => iSCSI +port 873 => rsync +port 888 => CDDB +port 901 => SWAT +port 902-904 => VMware +port 911 => NCA +port 953 => DNS RNDC +port 981 => SofaWare Firewall +port 989 => FTPS +port 990 => FTPS +port 991 => NAS +port 992 => Telnet +port 993 => IMAPS +port 994 => IRC +port 995 => POP3S +port 999 => ScimoreDB +port 1001 => JtoMB +port 1002 => cogbot + +port 1080 => SOCKS +port 1085 => WebObjects +port 1109 => KPOP +port 1169 => Tripwire +port 1194 => OpenVPN +port 1214 => Kazaa +port 1220 => QuickTime +port 1234 => VLC +port 1241 => Nessus +port 1270 => SCOM +port 1293 => IPSec +port 1433 => MSSQL +port 1434 => MSSQL +port 1500 => NetGuard +port 1503 => MSN +port 1512 => WINS +port 1521 => Oracle +port 1526 => Oracle +port 1533 => Sametime +port 1666 => Perforce +port 1677 => GroupWise +port 1723 => PPTP +port 1725 => Steam +port 1863 => MSNP +port 2049 => NFS +port 2082 => Infowave +port 2083 => radsec +port 2086 => GNUnet +port 2087 => ELI +port 2095 => NBX SER +port 2096 => NBX DIR +port 2102-2104 => Zephyr +port 2401 => CVS +port 2525 => SMTP +port 2710 => BitTorrent +port 3074 => XBox LIVE +port 3101 => BlackBerry +port 3128 => SQUID +port 3306 => MySQL +port 3389 => WBT +port 3690 => SVN +port 3723 => Battle.net +port 3724 => WoW +port 4321 => RWHOIS +port 4643 => Virtuozzo +port 4662 => eMule +port 5003 => FileMaker +port 5050 => Yahoo IM +port 5060 => SIP +port 5061 => SIP +port 5190 => AIM/ICQ +port 5222 => Jabber +port 5223 => Jabber +port 5228 => Android Market +port 5269 => Jabber +port 5298 => Jabber +port 5432 => PostgreSQL +port 5500 => VNC +port 5556 => Freeciv +port 5666 => NRPE +port 5667 => NSCA +port 5800 => VNC +port 5900 => VNC +port 6346 => gnutella +port 6347 => gnutella +port 6660-6669 => IRC +port 6679 => IRC +port 6697 => IRC +port 6881-6999 => BitTorrent +port 8000 => iRDMI +port 8008 => HTTP Alternate +port 8010 => XMPP +port 8074 => Gadu-Gadu +port 8080 => HTTP Proxy +port 8087 => SPP +port 8088 => Radan HTTP +port 8118 => Privoxy +port 8123 => Polipo +port 8443 => PCsync HTTPS +port 8888 => NewsEDGE +port 9030 => Tor +port 9050 => Tor +port 9051 => Tor +port 9418 => Git +port 9999 => distinct +port 10000 => Webmin +port 19294 => Google Voice +port 19638 => Ensim +port 23399 => Skype +port 30301 => BitTorrent +port 33434 => traceroute + diff --git a/test/unit/util/connection.py b/test/unit/util/connection.py index e9c8980..2e0e4bd 100644 --- a/test/unit/util/connection.py +++ b/test/unit/util/connection.py @@ -121,6 +121,17 @@ class TestConnection(unittest.TestCase): is_available_mock.return_value = False self.assertEqual([Resolver.PROC], stem.util.connection.get_system_resolvers('Linux')) + def test_port_usage(self): + """ + Check that port_usage can load our config and provide the expected results. + """ + + self.assertEqual('HTTP', stem.util.connection.port_usage(80)) + self.assertEqual('HTTP', stem.util.connection.port_usage('80')) # query with a string + self.assertEqual('BitTorrent', stem.util.connection.port_usage(6881)) # min and max value of a range + self.assertEqual('BitTorrent', stem.util.connection.port_usage(6999)) + self.assertEqual(None, stem.util.connection.port_usage(30000)) # unrecognized port + @patch('stem.util.proc.get_connections') def test_get_connections_by_proc(self, proc_mock): """

1 0

[stem/master] New succinct test output reported errors as successes
by atagar＠torproject.org 06 Jan '14

06 Jan '14

commit 6837752b4c5d7f74bef8be0bca8c8fc7f6f3e8f7 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 5 21:50:40 2014 -0800 New succinct test output reported errors as successes We reported failures as failures but didn't interpret errors as failures as well. --- run_tests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/run_tests.py b/run_tests.py index 5450dea..f28ab8e 100755 --- a/run_tests.py +++ b/run_tests.py @@ -449,7 +449,7 @@ def _run_test(args, test_class, output_filters, logging_buffer): if args.verbose: sys.stdout.write(test.output.apply_filters(test_results.getvalue(), *output_filters)) println() - elif not run_result.failures: + elif not run_result.failures and not run_result.errors: println(" success (%0.2fs)" % (time.time() - start_time), SUCCESS) else: println(" failed (%0.2fs)" % (time.time() - start_time), ERROR)

1 0

[arm/master] Testing the ResourceTracker class
by atagar＠torproject.org 06 Jan '14

06 Jan '14

commit 0cede13371c0796a5a819ef65466b9d2aaf0b806 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 5 16:15:20 2014 -0800 Testing the ResourceTracker class Unit tests for our daemon that tracks the resource usage of tor. --- arm/util/tracker.py | 20 ++--- test/util/tracker/resource_tracker.py | 142 +++++++++++++++++++++++++++++++++ 2 files changed, 152 insertions(+), 10 deletions(-) diff --git a/arm/util/tracker.py b/arm/util/tracker.py index 1a716d3..b0f99b2 100644 --- a/arm/util/tracker.py +++ b/arm/util/tracker.py @@ -31,7 +31,7 @@ Background tasks for gathering information about the tor process. :var float cpu_average: average cpu usage since we first started tracking the process :var float cpu_total: total cpu time the process has used since starting :var int memory_bytes: memory usage of the process in bytes - :var float memory_precent: percentage of our memory used by this process + :var float memory_percent: percentage of our memory used by this process :var float timestamp: unix timestamp for when this information was fetched """ @@ -70,7 +70,7 @@ def get_connection_tracker(): global CONNECTION_TRACKER if CONNECTION_TRACKER is None: - CONNECTION_TRACKER = ConnectionTracker() + CONNECTION_TRACKER = ConnectionTracker(CONFIG['queries.connections.rate']) return CONNECTION_TRACKER @@ -83,7 +83,7 @@ def get_resource_tracker(): global RESOURCE_TRACKER if RESOURCE_TRACKER is None: - RESOURCE_TRACKER = ResourceTracker() + RESOURCE_TRACKER = ResourceTracker(CONFIG['queries.resources.rate']) return RESOURCE_TRACKER @@ -147,9 +147,9 @@ def _resources_via_ps(pid): total_cpu_time = str_tools.parse_short_time_label(stats[0]) uptime = str_tools.parse_short_time_label(stats[1]) memory_bytes = int(stats[2]) * 1024 # ps size is in kb - memory_precent = float(stats[3]) / 100.0 + memory_percent = float(stats[3]) / 100.0 - return (total_cpu_time, uptime, memory_bytes, memory_precent) + return (total_cpu_time, uptime, memory_bytes, memory_percent) except ValueError: pass @@ -320,8 +320,8 @@ class ConnectionTracker(Daemon): Periodically retrieves the connections established by tor. """ - def __init__(self): - super(ConnectionTracker, self).__init__(CONFIG['queries.connections.rate']) + def __init__(self, rate): + super(ConnectionTracker, self).__init__(rate) self._connections = [] self._resolvers = connection.get_system_resolvers() @@ -437,8 +437,8 @@ class ResourceTracker(Daemon): Periodically retrieves the resource usage of tor. """ - def __init__(self): - super(ResourceTracker, self).__init__(CONFIG['queries.resources.rate']) + def __init__(self, rate): + super(ResourceTracker, self).__init__(rate) self._resources = None self._use_proc = proc.is_available() # determines if we use proc or ps for lookups @@ -469,7 +469,7 @@ class ResourceTracker(Daemon): cpu_average = total_cpu_time / uptime, cpu_total = total_cpu_time, memory_bytes = memory_in_bytes, - memory_precent = memory_in_percent, + memory_percent = memory_in_percent, timestamp = time.time(), ) diff --git a/test/util/tracker/resource_tracker.py b/test/util/tracker/resource_tracker.py new file mode 100644 index 0000000..70cdf27 --- /dev/null +++ b/test/util/tracker/resource_tracker.py @@ -0,0 +1,142 @@ +import time +import unittest + +from arm.util.tracker import ResourceTracker, _resources_via_ps, _resources_via_proc + +from mock import Mock, patch + +PS_OUTPUT = """\ + TIME ELAPSED RSS %MEM +00:00:02 00:18 18848 0.4 +""" + + +class TestResourceTracker(unittest.TestCase): + @patch('arm.util.tracker.tor_controller') + @patch('arm.util.tracker._resources_via_proc') + @patch('arm.util.tracker.system', Mock(return_value = Mock())) + @patch('arm.util.tracker.proc.is_available', Mock(return_value = True)) + def test_fetching_samplings(self, resources_via_proc_mock, tor_controller_mock): + tor_controller_mock().get_pid.return_value = 12345 + resources_via_proc_mock.return_value = (105.3, 2.4, 8072, 0.3) + + with ResourceTracker(0.04) as daemon: + time.sleep(0.01) + + resources = daemon.get_resource_usage() + + self.assertEqual(1, daemon.run_counter()) + self.assertEqual(0.0, resources.cpu_sample) + self.assertEqual(43.875, resources.cpu_average) + self.assertEqual(105.3, resources.cpu_total) + self.assertEqual(8072, resources.memory_bytes) + self.assertEqual(0.3, resources.memory_percent) + self.assertTrue((time.time() - resources.timestamp) < 0.5) + + resources_via_proc_mock.return_value = (800.3, 3.2, 6020, 0.26) + time.sleep(0.05) + resources = daemon.get_resource_usage() + + self.assertEqual(2, daemon.run_counter()) + self.assertEqual(6.600189933523267, resources.cpu_sample) + self.assertEqual(250.09374999999997, resources.cpu_average) + self.assertEqual(800.3, resources.cpu_total) + self.assertEqual(6020, resources.memory_bytes) + self.assertEqual(0.26, resources.memory_percent) + self.assertTrue((time.time() - resources.timestamp) < 0.5) + + resources_via_proc_mock.assert_called_with(12345) + + @patch('arm.util.tracker.tor_controller') + @patch('arm.util.tracker.proc.is_available') + @patch('arm.util.tracker._resources_via_ps', Mock(return_value = (105.3, 2.4, 8072, 0.3))) + @patch('arm.util.tracker._resources_via_proc', Mock(return_value = (340.3, 3.2, 6020, 0.26))) + @patch('arm.util.tracker.system', Mock(return_value = Mock())) + def test_picking_proc_or_ps(self, is_proc_available_mock, tor_controller_mock): + tor_controller_mock().get_pid.return_value = 12345 + + is_proc_available_mock.return_value = True + + with ResourceTracker(0.04) as daemon: + time.sleep(0.01) + + resources = daemon.get_resource_usage() + + self.assertEqual(1, daemon.run_counter()) + self.assertEqual(0.0, resources.cpu_sample) + self.assertEqual(106.34375, resources.cpu_average) + self.assertEqual(340.3, resources.cpu_total) + self.assertEqual(6020, resources.memory_bytes) + self.assertEqual(0.26, resources.memory_percent) + self.assertTrue((time.time() - resources.timestamp) < 0.5) + + is_proc_available_mock.return_value = False + + with ResourceTracker(0.04) as daemon: + time.sleep(0.01) + + resources = daemon.get_resource_usage() + + self.assertEqual(1, daemon.run_counter()) + self.assertEqual(0.0, resources.cpu_sample) + self.assertEqual(43.875, resources.cpu_average) + self.assertEqual(105.3, resources.cpu_total) + self.assertEqual(8072, resources.memory_bytes) + self.assertEqual(0.3, resources.memory_percent) + self.assertTrue((time.time() - resources.timestamp) < 0.5) + + @patch('arm.util.tracker.tor_controller') + @patch('arm.util.tracker._resources_via_ps', Mock(return_value = (105.3, 2.4, 8072, 0.3))) + @patch('arm.util.tracker._resources_via_proc', Mock(side_effect = IOError())) + @patch('arm.util.tracker.system', Mock(return_value = Mock())) + @patch('arm.util.tracker.proc.is_available', Mock(return_value = True)) + def test_failing_over_to_ps(self, tor_controller_mock): + tor_controller_mock().get_pid.return_value = 12345 + + with ResourceTracker(0.01) as daemon: + time.sleep(0.03) + + self.assertEqual(True, daemon._use_proc) + resources = daemon.get_resource_usage() + + self.assertEqual(0, daemon.run_counter()) + self.assertEqual(0.0, resources.cpu_sample) + self.assertEqual(0.0, resources.cpu_average) + self.assertEqual(0, resources.cpu_total) + self.assertEqual(0, resources.memory_bytes) + self.assertEqual(0.0, resources.memory_percent) + self.assertEqual(0.0, resources.timestamp) + + time.sleep(0.05) + self.assertEqual(False, daemon._use_proc) + + resources = daemon.get_resource_usage() + + self.assertEqual(1, daemon.run_counter()) + self.assertEqual(0.0, resources.cpu_sample) + self.assertEqual(43.875, resources.cpu_average) + self.assertEqual(105.3, resources.cpu_total) + self.assertEqual(8072, resources.memory_bytes) + self.assertEqual(0.3, resources.memory_percent) + self.assertTrue((time.time() - resources.timestamp) < 0.5) + + @patch('arm.util.tracker.system.call', Mock(return_value = PS_OUTPUT.split('\n'))) + def test_resources_via_ps(self): + total_cpu_time, uptime, memory_in_bytes, memory_in_percent = _resources_via_ps(12345) + + self.assertEqual(2.0, total_cpu_time) + self.assertEqual(18, uptime) + self.assertEqual(19300352, memory_in_bytes) + self.assertEqual(0.004, memory_in_percent) + + @patch('time.time', Mock(return_value = 1388967218.973117)) + @patch('arm.util.tracker.proc.get_stats', Mock(return_value = (1.5, 0.5, 1388967200.9))) + @patch('arm.util.tracker.proc.get_memory_usage', Mock(return_value = (19300352, 6432))) + @patch('arm.util.tracker.proc.get_physical_memory', Mock(return_value = 4825088000)) + def test_resources_via_proc(self): + total_cpu_time, uptime, memory_in_bytes, memory_in_percent = _resources_via_proc(12345) + + self.assertEqual(2.0, total_cpu_time) + self.assertEqual(18, int(uptime)) + self.assertEqual(19300352, memory_in_bytes) + self.assertEqual(0.004, memory_in_percent)

1 0