April 2015 - tor-commits - lists.torproject.org

[tor-browser-spec/master] Add update security info.
by mikeperry＠torproject.org 30 Apr '15

30 Apr '15

commit 351f4868291f16da605191c6f0597b632277d841 Author: Mike Perry <mikeperry-git(a)torproject.org> Date: Wed Apr 29 20:55:25 2015 -0700 Add update security info. --- design-doc/design.xml | 55 +++++++++++++++++++++++++------------------------ 1 file changed, 28 insertions(+), 27 deletions(-) diff --git a/design-doc/design.xml b/design-doc/design.xml index 5c16ce8..90f8032 100644 --- a/design-doc/design.xml +++ b/design-doc/design.xml @@ -221,19 +221,6 @@ ephemeral-keyed encrypted swap. </para></listitem> - - </orderedlist> </sect2> @@ -1121,13 +1108,6 @@ $HOME environment variable to be the TBB extraction directory. </para> </sect2> - <sect2 id="identifier-linkability"> <title>Cross-Origin Identifier Unlinkability</title> <para> @@ -2367,7 +2347,6 @@ of its update pings. <sect1 id="BuildSecurity"> <title>Build Security and Package Integrity</title> <para> - In the age of state-sponsored malware, <ulink url="https://blog.torproject.org/blog/deterministic-builds-part-one-cyberwar-and…">we @@ -2532,7 +2511,6 @@ time-based dependency tracking</ulink> that only appear in LXC containers. </sect2> <sect2> - <title>Package Signatures and Verification</title> <para> @@ -2565,11 +2543,11 @@ consensus, and encoding the package hashes in the Bitcoin blockchain. </para> <para> -At the time of this writing, we do not yet support native code signing for Mac -OS or Windows. Because these signatures are embedded in the actual packages, -and by their nature are based on non-public key material, providing native -code-signed packages while still preserving ease of reproducibility -verification has not yet been achieved. +The Windows releases are also signed by a hardware token provided by Digicert. +In order to verify package integrity, the signature must be sripped off using +the osslsigncode tool, as described on the <ulink +url="https://www.torproject.org/docs/verifying-signatures.html.en#BuildVerificat…">Signature +Vericication</ulink> page. </para> </sect2> @@ -2598,6 +2576,29 @@ verifier. </para> </sect2> + <sect2 id="update-safety"> + <title>Update Safety</title> + <para> + +We make use of the Firefox updater in order to provide automatic updates to +users. We make use of certificate pinning to ensure that update checks +be tampered with, and we sign the individual MAR update files with an offline +signing key. + + </para> + <para> + +The Firefox updater also has code to ensure that it can reliably access the +update server to prevent availability attacks, and complains to the user of 48 +hours go by without a successful response from the server. Additionally, we +use Tor's SOCKS username and password isolation to ensure that every new +request to the updater traverses a separate circuit, to avoid holdback attacks +by exit nodes. + + </para> + </sect2> + + </sect1> <!-- <sect2 id="components">

1 0

[tor-browser-spec/master] Update and clarify security slider section.
by mikeperry＠torproject.org 30 Apr '15

30 Apr '15

commit 4aa6d6fd6de7f17be33813848893a4135a9f6ff1 Author: Mike Perry <mikeperry-git(a)torproject.org> Date: Wed Apr 29 22:00:34 2015 -0700 Update and clarify security slider section. --- design-doc/design.xml | 81 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 53 insertions(+), 28 deletions(-) diff --git a/design-doc/design.xml b/design-doc/design.xml index 90f8032..01559a6 100644 --- a/design-doc/design.xml +++ b/design-doc/design.xml @@ -2163,10 +2163,11 @@ privacy and security issues. <para> In order to provide vulnerability surface reduction for users that need high -security, we have implemented a "Security Slider" that essentially represents a -tradeoff between usability and security. Using metrics collected from -Mozilla's bug tracker, we analyzed the vulnerability counts of core components, -and used <ulink +security, we have implemented a "Security Slider" to allow users to make a +tradeoff between usability and security while minimizing the total number of +choices (to reduce fingerprinting). Using metrics collected from +Mozilla's bug tracker, we analyzed the vulnerability counts of core +components, and used <ulink url="https://github.com/iSECPartners/publications/tree/master/reports/Tor%20Brow…">information gathered from a study performed by iSec Partners</ulink> to inform which features should be disabled at which security levels. @@ -2174,32 +2175,56 @@ features should be disabled at which security levels. </para> <para> - - -The Security Slider consists of four positions. At the lowest security level -(the default), we disable -<command>gfx.font_rendering.graphite.enabled</command> for Latin locales, as -well as <command>gfx.font_rendering.graphite.enabled</command>. At the -medium-low level, we disable most Javascript JIT and related optimizations -(<command>javascript.options.ion.content</command>, -<command>javascript.options.typeinference</command>, -<command>javascript.options.asmjs</command>). We also make HTML5 media -click-to-play (<command>noscript.forbidMedia</command>), and disable WebAudio -(<command>media.webaudio.enabled</command>). At the medium-high level, we -disable the baseline JIT -(<command>javascript.options.baselinejit.content</command>), disable -Javascript entirely all elements that are loaded when the URL bar is not -HTTPS (<command>noscript.globalHttpsWhitelist</command>), and fully disable -graphite font rendering for all locales -(<command>gfx.font_rendering.graphite.enable</command>). At the highest level, -Javascript is fully disabled (<command>noscript.global</command>), as well as -all non-WebM HTML5 codecs (<command>media.ogg.enabled</command>, -<command>media.opus.enabled</command>, <command>media.opus.enabled</command>, -<command>media.DirectShow.enabled</command>, -<command>media.wave.enabled</command>, and -<command>media.apple.mp3.enabled</command>). +The Security Slider consists of four positions: </para> + <itemizedlist> + <listitem><command>Low</command> + <para> + +At this security level, the preferences are the Tor Browser defaults. + + </para> + </listitem> + <listitem><command>Medium-Low</command> + <para> + +At this security level, we disable the ION JIT +(<command>javascript.options.ion.content</command>), TypeInference JIT +(<command>javascript.options.typeinference</command>), ASM.JS +(<command>javascript.options.asmjs</command>), WebAudio +(<command>media.webaudio.enabled</command>), MathML +(<command>mathml.disabled</command>), block remote JAR files +(<command>network.jar.block-remote-files</command>), and make HTML5 audio and +video click-to-play via NoScript (<command>noscript.forbidMedia</command>). + + </para> + </listitem> + <listitem><command>Medium-High</command> + <para> + +This security level inherits the preferences from the Medium-Low level, and +additionally disables the baseline JIT +(<command>javascript.options.baselinejit.content</command>), disables graphite +font rendering (<command>gfx.font_rendering.graphite.enabled</command>), and +only allows Javascript to run if it is loaded over HTTPS and the URL bar is +HTTPS (by setting <command>noscript.global</command> to false and +<command>noscript.globalHttpsWhitelist</command> to true). + + </para> + </listitem> + <listitem><command>High</command> + <para> + +This security level inherits the preferences from the Medium-Low and +Medium-High levels, and additionally disables remote fonts +(<command>noscript.forbidFonts</command>), completely disables Javascript (by +unsetting <command>noscript.globalHttpsWhitelist</command>), and disables SVG +images (<command>svg.in-content.enabled</command>). + + </para> + </listitem> + </itemizedlist> </listitem> <listitem id="traffic-fingerprinting-defenses"><command>Website Traffic Fingerprinting Defenses</command> <para>

1 0

[tor-browser-spec/master] Speel chek desine dock.
by mikeperry＠torproject.org 30 Apr '15

30 Apr '15

commit d7662cd672c7af14eee7609a45cc0da36c888e66 Author: Mike Perry <mikeperry-git(a)torproject.org> Date: Wed Apr 29 22:18:48 2015 -0700 Speel chek desine dock. --- design-doc/design.xml | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/design-doc/design.xml b/design-doc/design.xml index 01559a6..7711d19 100644 --- a/design-doc/design.xml +++ b/design-doc/design.xml @@ -936,7 +936,7 @@ geolocation queries, searchbox queries, XPCOM addon HTTPS/HTTP activity, WebSockets, and live bookmark updates. We have also verified that IPv6 connections are not attempted, through the proxy or otherwise (Tor does not yet support IPv6). We have also verified that external protocol helpers, such -as smb urls and other custom protocol handlers are all blocked. +as SMB URLs and other custom protocol handlers are all blocked. </para> </listitem> @@ -1321,7 +1321,7 @@ interest to an adversary. </para> <para> -URIs created with URL.createObjectURI MUST be limited in scope to the first +URIs created with URL.createObjectURL MUST be limited in scope to the first party URL bar domain that created them. We provide this isolation in Tor Browser via a <ulink url="https://gitweb.torproject.org/tor-browser.git/commit/?h=tor-browser-31.6.0e…">direct @@ -1445,7 +1445,7 @@ fingerprinting makes use of any information that can be extracted from the browser by some specific website action, usually involving Javascript. Some definitions of browser fingerprinting also include supercookies and cookie-like identifier storage, but we deal with those issues separately in -the <link linkend="identifier-linkability">preceeding section on identifier +the <link linkend="identifier-linkability">preceding section on identifier linkability</link>. </para> @@ -1503,14 +1503,14 @@ concurrent activity, we are less concerned with this mechanism of extracting this information. We also expect that reducing the resolution of Javascript's time sources will significantly increase the duration of execution required to extract accurate results, and thus make statistical approaches both -unattractive and highly noticable due to execessive resource consumption. +unattractive and highly noticeable due to excessive resource consumption. </para> </listitem> <listitem><command>Operating System Vendor and Version Differences</command> <para> -Operating system vendor and version differences permiate many different +Operating system vendor and version differences permeate many different aspects of the browser. While it is possible to address these issues with some effort, the relative lack of diversity in operating systems causes us to primarily focus our efforts on passive operating system fingerprinting @@ -1529,7 +1529,7 @@ Due to vast differences in feature set and implementation behavior even between different versions of the same browser, browser vendor and version differences are simply not possible to conceal in any realistic way. It is only possible to minimize the differences among different installations of -the same browser vendor and version. We make no effort to mimick any other +the same browser vendor and version. We make no effort to mimic any other major browser vendor, and in fact most of our fingerprinting defenses serve to differentiate Tor Browser users from normal Firefox users. Because of this, any study that lumps browser vendor and version differences in to its analysis @@ -2121,7 +2121,7 @@ closed (this does not spawn a new Firefox process, only a new window). Upon the close of the final window, an unload handler is fired to invoke the <ulink url="https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Inter…">garbage collector</ulink>, which has the effect of immediately purging any blob:UUID -urls that were created by website content via <ulink +URLs that were created by website content via <ulink url="https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL">URL.createObjectURL</ulink>. </para> @@ -2569,10 +2569,10 @@ consensus, and encoding the package hashes in the Bitcoin blockchain. <para> The Windows releases are also signed by a hardware token provided by Digicert. -In order to verify package integrity, the signature must be sripped off using +In order to verify package integrity, the signature must be stripped off using the osslsigncode tool, as described on the <ulink url="https://www.torproject.org/docs/verifying-signatures.html.en#BuildVerificat…">Signature -Vericication</ulink> page. +Verification</ulink> page. </para> </sect2>

1 0

[translation/tor-launcher-network-settings_completed] Update translations for tor-launcher-network-settings_completed
by translation＠torproject.org 30 Apr '15

30 Apr '15

commit 5c32e84f8e6da94b001d52d85e68a902d176a76d Author: Translation commit bot <translation(a)torproject.org> Date: Wed Apr 29 22:15:38 2015 +0000 Update translations for tor-launcher-network-settings_completed --- fr/network-settings.dtd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fr/network-settings.dtd b/fr/network-settings.dtd index 877dc1a..0ca1938 100644 --- a/fr/network-settings.dtd +++ b/fr/network-settings.dtd @@ -8,7 +8,7 @@ <!ENTITY torSettings.no "Non"> <!ENTITY torSettings.firstQuestion "Laquelle des phrases suivantes décrit le mieux votre situation ?"> -<!ENTITY torSettings.configurePrompt1 "Le connexion internet de cet ordinateur est bloqué ou utilise un proxy."> +<!ENTITY torSettings.configurePrompt1 "Le connexion internet de cet ordinateur est restreinte (censurée) ou passe par un proxy."> <!ENTITY torSettings.configurePrompt2 "Je dois configurer le pont réseau ou les paramètres du proxy."> <!ENTITY torSettings.configure "Configurer"> <!ENTITY torSettings.connectPrompt2 "Je souhaite me connecter directement au réseau Tor.">

1 0

[translation/tor-launcher-network-settings] Update translations for tor-launcher-network-settings
by translation＠torproject.org 30 Apr '15

30 Apr '15

commit 6a10fd878b3295c93e2462a21d63d88db9d7887e Author: Translation commit bot <translation(a)torproject.org> Date: Wed Apr 29 22:15:32 2015 +0000 Update translations for tor-launcher-network-settings --- fr/network-settings.dtd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fr/network-settings.dtd b/fr/network-settings.dtd index 877dc1a..0ca1938 100644 --- a/fr/network-settings.dtd +++ b/fr/network-settings.dtd @@ -8,7 +8,7 @@ <!ENTITY torSettings.no "Non"> <!ENTITY torSettings.firstQuestion "Laquelle des phrases suivantes décrit le mieux votre situation ?"> -<!ENTITY torSettings.configurePrompt1 "Le connexion internet de cet ordinateur est bloqué ou utilise un proxy."> +<!ENTITY torSettings.configurePrompt1 "Le connexion internet de cet ordinateur est restreinte (censurée) ou passe par un proxy."> <!ENTITY torSettings.configurePrompt2 "Je dois configurer le pont réseau ou les paramètres du proxy."> <!ENTITY torSettings.configure "Configurer"> <!ENTITY torSettings.connectPrompt2 "Je souhaite me connecter directement au réseau Tor.">

1 0

[bridgedb/develop] Refactor b.p.descriptors.deduplicate() to improve benchmark results.
by isis＠torproject.org 30 Apr '15

30 Apr '15

commit 1befc9a07f30490decb01c9ab873d37ea77e1fae Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Apr 18 00:04:34 2015 +0000 Refactor b.p.descriptors.deduplicate() to improve benchmark results. --- lib/bridgedb/parse/descriptors.py | 104 +++++++++++++++++++------------------ 1 file changed, 53 insertions(+), 51 deletions(-) diff --git a/lib/bridgedb/parse/descriptors.py b/lib/bridgedb/parse/descriptors.py index efdedc9..bec10b3 100644 --- a/lib/bridgedb/parse/descriptors.py +++ b/lib/bridgedb/parse/descriptors.py @@ -144,7 +144,29 @@ def parseServerDescriptorsFile(filename, validate=True): routers = list(document) return routers -def deduplicate(descriptors): +def __cmp_published__(x, y): + """A custom ``cmp()`` which sorts descriptors by published date. + + :rtype: int + :returns: Return negative if x<y, zero if x==y, positive if x>y. + """ + if x.published < y.published: + return -1 + elif x.published == y.published: + # This *shouldn't* happen. It would mean that two descriptors for + # the same router had the same timestamps, probably meaning there + # is a severely-messed up OR implementation out there. Let's log + # its fingerprint (no matter what!) so that we can look up its + # ``platform`` line in its server-descriptor and tell whoever + # wrote that code that they're probably (D)DOSing the Tor network. + logging.warn(("Duplicate descriptor with identical timestamp (%s) " + "for bridge %s with fingerprint %s !") % + (x.published, x.nickname, x.fingerprint)) + return 0 + elif x.published > y.published: + return 1 + +def deduplicate(descriptors, statistics=False): """Deduplicate some descriptors, returning only the newest for each router. .. note:: If two descriptors for the same router are discovered, AND both @@ -155,68 +177,48 @@ def deduplicate(descriptors): :param list descriptors: A list of :api:`stem.descriptor.server_descriptor.RelayDescriptor`s, :api:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor`s, - :api:`stem.descriptor.router_status_entry.RouterStatusEntryV2`s. + or :api:`stem.descriptor.router_status_entry.RouterStatusEntryV2`s. + :param bool statistics: If ``True``, log some extra statistics about the + number of duplicates. + :rtype: dict + :returns: A dictionary mapping router fingerprints to their newest + available descriptor. """ duplicates = {} - nonDuplicates = {} + newest = {} for descriptor in descriptors: fingerprint = descriptor.fingerprint - logging.debug("Deduplicating %s descriptor for router %s" % (str(descriptor.__class__).rsplit('.', 1)[1], safelog.logSafely(fingerprint))) - - if fingerprint in nonDuplicates.keys(): - # We already found a descriptor for this fingerprint: - conflict = nonDuplicates[fingerprint] - - # If the descriptor we are currently parsing is newer than the - # last one we found: - if descriptor.published > conflict.published: - # And if this is the first duplicate we've found for this - # router, then create a list in the ``duplicates`` dictionary - # for the router: - if not fingerprint in duplicates.keys(): - duplicates[fingerprint] = list() - # Add this to the list of duplicates for this router: - duplicates[fingerprint].append(conflict) - # Finally, put the newest descriptor in the ``nonDuplicates`` - # dictionary: - nonDuplicates[fingerprint] = descriptor - # Same thing, but this time the one we're parsing is older: - elif descriptor.published < conflict.published: - if not fingerprint in duplicates.keys(): - duplicates[fingerprint] = list() - duplicates[fingerprint].append(descriptor) - # This *shouldn't* happen. It would mean that two descriptors for - # the same router had the same timestamps, probably meaning there - # is a severely-messed up OR implementation out there. Let's log - # its fingerprint (no matter what!) so that we can look up its - # ``platform`` line in its server-descriptor and tell whoever - # wrote that code that they're probably (D)DOSing the Tor network. - else: - try: - raise DescriptorWarning( - ("Duplicate descriptor with identical timestamp (%s) " - "for router with fingerprint '%s'!") - % (descriptor.published, fingerprint)) - # And just in case it does happen, catch the warning: - except DescriptorWarning as descwarn: - logging.warn("DescriptorWarning: %s" % str(descwarn)) - - # Hoorah! No duplicates! (yet...) + if fingerprint in duplicates: + duplicates[fingerprint].append(descriptor) else: - nonDuplicates[fingerprint] = descriptor + duplicates[fingerprint] = [descriptor,] + + for fingerprint, dupes in duplicates.items(): + dupes.sort(cmp=__cmp_published__) + first = dupes.pop() + newest[fingerprint] = first + duplicates[fingerprint] = dupes + + if statistics: + # sorted() won't sort by values (or anything that isn't the first item + # in its container), period, no matter what the cmp function is. + totals = sorted([(len(v), k,) for k, v in duplicates.viewitems()]) + total = sum([k for (k, v) in totals]) + bridges = len(duplicates) + top = 10 if bridges >= 10 else bridges + logging.info("Number of bridges with duplicates: %5d" % bridges) + logging.info("Total duplicate descriptors: %5d" % total) + logging.info("Bridges with the most duplicates (Top %d):" % top) + for i, (subtotal, bridge) in zip(range(1, top + 1), totals[:top]): + logging.info(" #%d %s: %d duplicates" % (i, bridge, subtotal)) logging.info("Descriptor deduplication finished.") - logging.info("Number of duplicates: %d" % len(duplicates)) - for (fingerprint, dittos) in duplicates.items(): - logging.info(" For %s: %d duplicates" - % (safelog.logSafely(fingerprint), len(dittos))) - logging.info("Number of non-duplicates: %d" % len(nonDuplicates)) - return nonDuplicates + return newest def parseExtraInfoFiles(*filenames, **kwargs): """Parse files which contain ``@type bridge-extrainfo-descriptor``s.

1 0

[bridgedb/develop] Add benchmark tests for b.p.descriptors.deduplicate().
by isis＠torproject.org 30 Apr '15

30 Apr '15

commit c9ad39af644d5d4ea7165203b4719ee25f187b29 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Apr 18 00:03:23 2015 +0000 Add benchmark tests for b.p.descriptors.deduplicate(). --- lib/bridgedb/test/test_parse_descriptors.py | 77 +++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) diff --git a/lib/bridgedb/test/test_parse_descriptors.py b/lib/bridgedb/test/test_parse_descriptors.py index aa3b662..5ef3f6a 100644 --- a/lib/bridgedb/test/test_parse_descriptors.py +++ b/lib/bridgedb/test/test_parse_descriptors.py @@ -13,11 +13,13 @@ from __future__ import print_function import datetime import glob +import hashlib import io import os import textwrap from twisted.trial import unittest +from twisted.trial.unittest import SkipTest HAS_STEM = False @@ -31,6 +33,8 @@ except (ImportError, NameError), error: else: HAS_STEM = True +from bridgedb.test.util import Benchmarker + BRIDGE_NETWORKSTATUS_0 = '''\ r MiserLandfalls 4IsyTSCtChPhFPAnq5rD8yymlqA /GMC4lz8RXT/62v6kZNdmzSmopk 2014-11-04 06:23:22 2.215.61.223 4056 0 @@ -441,6 +445,79 @@ class ParseDescriptorsTests(unittest.TestCase): datetime.datetime.strptime("2014-12-04 03:10:25", "%Y-%m-%d %H:%M:%S"), "We should have the newest available descriptor for this router.") + def createDuplicatesForBenchmark(self, b=1, n=1200): + """Create a bunch of duplicate extrainfos for benchmark tests. + + :param int b: The number of fake "bridges" to create **n** duplicate + descriptors for. + :param int n: The number of duplicate descriptors for each bridge + **b**. + """ + descFiles = [] + + # The timestamp and fingerprint from BRIDGE_EXTRA_INFO_DESCRIPTOR: + timestamp = "2014-11-04 06:23:22" + Y, M, rest = timestamp.split("-") + fpr = "E08B324D20AD0A13E114F027AB9AC3F32CA696A0" + newerFpr = "E08B324D20AD0A13E114F027AB9AC3F32CA696A0" + + total = 0 + needed = b * n + for x in range(b): + if total >= needed: + break + # Re-digest the fingerprint to create a "new" bridge + newerFpr = hashlib.sha1(newerFpr).hexdigest().upper() + # Generate n extrainfos with different timestamps: + count = 0 + for year in range(1, ((n + 1)/ 12) + 2): # Start from the next year + if count >= n: + break + for month in range(1, 13): + if count < n: + newerTimestamp = "-".join([str(int(Y) + year), "%02d" % month, rest]) + newerDuplicate = BRIDGE_EXTRA_INFO_DESCRIPTOR[:].replace( + fpr, newerFpr).replace( + timestamp, newerTimestamp) + descFiles.append(io.BytesIO(newerDuplicate)) + count += 1 + total += 1 + else: + break + + print("Deduplicating %5d total descriptors (%4d per bridge; %3d bridges):" + % (len(descFiles), n, b), end='\t') + return descFiles + + def test_parse_descriptors_parseExtraInfoFiles_benchmark_100_bridges(self): + """Benchmark test for ``b.p.descriptors.parseExtraInfoFiles``.""" + print() + for i in range(1, 11): + descFiles = self.createDuplicatesForBenchmark(b=100, n=i) + with Benchmarker(): + routers = descriptors.parseExtraInfoFiles(*descFiles) + + def test_parse_descriptors_parseExtraInfoFiles_benchmark_1000_bridges(self): + """Benchmark test for ``b.p.descriptors.parseExtraInfoFiles``.""" + print() + for i in range(1, 11): + descFiles = self.createDuplicatesForBenchmark(b=1000, n=i) + with Benchmarker(): + routers = descriptors.parseExtraInfoFiles(*descFiles) + + def test_parse_descriptors_parseExtraInfoFiles_benchmark_10000_bridges(self): + """Benchmark test for ``b.p.descriptors.parseExtraInfoFiles``. + The algorithm should grow linearly in the number of duplicates. + """ + raise SkipTest(("This test takes ~7 minutes to complete. " + "Run it on your own free time.")) + + print() + for i in range(1, 11): + descFiles = self.createDuplicatesForBenchmark(b=10000, n=i) + with Benchmarker(): + routers = descriptors.parseExtraInfoFiles(*descFiles) + def test_parse_descriptors_parseExtraInfoFiles_no_validate(self): """Test for ``b.p.descriptors.parseExtraInfoFiles`` with descriptor validation disabled.

1 0

[bridgedb/develop] Add a test utility for benchmarking code.
by isis＠torproject.org 30 Apr '15

30 Apr '15

commit d49112515ce338334b75a795ab6aaf08daa5b489 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 17 23:49:13 2015 +0000 Add a test utility for benchmarking code. --- lib/bridgedb/test/util.py | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/lib/bridgedb/test/util.py b/lib/bridgedb/test/util.py index 234a9b6..9e401a5 100644 --- a/lib/bridgedb/test/util.py +++ b/lib/bridgedb/test/util.py @@ -18,6 +18,7 @@ import errno import ipaddr import os import random +import time from functools import wraps @@ -146,3 +147,33 @@ def randomIPString(): #: being run as a ``TestCase`` by ``twisted.trial``. TestCaseMixin = bdbutil.mixin TestCaseMixin.register(unittest.TestCase) + + +class Benchmarker(object): + """Wrap a context with a timer to benchmark execution time. + + .. hint:: Use like so:: + + with Benchmarker(): + foo(bar, baz) + + Once the ``with`` context exits, something like:: + + Benchmark: 180.269957ms (0s) + + will be printed to stdout (if **verbose** is set to ``True``). + """ + + def __init__(self, verbose=True): + self.verbose = verbose + + def __enter__(self): + self.start = time.time() + return self + + def __exit__(self, *args): + self.end = time.time() + self.seconds = self.end - self.start + self.milliseconds = self.seconds * 1000 + if self.verbose: + print("Benchmark: %12fms %12fs" % (self.milliseconds, self.seconds))

1 0

[bridgedb/develop] Fix whitespace identation for Sphinx docstrings in bridgedb.Bridges.
by isis＠torproject.org 30 Apr '15

30 Apr '15

commit 77782beea451270bdf0401c42798be7fbc2002dc Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 19 03:19:33 2015 +0000 Fix whitespace identation for Sphinx docstrings in bridgedb.Bridges. --- lib/bridgedb/Bridges.py | 73 +++++++++++++++++++++-------------------------- 1 file changed, 33 insertions(+), 40 deletions(-) diff --git a/lib/bridgedb/Bridges.py b/lib/bridgedb/Bridges.py index 0239730..191dc08 100644 --- a/lib/bridgedb/Bridges.py +++ b/lib/bridgedb/Bridges.py @@ -109,54 +109,49 @@ class BridgeRing(BridgeHolder): :type key: bytes :param key: The HMAC key, generated with - :func:`bridgedb.crypto.getKey`. + :func:`~bridgedb.crypto.getKey`. :type answerParameters: :class:`BridgeRingParameters` :param answerParameters: DOCDOC :ivar dict bridges: A dictionary which maps HMAC keys to - :class:`~bridgedb.Bridges.Bridge`s. + :class:`~bridgedb.bridges.Bridge`s. :ivar dict bridgesByID: A dictionary which maps raw hash digests of - bridge ID keys to - :class:`~bridgedb.Bridges.Bridge`s. + bridge ID keys to :class:`~bridgedb.bridges.Bridge`s. :type hmac: callable :ivar hmac: An HMAC function, which uses the **key** parameter to - generate new HMACs for storing, inserting, and retrieving - :class:`~bridgedb.Bridges.Bridge`s within mappings. + generate new HMACs for storing, inserting, and retrieving + :class:`~bridgedb.bridges.Bridge`s within mappings. :ivar bool isSorted: ``True`` if ``sortedKeys`` is currently sorted. :ivar list sortedKeys: A sorted list of all of the HMACs. :ivar str name: A string which identifies this hashring, used mostly - for differentiating this hashring in log messages, but - it is also used for naming subrings. If this hashring - is a subring, the ``name`` will include whatever - distinguishing parameters differentiate that - particular subring (i.e. ``'(port-443 subring)'`` or - ``'(Stable subring)'``) + for differentiating this hashring in log messages, but it is also + used for naming subrings. If this hashring is a subring, the + ``name`` will include whatever distinguishing parameters + differentiate that particular subring (i.e. ``'(port-443 + subring)'`` or ``'(Stable subring)'``) :type subrings: list :ivar subrings: A list of other ``BridgeRing``s, each of which - contains bridges of a particular type. For example, a - subring might contain only ``Bridge``s which have been - given the "Stable" flag, or it might contain only IPv6 - bridges. Each item in this list should be a 4-tuple: + contains bridges of a particular type. For example, a subring + might contain only ``Bridge``s which have been given the "Stable" + flag, or it might contain only IPv6 bridges. Each item in this + list should be a 4-tuple:: - ``(type, value, count, ring)`` + (type, value, count, ring) - where: + where: - * ``type`` is a string which describes what kind of - parameter is used to determine if a ``Bridge`` - belongs in that subring, i.e. ``'port'`` or - ``'flag'``. + - ``type`` is a string which describes what kind of parameter is + used to determine if a ``Bridge`` belongs in that subring, + i.e. ``'port'`` or ``'flag'``. - * ``value`` is a specific value pertaining to the - ``type``, e.g. ``type='port'; value=443``. + - ``value`` is a specific value pertaining to the ``type``, + e.g. ``type='port'; value=443``. - * ``count`` is an integer for the current total - number of bridges in the subring. + - ``count`` is an integer for the current total number of + bridges in the subring. - * ``ring`` is a - :class:`~bridgedb.Bridges.BridgeRing`; it is the - sub hashring which contains ``count`` number of - :class:`~bridgedb.Bridges.Bridge`s of a certain - ``type``. + - ``ring`` is a :class:`BridgeRing`; it is the subhashring which + contains ``count`` number of + :class:`~bridgedb.bridges.Bridge`s of a certain ``type``. """ self.bridges = {} self.bridgesByID = {} @@ -268,10 +263,9 @@ class BridgeRing(BridgeHolder): return them. :param bytes pos: The position to jump to. Any bridges returned will - start at this position in the hashring, if there is - a bridge assigned to that position. Otherwise, - indexing will start at the first position after this - one which has a bridge assigned to it. + start at this position in the hashring, if there is a bridge + assigned to that position. Otherwise, indexing will start at the + first position after this one which has a bridge assigned to it. :param int N: The number of bridges to return. :rtype: list :returns: A list of :class:`~bridgedb.Bridges.Bridge`s. @@ -597,7 +591,7 @@ class FilteredBridgeSplitter(BridgeHolder): :param str ringname: A unique name identifying a sub hashring. :rtype: list :returns: A sorted list of strings, all the function names of the - filters applied to the sub hashring named **ringname**. + filters applied to the sub hashring named **ringname**. """ filterNames = [] @@ -618,14 +612,13 @@ class FilteredBridgeSplitter(BridgeHolder): :param subring: The subring to add. :param str ringname: A unique name for identifying the new subring. :param filterFn: A function whose input is a :class:`Bridge`, and - returns True/False based on some filtration criteria. + returns True/False based on some filtration criteria. :type populate_from: iterable or None :param populate_from: A group of :class:`Bridge`s. If given, the newly - added subring will be populated with these - bridges. + added subring will be populated with these bridges. :rtype: bool :returns: False if there was a problem adding the subring, True - otherwise. + otherwise. """ # XXX I think subring and ringname are switched in this function, or # at least that whatever is passed into this function as as the

1 0

[bridgedb/develop] Fix log statement parameter for Stem descriptor class names.
by isis＠torproject.org 30 Apr '15

30 Apr '15

commit 7e99a163a15cdc6c66b202aa91cc2fac2c440b52 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Apr 19 03:21:00 2015 +0000 Fix log statement parameter for Stem descriptor class names. --- lib/bridgedb/parse/descriptors.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bridgedb/parse/descriptors.py b/lib/bridgedb/parse/descriptors.py index bec10b3..3f8d48e 100644 --- a/lib/bridgedb/parse/descriptors.py +++ b/lib/bridgedb/parse/descriptors.py @@ -190,7 +190,7 @@ def deduplicate(descriptors, statistics=False): for descriptor in descriptors: fingerprint = descriptor.fingerprint logging.debug("Deduplicating %s descriptor for router %s" - % (str(descriptor.__class__).rsplit('.', 1)[1], + % (descriptor.__class__.__name__.rsplit('.', 1)[-1], safelog.logSafely(fingerprint))) if fingerprint in duplicates: duplicates[fingerprint].append(descriptor)

1 0