April 2014 - tor-commits - lists.torproject.org

[bridgedb/master] Add file headers and fix module docstring for bridgedb.Bridges.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit 8c04ac62954e99de131ac9b409412aeda3ac1c12 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 12:06:17 2014 +0000 Add file headers and fix module docstring for bridgedb.Bridges. --- lib/bridgedb/Bridges.py | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/lib/bridgedb/Bridges.py b/lib/bridgedb/Bridges.py index bfe1316..14820cc 100644 --- a/lib/bridgedb/Bridges.py +++ b/lib/bridgedb/Bridges.py @@ -1,8 +1,13 @@ -# See LICENSE for licensing information - -""" -This module has low-level functionality for parsing bridges and arranging -them in rings. +# -*- coding: utf-8 ; test-case-name: bridgedb.test.test_Bridges -*- +# +# This file is part of BridgeDB, a Tor bridge distribution system. +# +# :authors: see AUTHORS file +# :copyright: (c) 2007-2014, The Tor Project, Inc. +# :license: 3-Clause BSD, see LICENSE for licensing information + +"""This module has low-level functionality for parsing bridges and arranging +them into hashrings for distributors. """ import binascii

1 0

[bridgedb/master] Fix a bug in bridgedb.Bridges due to catching a non-existent exception.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit 88e6aeaed8c55362739550561369a50bdddbed33 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 12:06:52 2014 +0000 Fix a bug in bridgedb.Bridges due to catching a non-existent exception. --- lib/bridgedb/Bridges.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bridgedb/Bridges.py b/lib/bridgedb/Bridges.py index 14820cc..5c406f6 100644 --- a/lib/bridgedb/Bridges.py +++ b/lib/bridgedb/Bridges.py @@ -683,7 +683,7 @@ def parseStatusFile(networkstatusFile): elif ID and line.startswith("a "): try: addr, portlist = networkstatus.parseALine(line, toHex(ID)) - except networkstatus.ParseNetstatusError as error: + except networkstatus.NetworkstatusParsingError as error: logging.error(error) else: if (addr is not None) and (portlist is not None):

1 0

[bridgedb/master] Add missing CHANGELOG entry for #10724 to 0.1.1.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit dee20f0db51ff616fb89595fe290277c294b8ebe Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 11:03:10 2014 +0000 Add missing CHANGELOG entry for #10724 to 0.1.1. --- CHANGELOG | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/CHANGELOG b/CHANGELOG index 7f160df..7dc867b 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -81,6 +81,20 @@ BridgeDB 0.1.1 includes fixes for the following bugs: * FIXES #9127 Users can't ask for ipv6 bridges with the new bridgedb interface * FIXES #9988 Refactor BridgeDB's use of `sha` module to use `hashlib` + * FIXES #10724 Most of the time spent in testing, as well as most + of the time spent when "BridgeDB is down" (i.e. when I reply + with "BridgeDB is single-threaded (see #5232) and is parsing + millions of descriptors"), is within the same + `bridgedb.Stability.addOrUpdateBridgeHistory()` function (see + #10724). This function is pretty brutal on CPU and memory, is + blocking, and it needs to runs thousands and thousands of + times whenever BridgeDB is restarted (every half hour). The + algorithm within that function has a time complexity increasing + linearithmically relative to the number of bridges and timestamps + already within the database. [0] + This patch adds a config option to disable/enable running + the `addOrUpdateBridgeHistory()` function. + [0]: https://en.wikipedia.org/wiki/Time_complexity#Linearithmic_time * FIXES #10737 POST arguments to bridges.torproject.org are dropped if entering a CAPTCHA fails

1 0

[bridgedb/master] Whitespace fixes in CHANGELOG.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit 26d0271d3fbb7c214a2fbbe80be449f607ac4e6b Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 11:20:00 2014 +0000 Whitespace fixes in CHANGELOG. --- CHANGELOG | 228 ++++++++++++++++++++++++++++++------------------------------- 1 file changed, 114 insertions(+), 114 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 2a043a0..22b5baa 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -17,157 +17,157 @@ And includes the following general changes: Changes in version 0.1.6 - 2014-03-26 BridgeDB 0.1.6 includes fixes for the following bugs: - * FIXES #11196 BridgeDB should use leekspin - * FIXES #11218 ReCaptchaProtectedResource.checkSolution() - doesn't expect a deferred - * FIXES #11219 BridgeDB's twisted version doesn´t have a - `t.w.client.HTTPConnectionPool` class - * FIXES #11231 BridgeDB's txrecaptcha returns the "No bridges - available!" page if 'captcha_response_field' is blank + * FIXES #11196 BridgeDB should use leekspin + * FIXES #11218 ReCaptchaProtectedResource.checkSolution() + doesn't expect a deferred + * FIXES #11219 BridgeDB's twisted version doesn´t have a + `t.w.client.HTTPConnectionPool` class + * FIXES #11231 BridgeDB's txrecaptcha returns the "No bridges + available!" page if 'captcha_response_field' is blank And includes the following general changes: - * ADDS several new translations languages. + * ADDS several new translations languages. Changes in version 0.1.5 - 2014-02-27 BridgeDB 0.1.5 includes fixes for the following bugs: - * FIXES #9264 Problem with transport lines in BridgeDB's bridge - pool assignment files - * FIXES #10809 reCAPTCHA on bridges.torproject.org are impossible - to solve for humans - * FIXES #10834 Configurable reCAPTCHA remoteip - * FIXES #11127 reCaptcha verification is hardcoded to use - plaintext HTTP + * FIXES #9264 Problem with transport lines in BridgeDB's bridge + pool assignment files + * FIXES #10809 reCAPTCHA on bridges.torproject.org are impossible + to solve for humans + * FIXES #10834 Configurable reCAPTCHA remoteip + * FIXES #11127 reCaptcha verification is hardcoded to use + plaintext HTTP And includes the following general changes: - * CHANGES the way the client's IP address is reported to the - reCaptcha API server. Previously, for each client request, a - completely random IP address was generated, and BridgeDB lied to - the reCaptcha server by sending this random IP and saying that it - was the client's IP. With these changes, BridgeDB can be - configured to report a static IP address (it's own IP) as the - client's IP, in the hopes that the IP whitelisting used by - reCaptcha makes the returned CAPTCHAs less impossible to solve. - * ADDS the ability to use a local cache of CAPTCHAs created by - scripting Gimp with Python. See: - https://github.com/isislovecruft/gimp-captcha - * USES TLS during CAPTCHA solution verification when using - reCaptcha. + * CHANGES the way the client's IP address is reported to the + reCaptcha API server. Previously, for each client request, a + completely random IP address was generated, and BridgeDB lied to + the reCaptcha server by sending this random IP and saying that it + was the client's IP. With these changes, BridgeDB can be + configured to report a static IP address (it's own IP) as the + client's IP, in the hopes that the IP whitelisting used by + reCaptcha makes the returned CAPTCHAs less impossible to solve. + * ADDS the ability to use a local cache of CAPTCHAs created by + scripting Gimp with Python. See: + https://github.com/isislovecruft/gimp-captcha + * USES TLS during CAPTCHA solution verification when using + reCaptcha. Changes in version 0.1.4 - 2014-02-21 BridgeDB 0.1.4 includes fixes for the following bugs: - * FIXES an RTL encoding issue in the "Step 2" text on index.html - of the HTTPS distributor. + * FIXES an RTL encoding issue in the "Step 2" text on index.html + of the HTTPS distributor. Changes in version 0.1.3 - 2014-02-21 BridgeDB 0.1.3 includes fixes for the following bugs: - * FIXES #9264 Problem with transport lines in BridgeDB's bridge - pool assignment files - * FIXES a bug caused by attempts to convert descriptor digests - which could not be parsed into hexadecimal. - * FIXES a unicode decoding error within the bridgedb.persistent - module. - * REMOVES continuous integration testing for Python 2.6. + * FIXES #9264 Problem with transport lines in BridgeDB's bridge + pool assignment files + * FIXES a bug caused by attempts to convert descriptor digests + which could not be parsed into hexadecimal. + * FIXES a unicode decoding error within the bridgedb.persistent + module. + * REMOVES continuous integration testing for Python 2.6. Including the following general changes: - * ADD volunteer page text for GSoC 2014 applications. + * ADD volunteer page text for GSoC 2014 applications. Changes in version 0.1.2 - 2014-02-05 BridgeDB 0.1.2 includes fixes for the following bugs: - * FIXES #10811 BridgeDB's assignments.log files are not being updated + * FIXES #10811 BridgeDB's assignments.log files are not being updated Changes in version 0.1.1 - 2014-02-03 BridgeDB 0.1.1 includes fixes for the following bugs: - * FIXES #9127 Users can't ask for ipv6 bridges with the new bridgedb - interface - * FIXES #9988 Refactor BridgeDB's use of `sha` module to use `hashlib` - * FIXES #10724 Most of the time spent in testing, as well as most - of the time spent when "BridgeDB is down" (i.e. when I reply - with "BridgeDB is single-threaded (see #5232) and is parsing - millions of descriptors"), is within the same - `bridgedb.Stability.addOrUpdateBridgeHistory()` function (see - #10724). This function is pretty brutal on CPU and memory, is - blocking, and it needs to runs thousands and thousands of - times whenever BridgeDB is restarted (every half hour). The - algorithm within that function has a time complexity increasing - linearithmically relative to the number of bridges and timestamps - already within the database. [0] - This patch adds a config option to disable/enable running - the `addOrUpdateBridgeHistory()` function. - [0]: https://en.wikipedia.org/wiki/Time_complexity#Linearithmic_time - * FIXES #10737 POST arguments to bridges.torproject.org are dropped if - entering a CAPTCHA fails + * FIXES #9127 Users can't ask for ipv6 bridges with the new bridgedb + interface + * FIXES #9988 Refactor BridgeDB's use of `sha` module to use `hashlib` + * FIXES #10724 Most of the time spent in testing, as well as most + of the time spent when "BridgeDB is down" (i.e. when I reply + with "BridgeDB is single-threaded (see #5232) and is parsing + millions of descriptors"), is within the same + `bridgedb.Stability.addOrUpdateBridgeHistory()` function (see + #10724). This function is pretty brutal on CPU and memory, is + blocking, and it needs to runs thousands and thousands of + times whenever BridgeDB is restarted (every half hour). The + algorithm within that function has a time complexity increasing + linearithmically relative to the number of bridges and timestamps + already within the database. [0] + This patch adds a config option to disable/enable running + the `addOrUpdateBridgeHistory()` function. + [0]: https://en.wikipedia.org/wiki/Time_complexity#Linearithmic_time + * FIXES #10737 POST arguments to bridges.torproject.org are dropped if + entering a CAPTCHA fails Including the following general changes: - * ADDS A general simple error page to display rather than - webserver tracebacks. See #6127. - * MOVES The recently added BridgeDB specification file - ('doc/bridge-db-spec.txt') to the torspec repo: - https://gitweb.torproject.org/torspec.git - * FIXES template caching for the web interface. + * ADDS A general simple error page to display rather than + webserver tracebacks. See #6127. + * MOVES The recently added BridgeDB specification file + ('doc/bridge-db-spec.txt') to the torspec repo: + https://gitweb.torproject.org/torspec.git + * FIXES template caching for the web interface. Changes in version 0.1.0 - 2014-01-14 BridgeDB 0.1.0 includes fixes for the following bugs: - * FIXES #1606 Write a BridgeDB spec - * FIXES #3573 bridges.torproject.org doesn't have a robots.txt - * FIXES #6127 bridges.tpo runs in development mode - * FIXES #9013 BridgeDB should pass pluggable transport - shared-secrets to clients - * FIXES #9157 Persian and Arabic should be right aligned on bridges.tpo - * FIXES #9462 BridgeDB netstatus descriptor parsers need refactoring - * FIXES #9959 BridgeDB seems to be missing English translations - * FIXES #9865 Add automated code coverage report generation - * FIXES #9872 Create a test runner for BridgeDB unittests - * FIXES #9873 Convert BridgeDB's old unittests to use twisted.trial - * FIXES #9937 Create fake non-sanitised bridge descriptors for - BridgeDB testing purposes - * FIXES #10333 Indexing list-like objects by 0L in - Bridges.getConfigLine - * FIXES #10446 BridgeDB is/was using a GeoIP module which is - incompatible with virtualenvs - * FIXES #10559 BridgeDB writes `keyid=` before fingerprints + * FIXES #1606 Write a BridgeDB spec + * FIXES #3573 bridges.torproject.org doesn't have a robots.txt + * FIXES #6127 bridges.tpo runs in development mode + * FIXES #9013 BridgeDB should pass pluggable transport + shared-secrets to clients + * FIXES #9157 Persian and Arabic should be right aligned on bridges.tpo + * FIXES #9462 BridgeDB netstatus descriptor parsers need refactoring + * FIXES #9959 BridgeDB seems to be missing English translations + * FIXES #9865 Add automated code coverage report generation + * FIXES #9872 Create a test runner for BridgeDB unittests + * FIXES #9873 Convert BridgeDB's old unittests to use twisted.trial + * FIXES #9937 Create fake non-sanitised bridge descriptors for + BridgeDB testing purposes + * FIXES #10333 Indexing list-like objects by 0L in + Bridges.getConfigLine + * FIXES #10446 BridgeDB is/was using a GeoIP module which is + incompatible with virtualenvs + * FIXES #10559 BridgeDB writes `keyid=` before fingerprints Including the following general changes: - * ADD Numerous unittests and automated continuous integration testing. - * ADD Patches by sysrqb to correctly render right-to-left languages. - * FIXES fallback languages for translations. - * ADD Sphinx makefiles, substantial amounts of documentation. - * ADD Documentations builds (currently at - https://para.noid.cat/bridgedb) + * ADD Numerous unittests and automated continuous integration testing. + * ADD Patches by sysrqb to correctly render right-to-left languages. + * FIXES fallback languages for translations. + * ADD Sphinx makefiles, substantial amounts of documentation. + * ADD Documentations builds (currently at + https://para.noid.cat/bridgedb) Changes in version 0.0.1 - 2013-08-20 BridgeDB 0.0.1 includes fixes for the following bugs: - * FIXES #5332 Update BridgeDB documentation with deployment - instructions - * FIXES #9156 BridgeDB: Users try to add obfsbridges to their - normal TBB - These commits were added to fix a compatibility issue with - Vidalia, though they may be reverted to switch back to the old - behaviour of returning bridge lines in the form: - Bridge [transport_method] address:port [keyid=fingerprint] [K=v] […] - in order to work with torrc files and the new TBB-3.x packages - which use TorLauncher instead of Vidalia. - * PARTIAL FIX #9264 Problem with transport lines in BridgeDB's - bridge pool assignment files. - * FIXES #9425 Create and document a better BridgeDB (re)deployment - strategy + * FIXES #5332 Update BridgeDB documentation with deployment + instructions + * FIXES #9156 BridgeDB: Users try to add obfsbridges to their + normal TBB + These commits were added to fix a compatibility issue with + Vidalia, though they may be reverted to switch back to the old + behaviour of returning bridge lines in the form: + Bridge [transport_method] address:port [keyid=fingerprint] [K=v] […] + in order to work with torrc files and the new TBB-3.x packages + which use TorLauncher instead of Vidalia. + * PARTIAL FIX #9264 Problem with transport lines in BridgeDB's + bridge pool assignment files. + * FIXES #9425 Create and document a better BridgeDB (re)deployment + strategy Including the following general changes: - * UPDATE translations files with finished files from Transifex for - the strings for the newly-refactored web interface created by - gsathya. - * ADD an automatic version-numbering system as part of developing - a better deployment strategy. - * CHANGE setup.py script to automatically install Python-based - dependencies from requirements.txt. - * REMOVE MANIFEST.in and put equivalent 'include' directives into - setup.py (the less places we have for manually keeping track of - files, the better). - * REMOVE the "bridge " prefix from the lines returned on the web - interface. + * UPDATE translations files with finished files from Transifex for + the strings for the newly-refactored web interface created by + gsathya. + * ADD an automatic version-numbering system as part of developing + a better deployment strategy. + * CHANGE setup.py script to automatically install Python-based + dependencies from requirements.txt. + * REMOVE MANIFEST.in and put equivalent 'include' directives into + setup.py (the less places we have for manually keeping track of + files, the better). + * REMOVE the "bridge " prefix from the lines returned on the web + interface.

1 0

[bridgedb/master] Add file headers and module docstring to bridgedb.Util.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit cc2aab80c9b955ab888b8ee7e7ecae72bd1bd713 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 14:16:18 2014 +0000 Add file headers and module docstring to bridgedb.Util. --- lib/bridgedb/Util.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/lib/bridgedb/Util.py b/lib/bridgedb/Util.py index e0ce81f..6c70dff 100644 --- a/lib/bridgedb/Util.py +++ b/lib/bridgedb/Util.py @@ -1,3 +1,16 @@ +# -*- coding: utf-8 ; test-case-name: bridgedb.test.test_Util -*- +# +# This file is part of BridgeDB, a Tor bridge distribution system. +# +# :authors: Isis Lovecruft 0xA3ADB67A2CDB8B35 <isis(a)torproject.org> +# Matthew Finkel 0x017DD169EA793BE2 <sysrqb(a)torproject.org> +# :copyright: (c) 2013-2014, Isis Lovecruft +# (c) 2013-2014, Matthew Finkel +# (c) 2007-2014, The Tor Project, Inc. +# :license: 3-Clause BSD, see LICENSE for licensing information + +"""Common utilities for BridgeDB.""" + safe_logging = True def set_safe_logging(safe):

1 0

[bridgedb/master] Remove extra EOF newline from bridgedb.Util.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit 309d098ee2fa00451d9062cb5e28be81b556174e Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 14:16:41 2014 +0000 Remove extra EOF newline from bridgedb.Util. --- lib/bridgedb/Util.py | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/bridgedb/Util.py b/lib/bridgedb/Util.py index 6c70dff..de512d5 100644 --- a/lib/bridgedb/Util.py +++ b/lib/bridgedb/Util.py @@ -22,4 +22,3 @@ def logSafely(val): return "[scrubbed]" else: return val -

1 0

[bridgedb/master] Move bridgedb.Util → bridgedb.util.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit d8088ca7a6955dc0c528092faaf135817fa28dc2 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 14:36:39 2014 +0000 Move bridgedb.Util → bridgedb.util. --- doc/sphinx/source/bridgedb.Util.rst | 6 +++--- doc/sphinx/source/bridgedb.rst | 3 ++- doc/sphinx/source/conf.py | 2 +- lib/bridgedb/Bridges.py | 10 +++++----- lib/bridgedb/Dist.py | 16 ++++++++-------- lib/bridgedb/EmailServer.py | 22 +++++++++++----------- lib/bridgedb/HTTPServer.py | 12 ++++++------ lib/bridgedb/Main.py | 4 ++-- lib/bridgedb/Util.py | 24 ------------------------ lib/bridgedb/util.py | 24 ++++++++++++++++++++++++ 10 files changed, 62 insertions(+), 61 deletions(-) diff --git a/doc/sphinx/source/bridgedb.Util.rst b/doc/sphinx/source/bridgedb.Util.rst index 70b8713..63db16d 100644 --- a/doc/sphinx/source/bridgedb.Util.rst +++ b/doc/sphinx/source/bridgedb.Util.rst @@ -1,9 +1,9 @@ -.. _Util: +.. _util: -bridgedb.Util +bridgedb.util ------------- -.. automodule:: bridgedb.Util +.. automodule:: bridgedb.util :members: :undoc-members: :show-inheritance: diff --git a/doc/sphinx/source/bridgedb.rst b/doc/sphinx/source/bridgedb.rst index 38dc9a2..40e0151 100644 --- a/doc/sphinx/source/bridgedb.rst +++ b/doc/sphinx/source/bridgedb.rst @@ -21,5 +21,6 @@ bridgedb bridgedb.Stability bridgedb.Storage bridgedb.Tests - bridgedb.Util + bridgedb.txrecaptcha + bridgedb.util gen_bridge_descriptors diff --git a/doc/sphinx/source/conf.py b/doc/sphinx/source/conf.py index de44db4..4caac64 100644 --- a/doc/sphinx/source/conf.py +++ b/doc/sphinx/source/conf.py @@ -43,7 +43,7 @@ import bridgedb.Stability import bridgedb.Storage import bridgedb.test import bridgedb.Tests -import bridgedb.Util +import bridgedb.util import gen_bridge_descriptors diff --git a/lib/bridgedb/Bridges.py b/lib/bridgedb/Bridges.py index 5c406f6..a60ff85 100644 --- a/lib/bridgedb/Bridges.py +++ b/lib/bridgedb/Bridges.py @@ -22,8 +22,8 @@ import random import bridgedb.Storage import bridgedb.Bucket -import bridgedb.Util as Util +from bridgedb import util from bridgedb.crypto import getHMACFunc from bridgedb.parse import addr from bridgedb.parse import networkstatus @@ -1035,7 +1035,7 @@ class BridgeRing(BridgeHolder): else: logging.debug( "Got duplicate bridge %r in main hashring for position %r." - % (Util.logSafely(k.encode('hex')), pos.encode('hex'))) + % (util.logSafely(k.encode('hex')), pos.encode('hex'))) keys = keys[:N] keys.sort() @@ -1280,12 +1280,12 @@ class FilteredBridgeSplitter(BridgeHolder): if not bridge.running: logging.warn( "Skipping hashring insertion for non-running bridge: '%s'" - % Util.logSafely(bridge.fingerprint)) + % util.logSafely(bridge.fingerprint)) return index = 0 logging.debug("Inserting %s into splitter" - % (Util.logSafely(bridge.fingerprint))) + % (util.logSafely(bridge.fingerprint))) for old_bridge in self.bridges[:]: if bridge.fingerprint == old_bridge.fingerprint: self.bridges[index] = bridge @@ -1297,7 +1297,7 @@ class FilteredBridgeSplitter(BridgeHolder): if filterFn(bridge): subring.insert(bridge) logging.debug("Inserted bridge '%s' into '%s' sub hashring" - % (Util.logSafely(bridge.fingerprint), ringname)) + % (util.logSafely(bridge.fingerprint), ringname)) def extractFilterNames(self, ringname): """Get the names of the filters applied to a particular sub hashring. diff --git a/lib/bridgedb/Dist.py b/lib/bridgedb/Dist.py index 7107867..e7bc443 100644 --- a/lib/bridgedb/Dist.py +++ b/lib/bridgedb/Dist.py @@ -14,7 +14,7 @@ import bridgedb.Bridges import bridgedb.Storage -import bridgedb.Util as Util +import bridgedb.util as util import logging import re @@ -235,7 +235,7 @@ class IPBasedDistributor(Distributor): for an example of how this is used. """ logging.info("Attempting to return %d bridges to client %s..." - % (N, Util.logSafely(ip))) + % (N, util.logSafely(ip))) if not bridgeFilterRules: bridgeFilterRules=[] @@ -251,7 +251,7 @@ class IPBasedDistributor(Distributor): area = self.areaMapper(ip) logging.debug("IP mapped to area:\t%s" - % Util.logSafely("{0}.0/24".format(area))) + % util.logSafely("{0}.0/24".format(area))) key1 = '' pos = 0 @@ -267,7 +267,7 @@ class IPBasedDistributor(Distributor): len(self.categories), n) bridgeFilterRules.append(g) - logging.info("category<%s>%s", epoch, Util.logSafely(area)) + logging.info("category<%s>%s", epoch, util.logSafely(area)) pos = self.areaOrderHmac("category<%s>%s" % (epoch, area)) key1 = getHMAC(self.splitter.key, "Order-Bridges-In-Ring-%d" % n) @@ -485,19 +485,19 @@ class EmailBasedDistributor(Distributor): lastSaw = db.getEmailTime(emailaddress) logging.info("Attempting to return for %d bridges for %s..." - % (N, Util.logSafely(emailaddress))) + % (N, util.logSafely(emailaddress))) if lastSaw is not None and lastSaw + MAX_EMAIL_RATE >= now: logging.info("Client %s sent duplicate request within %d seconds." - % (Util.logSafely(emailaddress), MAX_EMAIL_RATE)) + % (util.logSafely(emailaddress), MAX_EMAIL_RATE)) if wasWarned: logging.info( "Client was already warned about duplicate requests.") raise IgnoreEmail("Client was warned", - Util.logSafely(emailaddress)) + util.logSafely(emailaddress)) else: logging.info("Sending duplicate request warning to %s..." - % Util.logSafely(emailaddress)) + % util.logSafely(emailaddress)) db.setWarnedEmail(emailaddress, True, now) db.commit() diff --git a/lib/bridgedb/EmailServer.py b/lib/bridgedb/EmailServer.py index 90b22af..0242103 100644 --- a/lib/bridgedb/EmailServer.py +++ b/lib/bridgedb/EmailServer.py @@ -29,7 +29,7 @@ from twisted.internet.error import ConnectionRefusedError from zope.interface import implements import bridgedb.Dist -import bridgedb.Util as Util +import bridgedb.util as util from bridgedb.Dist import BadEmail, TooSoonEmail, IgnoreEmail from bridgedb.Filters import filterBridgesByIP6, filterBridgesByIP4 from bridgedb.Filters import filterBridgesByTransport @@ -104,11 +104,11 @@ def getMailResponse(lines, ctx): logging.info("Ignoring bad address on incoming email.") return None,None if not addrdomain: - logging.info("Couldn't parse domain from %r", Util.logSafely(clientAddr)) + logging.info("Couldn't parse domain from %r", util.logSafely(clientAddr)) if addrdomain and ctx.cfg.EMAIL_DOMAIN_MAP: addrdomain = ctx.cfg.EMAIL_DOMAIN_MAP.get(addrdomain, addrdomain) if addrdomain not in ctx.cfg.EMAIL_DOMAINS: - logging.info("Unrecognized email domain %r", Util.logSafely(addrdomain)) + logging.info("Unrecognized email domain %r", util.logSafely(addrdomain)) return None,None rules = ctx.cfg.EMAIL_DOMAIN_RULES.get(addrdomain, []) if 'dkim' in rules: @@ -181,7 +181,7 @@ def getMailResponse(lines, ctx): # Handle rate limited email except TooSoonEmail, e: logging.info("Got a mail too frequently; warning %r: %s.", - Util.logSafely(clientAddr), e) + util.logSafely(clientAddr), e) # Compose a warning email # MAX_EMAIL_RATE is in seconds, convert to hours @@ -191,12 +191,12 @@ def getMailResponse(lines, ctx): except IgnoreEmail, e: logging.info("Got a mail too frequently; ignoring %r: %s.", - Util.logSafely(clientAddr), e) + util.logSafely(clientAddr), e) return None, None except BadEmail, e: logging.info("Got a mail from a bad email address %r: %s.", - Util.logSafely(clientAddr), e) + util.logSafely(clientAddr), e) return None, None if bridges: @@ -269,18 +269,18 @@ def replyToMail(lines, ctx): if response is None: logging.debug("getMailResponse() said not to reply to %s, so I won't." - % Util.logSafely(sendToUser)) + % util.logSafely(sendToUser)) return response.seek(0) - logging.info("Sending reply to %r", Util.logSafely(sendToUser)) + logging.info("Sending reply to %r", util.logSafely(sendToUser)) d = Deferred() factory = twisted.mail.smtp.SMTPSenderFactory(ctx.smtpFromAddr, sendToUser, response, d, retries=0, timeout=30) d.addErrback(_ebReplyToMailFailure) - logging.info("Sending reply to %r", Util.logSafely(sendToUser)) + logging.info("Sending reply to %r", util.logSafely(sendToUser)) reactor.connectTCP(ctx.smtpServer, ctx.smtpPort, factory) return d @@ -461,11 +461,11 @@ def composeEmail(fromAddr, clientAddr, subject, body, msgID=False, # Only log the email text (including all headers) if SAFE_LOGGING is # disabled: - if not Util.safe_logging: + if not util.safe_logging: mail.seek(0) logging.debug("Email contents:\n%s" % mail.read()) else: - logging.debug("Email text for %r created." % Util.logSafely(clientAddr)) + logging.debug("Email text for %r created." % util.logSafely(clientAddr)) mail.seek(0) return clientAddr, mail diff --git a/lib/bridgedb/HTTPServer.py b/lib/bridgedb/HTTPServer.py index 3809f2d..c0e5b66 100644 --- a/lib/bridgedb/HTTPServer.py +++ b/lib/bridgedb/HTTPServer.py @@ -24,11 +24,11 @@ from twisted.web.util import redirectTo import bridgedb.Dist import bridgedb.I18n as I18n -import bridgedb.Util as Util from bridgedb import captcha from bridgedb import crypto from bridgedb import txrecaptcha +from bridgedb import util from bridgedb.Filters import filterBridgesByIP4 from bridgedb.Filters import filterBridgesByIP6 from bridgedb.Filters import filterBridgesByTransport @@ -287,7 +287,7 @@ class GimpCaptchaProtectedResource(CaptchaProtectedResource): valid = captcha.GimpCaptcha.check(challenge, solution, self.secretKey, clientHMACKey) logging.debug("%sorrect captcha from %r: %r." % ( - "C" if valid else "Inc", Util.logSafely(clientIP), solution)) + "C" if valid else "Inc", util.logSafely(clientIP), solution)) return valid @@ -463,7 +463,7 @@ class ReCaptchaProtectedResource(CaptchaProtectedResource): remoteIP = self.getRemoteIP() logging.debug("Captcha from %r. Parameters: %r" - % (Util.logSafely(clientIP), request.args)) + % (util.logSafely(clientIP), request.args)) def checkResponse(solution, request): """Check the :class:`txrecaptcha.RecaptchaResponse`. @@ -479,11 +479,11 @@ class ReCaptchaProtectedResource(CaptchaProtectedResource): # breaking). Hence, the 'no cover' pragma. if solution.is_valid: # pragma: no cover logging.info("Valid CAPTCHA solution from %r." - % Util.logSafely(clientIP)) + % util.logSafely(clientIP)) return (True, request) else: logging.info("Invalid CAPTCHA solution from %r: %r" - % (Util.logSafely(clientIP), solution.error_code)) + % (util.logSafely(clientIP), solution.error_code)) return (False, request) d = txrecaptcha.submit(challenge, response, self.recaptchaPrivKey, @@ -669,7 +669,7 @@ class WebResourceBridges(resource.Resource): unblocked = False logging.info("Replying to web request from %s. Parameters were %r" - % (Util.logSafely(ip), request.args)) + % (util.logSafely(ip), request.args)) rules = [] bridgeLines = None diff --git a/lib/bridgedb/Main.py b/lib/bridgedb/Main.py index d21239c..c2c69c9 100644 --- a/lib/bridgedb/Main.py +++ b/lib/bridgedb/Main.py @@ -26,7 +26,7 @@ import bridgedb.Bridges as Bridges import bridgedb.Dist as Dist import bridgedb.Time as Time import bridgedb.Storage -import bridgedb.Util as Util +from bridgedb import util def configureLogging(cfg): @@ -41,7 +41,7 @@ def configureLogging(cfg): logfile = getattr(cfg, 'LOGFILE', "") logfile_count = getattr(cfg, 'LOGFILE_COUNT', 5) logfile_rotate_size = getattr(cfg, 'LOGFILE_ROTATE_SIZE', 10000000) - Util.set_safe_logging(safelogging) + util.set_safe_logging(safelogging) logging.getLogger().setLevel(level) if logfile: diff --git a/lib/bridgedb/Util.py b/lib/bridgedb/Util.py deleted file mode 100644 index de512d5..0000000 --- a/lib/bridgedb/Util.py +++ /dev/null @@ -1,24 +0,0 @@ -# -*- coding: utf-8 ; test-case-name: bridgedb.test.test_Util -*- -# -# This file is part of BridgeDB, a Tor bridge distribution system. -# -# :authors: Isis Lovecruft 0xA3ADB67A2CDB8B35 <isis(a)torproject.org> -# Matthew Finkel 0x017DD169EA793BE2 <sysrqb(a)torproject.org> -# :copyright: (c) 2013-2014, Isis Lovecruft -# (c) 2013-2014, Matthew Finkel -# (c) 2007-2014, The Tor Project, Inc. -# :license: 3-Clause BSD, see LICENSE for licensing information - -"""Common utilities for BridgeDB.""" - -safe_logging = True - -def set_safe_logging(safe): - global safe_logging - safe_logging = safe - -def logSafely(val): - if safe_logging: - return "[scrubbed]" - else: - return val diff --git a/lib/bridgedb/util.py b/lib/bridgedb/util.py new file mode 100644 index 0000000..723ed8e --- /dev/null +++ b/lib/bridgedb/util.py @@ -0,0 +1,24 @@ +# -*- coding: utf-8 ; test-case-name: bridgedb.test.test_util -*- +# +# This file is part of BridgeDB, a Tor bridge distribution system. +# +# :authors: Isis Lovecruft 0xA3ADB67A2CDB8B35 <isis(a)torproject.org> +# Matthew Finkel 0x017DD169EA793BE2 <sysrqb(a)torproject.org> +# :copyright: (c) 2013-2014, Isis Lovecruft +# (c) 2013-2014, Matthew Finkel +# (c) 2007-2014, The Tor Project, Inc. +# :license: 3-Clause BSD, see LICENSE for licensing information + +"""Common utilities for BridgeDB.""" + +safe_logging = True + +def set_safe_logging(safe): + global safe_logging + safe_logging = safe + +def logSafely(val): + if safe_logging: + return "[scrubbed]" + else: + return val

1 0

[bridgedb/master] Remove pprinting of configuration changes from bridgedb.Main.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit 5957efdaf4b671ffe57133ce340d86bab9e9c87d Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 14:14:17 2014 +0000 Remove pprinting of configuration changes from bridgedb.Main. They went to stdout, not the logger, and they were annoying. --- lib/bridgedb/Main.py | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/lib/bridgedb/Main.py b/lib/bridgedb/Main.py index 8a1b650..d21239c 100644 --- a/lib/bridgedb/Main.py +++ b/lib/bridgedb/Main.py @@ -16,8 +16,6 @@ import logging import logging.handlers import gettext -from pprint import pprint - from twisted.internet import reactor from bridgedb import crypto @@ -267,8 +265,9 @@ def loadConfig(configFile=None, configCls=None): logging.info("Reloading over in-memory configurations...") if (len(configuration) > 0) and itsSafeToUseLogging: - logging.debug("Old configuration settings:\n%s" - % pprint(configuration, depth=4)) + logging.debug("Old configuration settings:") + logging.debug("\n".join(["{key} = {value}".format(key, value) + for key, value in configuration])) conffile = configFile if (configFile is None) and ('CONFIG_FILE' in configuration):

1 0

[bridgedb/master] Add sphinx build source for bridgedb.txrecaptcha module.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit 2f97b58b24d584d6b14f98941d2ebd6f6287c19d Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 14:43:39 2014 +0000 Add sphinx build source for bridgedb.txrecaptcha module. --- doc/sphinx/source/bridgedb.txrecaptcha.rst | 9 +++++++++ doc/sphinx/source/conf.py | 1 + 2 files changed, 10 insertions(+) diff --git a/doc/sphinx/source/bridgedb.txrecaptcha.rst b/doc/sphinx/source/bridgedb.txrecaptcha.rst new file mode 100644 index 0000000..afb6d1c --- /dev/null +++ b/doc/sphinx/source/bridgedb.txrecaptcha.rst @@ -0,0 +1,9 @@ +.. _txrecaptcha: + +bridgedb.txrecaptcha +-------------------- + +.. automodule:: bridgedb.txrecaptcha + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/sphinx/source/conf.py b/doc/sphinx/source/conf.py index 1fdfddd..2f05ca7 100644 --- a/doc/sphinx/source/conf.py +++ b/doc/sphinx/source/conf.py @@ -43,6 +43,7 @@ import bridgedb.Stability import bridgedb.Storage import bridgedb.test import bridgedb.Tests +import bridgedb.txrecaptcha import bridgedb.util

1 0

[bridgedb/master] Update bridgedb.runner.generateDescriptors to mention leekspin.
by isis＠torproject.org 19 Apr '14

19 Apr '14

commit a2f59909bdde9f5c31f1282ad028184eb3fe0a72 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Apr 2 14:41:40 2014 +0000 Update bridgedb.runner.generateDescriptors to mention leekspin. --- lib/bridgedb/runner.py | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/lib/bridgedb/runner.py b/lib/bridgedb/runner.py index d067bd9..4b4564f 100644 --- a/lib/bridgedb/runner.py +++ b/lib/bridgedb/runner.py @@ -22,15 +22,16 @@ import sys def generateDescriptors(count=None, rundir=None): """Run a script which creates fake bridge descriptors for testing purposes. - This will run ``scripts/gen_bridge_descriptors`` to create bridge server - descriptors, bridge extra-info descriptors, and networkstatus document. - - ** NOTE: ** - This function can take a very long time to run, especially in headless - environments where entropy sources are minimal, because it creates the - keys for each mocked OR, which are embedded in the server descriptors, - used to calculate the OR fingerprints, and sign the descriptors, among - other things. + This will run Leekspin_ to create bridge server descriptors, bridge + extra-info descriptors, and networkstatus document. + + .. warning: This function can take a very long time to run, especially in + headless environments where entropy sources are minimal, because it + creates the keys for each mocked OR, which are embedded in the server + descriptors, used to calculate the OR fingerprints, and sign the + descriptors, among other things. + + .. _Leekspin: https://gitweb.torproject.org/user/isis/leekspin.git :param integer count: Number of mocked bridges to generate descriptor for. (default: 3)

1 0