July 2016 - tor-commits - lists.torproject.org

[bridgedb/master] Fix an Emacs file encoding specification in a unittest file.
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit dec3b6e7ee9ab0034815150b4940e5a10f4b0190 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon May 2 16:19:44 2016 +0000 Fix an Emacs file encoding specification in a unittest file. --- test/test_https_server.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/test_https_server.py b/test/test_https_server.py index ce388a1..40855fa 100644 --- a/test/test_https_server.py +++ b/test/test_https_server.py @@ -1,4 +1,4 @@ -# -*- encoding: utf-8 -*- +# -*- coding: utf-8 -*- #_____________________________________________________________________________ # # This file is part of BridgeDB, a Tor bridge distribution system.

1 0

[bridgedb/develop] Add a CHANGELOG entry for #18237.
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit 743917a843be970847fdd3933af026f7c3e27e8e Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon May 2 15:05:24 2016 +0000 Add a CHANGELOG entry for #18237. --- CHANGELOG | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/CHANGELOG b/CHANGELOG index ce2e47f..036c057 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,3 +1,11 @@ +Changes in version 0.3.6 - 2016-XXX + + * FIXES #18237 https://bugs.torproject.org/18237 + During descriptor parsing, BridgeDB saves copies of descriptor files + which couldn't be parsed, for later debugging purposes. To avoid + filing up the runtime directory with these files, we now delete files + older than 24 hours, every 24 hours. + Changes in version 0.3.4 - 0.3.5 - 2015-11-30 * FIXES #14685 https://bugs.torproject.org/14685

1 0

[bridgedb/develop] Add a CHANGELOG entry for #18949.
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit 936958ae0bb138bf1a85cd311488da0c01fe2a57 Author: Isis Lovecruft <isis(a)torproject.org> Date: Thu Jul 28 16:37:22 2016 +0000 Add a CHANGELOG entry for #18949. --- CHANGELOG | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/CHANGELOG b/CHANGELOG index 036c057..2d7ab93 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,10 +1,17 @@ -Changes in version 0.3.6 - 2016-XXX +Changes in version 0.3.6 - 2016-07-28 + + * FIXES #18237 https://bugs.torproject.org/18237 + During descriptor parsing, BridgeDB saves copies of descriptor files + which couldn't be parsed, for later debugging purposes. To avoid + filing up the runtime directory with these files, we now delete files + older than 24 hours, every 24 hours. + + * FIXES #18949 https://bugs.torproject.org/18949 + Since we've upgraded the host machine which runs The Tor Project's + BridgeDB instance to Debian Jessie, this patch updates the testing + configurations and continuous integration infrastructure to run + tests on versions of Python dependencies in Debian Jessie and Stretch. - * FIXES #18237 https://bugs.torproject.org/18237 - During descriptor parsing, BridgeDB saves copies of descriptor files - which couldn't be parsed, for later debugging purposes. To avoid - filing up the runtime directory with these files, we now delete files - older than 24 hours, every 24 hours. Changes in version 0.3.4 - 0.3.5 - 2015-11-30

1 0

[bridgedb/develop] Fix a typo in a unittest docstring.
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit 0bb6014377e23c29bc774bc5f080c6534440c9bc Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon May 2 16:28:17 2016 +0000 Fix a typo in a unittest docstring. --- test/test_util.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/test_util.py b/test/test_util.py index da4ddf4..f7a2245 100644 --- a/test/test_util.py +++ b/test/test_util.py @@ -72,7 +72,7 @@ class MiscLoggingUtilTests(unittest.TestCase): class LevenshteinDistanceTests(unittest.TestCase): - """Unittests for `bridgedb.util.levenshteinDistance.""" + """Unittests for `bridgedb.util.levenshteinDistance`.""" def test_levenshteinDistance_blank_blank(self): """The Levenshtein Distance between '' and '' should be 0."""

1 0

[bridgedb/develop] Update CI test matrix to test using Debian Stretch versions.
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit edddab56cf064b4080905ba29116574bb892e977 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon May 2 16:15:59 2016 +0000 Update CI test matrix to test using Debian Stretch versions. * FIXES #18949: https://bugs.torproject.org/18949 --- .travis.yml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/.travis.yml b/.travis.yml index b058d71..4717c23 100644 --- a/.travis.yml +++ b/.travis.yml @@ -35,22 +35,22 @@ env: # https://github.com/pypa/virtualenv/issues/459 - PYTHON_EGG_CACHE="${HOME}/.python-eggs-$(echo $RANDOM$PPID$RANDOM | sha256sum | cut -d ' ' -f 1)" matrix: - # Debian Wheezy - - TWISTED_VERSION=13.2.0 PYOPENSSL_VERSION=0.13.1 # Debian Jessie - TWISTED_VERSION=14.0.2 PYOPENSSL_VERSION=0.14 + # Debian Stretch + - TWISTED_VERSION=16.1.1 PYOPENSSL_VERSION=16.0.0 matrix: include: - - python: "2.7" - env: TWISTED_VERSION=15.0.0 PYOPENSSL_VERSION=0.14 - python: "pypy" env: TWISTED_VERSION=14.0.2 PYOPENSSL_VERSION=0.14 + - python: "pypy" + env: TWISTED_VERSION=16.1.1 PYOPENSSL_VERSION=16.0.0 allow_failures: - - python: "2.7" - env: TWISTED_VERSION=15.0.0 PYOPENSSL_VERSION=0.14 - python: "pypy" env: TWISTED_VERSION=14.0.2 PYOPENSSL_VERSION=0.14 + - python: "pypy" + env: TWISTED_VERSION=16.1.1 PYOPENSSL_VERSION=16.0.0 fast_finish: true before_install:

1 0

[bridgedb/develop] Merge branch 'fix/18237' into develop
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit c5dd5cbeff223295e36f9b8d4e4f349893b821ff Merge: 6164c62 743917a Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon May 30 16:10:23 2016 +0000 Merge branch 'fix/18237' into develop CHANGELOG | 8 ++++++++ bridgedb.conf | 2 ++ bridgedb/Main.py | 12 ++++++++++++ bridgedb/runner.py | 29 +++++++++++++++++++++++++++++ bridgedb/util.py | 21 +++++++++++++++++++++ test/test_util.py | 31 +++++++++++++++++++++++++++++++ 6 files changed, 103 insertions(+) diff --cc test/test_util.py index f7a2245,848ce12..ea5d9ee --- a/test/test_util.py +++ b/test/test_util.py @@@ -71,8 -72,38 +72,38 @@@ class MiscLoggingUtilTests(unittest.Tes util.logging.info("BridgeDB's email address: bridges(a)torproject.org") + class FileUtilityTests(unittest.TestCase): + """Unittests for `bridgedb.util.deleteFilesOlderThan`.""" + + def setUp(self): + self._directory = self.id() + self.newfile = os.sep.join([self._directory, "newfile"]) + self.oldfile = os.sep.join([self._directory, "oldfile"]) + self.testfiles = [self.newfile, self.oldfile] + os.mkdir(self._directory) + + now = time.time() + + for fn in self.testfiles: + with open(fn, "w") as fd: + fd.flush() + + # Change the mtime of the "oldfile" to be two days old: + os.utime(self.oldfile, (now, now - (48 * 60 * 60))) + + def test_deleteFilesOlderThan_deletes_old_files(self): + """The function should delete appropriate files.""" + deleted = util.deleteFilesOlderThan(self.testfiles ,24 * 60 * 60) + self.assertIn(self.oldfile, deleted) + + def test_deleteFilesOlderThan_keeps_new_files(self): + """The function should delete appropriate files.""" + deleted = util.deleteFilesOlderThan(self.testfiles ,24 * 60 * 60) + self.assertNotIn(self.newfile, deleted) + + class LevenshteinDistanceTests(unittest.TestCase): - """Unittests for `bridgedb.util.levenshteinDistance.""" + """Unittests for `bridgedb.util.levenshteinDistance`.""" def test_levenshteinDistance_blank_blank(self): """The Levenshtein Distance between '' and '' should be 0."""

1 0

[bridgedb/develop] Revise HTTPS DummyRequest test utility for changes in Twisted>14.0.2.
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit 22817ee11fd7d21cf38ee6050a9bcd3dd2b7849a Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon May 2 16:16:40 2016 +0000 Revise HTTPS DummyRequest test utility for changes in Twisted>14.0.2. --- test/https_helpers.py | 228 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 226 insertions(+), 2 deletions(-) diff --git a/test/https_helpers.py b/test/https_helpers.py index cc1ddfa..14755ec 100644 --- a/test/https_helpers.py +++ b/test/https_helpers.py @@ -13,6 +13,10 @@ import io +from twisted.internet.defer import Deferred +from twisted.internet.address import IPv4Address +from twisted.web.http_headers import Headers +from twisted.web.server import NOT_DONE_YET, Session from twisted.web.test import requesthelper from bridgedb.persistent import Conf @@ -121,15 +125,232 @@ class DummyHTTPSDistributor(object): return [self._bridge_class() for _ in range(self._bridgesPerResponseMin)] -class DummyRequest(requesthelper.DummyRequest): +class RequestHelperDummyRequest(object): + """ + Represents a dummy or fake request. + + Taken from Twisted-14.0.2 because this helper class changed in 16.0.0. + + @ivar _finishedDeferreds: C{None} or a C{list} of L{Deferreds} which will + be called back with C{None} when C{finish} is called or which will be + errbacked if C{processingFailed} is called. + + @type headers: C{dict} + @ivar headers: A mapping of header name to header value for all request + headers. + + @type outgoingHeaders: C{dict} + @ivar outgoingHeaders: A mapping of header name to header value for all + response headers. + + @type responseCode: C{int} + @ivar responseCode: The response code which was passed to + C{setResponseCode}. + + @type written: C{list} of C{bytes} + @ivar written: The bytes which have been written to the request. + """ + uri = b'http://dummy/' + method = b'GET' + client = None + + def registerProducer(self, prod,s): + self.go = 1 + while self.go: + prod.resumeProducing() + + def unregisterProducer(self): + self.go = 0 + + + def __init__(self, postpath, session=None): + self.sitepath = [] + self.written = [] + self.finished = 0 + self.postpath = postpath + self.prepath = [] + self.session = None + self.protoSession = session or Session(0, self) + self.args = {} + self.outgoingHeaders = {} + self.requestHeaders = Headers() + self.responseHeaders = Headers() + self.responseCode = None + self.headers = {} + self._finishedDeferreds = [] + self._serverName = b"dummy" + self.clientproto = b"HTTP/1.0" + + def getHeader(self, name): + """ + Retrieve the value of a request header. + + @type name: C{bytes} + @param name: The name of the request header for which to retrieve the + value. Header names are compared case-insensitively. + + @rtype: C{bytes} or L{NoneType} + @return: The value of the specified request header. + """ + return self.headers.get(name.lower(), None) + + + def getAllHeaders(self): + """ + Retrieve all the values of the request headers as a dictionary. + + @return: The entire C{headers} L{dict}. + """ + return self.headers + + + def setHeader(self, name, value): + """TODO: make this assert on write() if the header is content-length + """ + self.outgoingHeaders[name.lower()] = value + + def getSession(self): + if self.session: + return self.session + assert not self.written, "Session cannot be requested after data has been written." + self.session = self.protoSession + return self.session + + + def render(self, resource): + """ + Render the given resource as a response to this request. + + This implementation only handles a few of the most common behaviors of + resources. It can handle a render method that returns a string or + C{NOT_DONE_YET}. It doesn't know anything about the semantics of + request methods (eg HEAD) nor how to set any particular headers. + Basically, it's largely broken, but sufficient for some tests at least. + It should B{not} be expanded to do all the same stuff L{Request} does. + Instead, L{DummyRequest} should be phased out and L{Request} (or some + other real code factored in a different way) used. + """ + result = resource.render(self) + if result is NOT_DONE_YET: + return + self.write(result) + self.finish() + + + def write(self, data): + if not isinstance(data, bytes): + raise TypeError("write() only accepts bytes") + self.written.append(data) + + def notifyFinish(self): + """ + Return a L{Deferred} which is called back with C{None} when the request + is finished. This will probably only work if you haven't called + C{finish} yet. + """ + finished = Deferred() + self._finishedDeferreds.append(finished) + return finished + + + def finish(self): + """ + Record that the request is finished and callback and L{Deferred}s + waiting for notification of this. + """ + self.finished = self.finished + 1 + if self._finishedDeferreds is not None: + observers = self._finishedDeferreds + self._finishedDeferreds = None + for obs in observers: + obs.callback(None) + + + def processingFailed(self, reason): + """ + Errback and L{Deferreds} waiting for finish notification. + """ + if self._finishedDeferreds is not None: + observers = self._finishedDeferreds + self._finishedDeferreds = None + for obs in observers: + obs.errback(reason) + + + def addArg(self, name, value): + self.args[name] = [value] + + + def setResponseCode(self, code, message=None): + """ + Set the HTTP status response code, but takes care that this is called + before any data is written. + """ + assert not self.written, "Response code cannot be set after data has been written: %s." % "@@@@".join(self.written) + self.responseCode = code + self.responseMessage = message + + + def setLastModified(self, when): + assert not self.written, "Last-Modified cannot be set after data has been written: %s." % "@@@@".join(self.written) + + + def setETag(self, tag): + assert not self.written, "ETag cannot be set after data has been written: %s." % "@@@@".join(self.written) + + + def getClientIP(self): + """ + Return the IPv4 address of the client which made this request, if there + is one, otherwise C{None}. + """ + if isinstance(self.client, IPv4Address): + return self.client.host + return None + + + def getRequestHostname(self): + """ + Get a dummy hostname associated to the HTTP request. + + @rtype: C{bytes} + @returns: a dummy hostname + """ + return self._serverName + + + def getHost(self): + """ + Get a dummy transport's host. + + @rtype: C{IPv4Address} + @returns: a dummy transport's host + """ + return IPv4Address('TCP', '127.0.0.1', 80) + + + def getClient(self): + """ + Stub to get the client doing the HTTP request. + This merely just ensures that this method exists here. Feel free to + extend it. + """ + + +class DummyRequest(RequestHelperDummyRequest): """Wrapper for :api:`twisted.test.requesthelper.DummyRequest` to add redirect support. """ def __init__(self, *args, **kwargs): - requesthelper.DummyRequest.__init__(self, *args, **kwargs) + RequestHelperDummyRequest.__init__(self, *args, **kwargs) self.redirect = self._redirect(self) self.content = io.StringIO() + self.headers = {} # Needed for Twisted>14.0.2 + #self.outgoingHeaders = {} + #self.responseHeaders = Headers() + #self.requestHeaders = Headers() + def writeContent(self, data): """Add some **data** to the faked body of this request. @@ -159,3 +380,6 @@ class DummyRequest(requesthelper.DummyRequest): newRequest = type(request) newRequest.uri = request.uri return newRequest + + +#DummyRequest = RequestHelperDummyRequest

1 0

[bridgedb/develop] Fix an Emacs file encoding specification in a unittest file.
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit dec3b6e7ee9ab0034815150b4940e5a10f4b0190 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon May 2 16:19:44 2016 +0000 Fix an Emacs file encoding specification in a unittest file. --- test/test_https_server.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/test_https_server.py b/test/test_https_server.py index ce388a1..40855fa 100644 --- a/test/test_https_server.py +++ b/test/test_https_server.py @@ -1,4 +1,4 @@ -# -*- encoding: utf-8 -*- +# -*- coding: utf-8 -*- #_____________________________________________________________________________ # # This file is part of BridgeDB, a Tor bridge distribution system.

1 0

[bridgedb/develop] Merge branch 'release/0.3.6'
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit 908294fde8cb818de29f73e8355411148dfba8f9 Merge: 7d6defe 936958a Author: Isis Lovecruft <isis(a)torproject.org> Date: Thu Jul 28 16:38:07 2016 +0000 Merge branch 'release/0.3.6' .travis.yml | 12 +-- CHANGELOG | 17 +++- bridgedb.conf | 2 + bridgedb/Main.py | 12 +++ bridgedb/runner.py | 29 ++++++ bridgedb/util.py | 21 +++++ requirements.txt | 2 +- test/https_helpers.py | 228 +++++++++++++++++++++++++++++++++++++++++++++- test/test_https_server.py | 2 +- test/test_txrecaptcha.py | 6 ++ test/test_util.py | 33 ++++++- 11 files changed, 352 insertions(+), 12 deletions(-)

1 0

[bridgedb/develop] Only run txrecaptcha.submit() unittest on CI infrastructure.
by isis＠torproject.org 28 Jul '16

28 Jul '16

commit ad647e38847cf88f6b4f99951d2f79cbe4919c16 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon May 2 16:28:00 2016 +0000 Only run txrecaptcha.submit() unittest on CI infrastructure. --- test/test_txrecaptcha.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/test/test_txrecaptcha.py b/test/test_txrecaptcha.py index 4704d49..9e4f5f6 100644 --- a/test/test_txrecaptcha.py +++ b/test/test_txrecaptcha.py @@ -10,6 +10,7 @@ """Unittests for the bridgedb.txrecaptcha module.""" import logging +import os from twisted.internet import defer from twisted.internet import reactor @@ -19,6 +20,7 @@ from twisted.internet.error import ConnectionLost from twisted.internet.error import ConnectionRefusedError from twisted.test import proto_helpers from twisted.trial import unittest +from twisted.trial.unittest import SkipTest from twisted.python import failure from twisted.web.client import ResponseDone from twisted.web.http_headers import Headers @@ -208,6 +210,10 @@ class SubmitTests(unittest.TestCase): :func:`txrecaptcha.submit` should be a :class:`txcaptcha.RecaptchaResponse`. """ + if not os.environ.get("CI"): + raise SkipTest(("This test requires network so it is only run " + "on CI servers.")) + def checkResponse(response): """Check that the response is a :class:`txcaptcha.RecaptchaResponse`.

1 0