December 2017 - tor-commits - lists.torproject.org

[bridgedb/master] Merge branch 'fix/24460' into develop
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit 16e7a2d5fad1658244c204bdfb54e5e385e4632f Merge: a316083 77776de Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Nov 28 19:44:42 2017 +0000 Merge branch 'fix/24460' into develop bridgedb/distributors/https/server.py | 6 ++++-- bridgedb/test/test_https_server.py | 14 ++++++++++++++ 2 files changed, 18 insertions(+), 2 deletions(-)

1 0

[bridgedb/master] Merge branch 'master' into develop
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit 9a926f28de7af283558bf2688d5fa642d059c678 Merge: b828d57 9bb5de6 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 13 20:12:17 2017 +0000 Merge branch 'master' into develop bridgedb/distributors/moat/server.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)

1 0

[bridgedb/master] Merge branch 'fix/3015-remove-buckets' into develop
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit e3a2ea593dcc6614b865572d9fc1d3b333b5770d Merge: 16e7a2d f4d5591 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Nov 28 21:22:24 2017 +0000 Merge branch 'fix/3015-remove-buckets' into develop README.rst | 23 +-- bridgedb.conf | 6 - bridgedb/Bridges.py | 18 +-- bridgedb/Bucket.py | 288 ---------------------------------- bridgedb/main.py | 51 +++--- bridgedb/parse/options.py | 29 ++-- bridgedb/runner.py | 14 -- bridgedb/test/test_Bucket.py | 63 -------- bridgedb/test/test_bridgedb_script.py | 19 --- bridgedb/test/test_main.py | 26 +-- doc/sphinx/source/bridgedb.Bucket.rst | 8 - doc/sphinx/source/bridgedb.rst | 1 - doc/sphinx/source/conf.py | 1 - 13 files changed, 41 insertions(+), 506 deletions(-)

1 0

[bridgedb/master] Fix quoting in moat specification.
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit 8d8db4a06d3dff5e9f82adcb4e0e921fe3577098 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 13 18:18:55 2017 +0000 Fix quoting in moat specification. --- README.rst | 74 +++++++++++++++++++++++++++++++------------------------------- 1 file changed, 37 insertions(+), 37 deletions(-) diff --git a/README.rst b/README.rst index c2f32c1..8481382 100644 --- a/README.rst +++ b/README.rst @@ -429,10 +429,10 @@ Requesting Bridges The client MUST send a ``POST /meek/moat/fetch`` containing the following JSON:: { - 'data': { - 'version': '0.1.0', - 'type': 'client-transports', - 'supported': [ 'TRANSPORT', 'TRANSPORT', ... ], + "data": { + "version": "0.1.0", + "type": "client-transports", + "supported": [ "TRANSPORT", "TRANSPORT", ... ], } } @@ -456,10 +456,10 @@ If there is no overlap with the transports which BridgeDB supports, the moat server will respond with the list of transports which is *does* support:: { - 'data': { - 'version': '0.1.0', - 'type': 'moat-transports', - 'supported': [ 'TRANSPORT', 'TRANSPORT', ... ], + "data": { + "version": "0.1.0", + "type": "moat-transports", + "supported": [ "TRANSPORT", "TRANSPORT", ... ], } } @@ -468,13 +468,13 @@ the "best" transport from the list of supported transports, and respond with the following JSON containing a CAPTCHA challenge:: { - 'data': { - 'id': 1, - 'type': 'moat-challenge', - 'version': '0.1.0', - 'transport': TRANSPORT, - 'image': CAPTCHA, - 'challenge': CHALLENGE, + "data": { + "id": "1", + "type": "moat-challenge", + "version": "0.1.0", + "transport": "TRANSPORT", + "image": "CAPTCHA", + "challenge": "CHALLENGE", } } @@ -498,14 +498,14 @@ To propose a solution to a CAPTCHA, the client MUST send a request for ``POST /meek/moat/check``, where the body of the request contains the following JSON:: { - 'data': { - 'id': 2, - 'type': 'moat-solution', - 'version': '0.1.0', - 'transport': TRANSPORT, - 'challenge': CHALLENGE, - 'solution': SOLUTION, - 'qrcode': BOOLEAN, + "data": { + "id": "2", + "type": "moat-solution", + "version": "0.1.0", + "transport": "TRANSPORT", + "challenge": "CHALLENGE", + "solution": "SOLUTION", + "qrcode": "BOOLEAN", } } @@ -518,8 +518,8 @@ where: * ``SOLUTION`` is a valid unicode string, up to 20 bytes in length, containing the client's answer (i.e. what characters the CAPTCHA image displayed). The solution is *not* case-sensitive. -* ``BOOLEAN`` is ``'true'`` if the client wants a qrcode containing the bridge - lines to be generated and returned; ``'false'`` otherwise. +* ``BOOLEAN`` is ``"true"`` if the client wants a qrcode containing the bridge + lines to be generated and returned; ``"false"`` otherwise. Receiving Bridges @@ -532,12 +532,12 @@ If the ``SOLUTION`` was successful for the supplied ``CHALLENGE``, the server responds ``200 OK`` with the following JSON:: { - 'data': { - 'id': 3, - 'type': 'moat-bridges', - 'version': '0.1.0', - 'bridges': [ 'BRIDGE_LINE', ... ], - 'qrcode': QRCODE, + "data": { + "id": "3", + "type": "moat-bridges", + "version": "0.1.0", + "bridges": [ "BRIDGE_LINE", ... ], + "qrcode": "QRCODE", } } @@ -556,12 +556,12 @@ server is unable to distribute the requested Bridges, the server responds ``200 OK`` with the following JSON:: { - 'error': { - 'id': 1, - 'code': '404', - 'status': 'Not Found', - 'title': 'Could not fetch the type of bridges you requested', - 'detail': DETAILS, + "error": { + "id": "1", + "code": "404", + "status": "Not Found", + "title": "Could not fetch the type of bridges you requested", + "detail": "DETAILS", } }

1 0

[bridgedb/master] Change moat API specification to remove "moat-transports" response.
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit da84c257cc600a1aeb6bafb70bd3fc0878ed212f Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 22:51:38 2017 +0000 Change moat API specification to remove "moat-transports" response. * FIXES #24636: https://bugs.torproject.org/24636 --- README.rst | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/README.rst b/README.rst index 8481382..e2d17a5 100644 --- a/README.rst +++ b/README.rst @@ -452,17 +452,6 @@ Receiving a CAPTCHA challenge The moat server will respond with ``200 OK``. -If there is no overlap with the transports which BridgeDB supports, the moat -server will respond with the list of transports which is *does* support:: - - { - "data": { - "version": "0.1.0", - "type": "moat-transports", - "supported": [ "TRANSPORT", "TRANSPORT", ... ], - } - } - If there is an overlap with what BridgeDB supports, the moat server will select the "best" transport from the list of supported transports, and respond with the following JSON containing a CAPTCHA challenge:: @@ -490,6 +479,20 @@ The challenge contains an encrypted-then-HMACed timestamp, and solutions submitted more than 30 minutes after requesting the CAPTCHA are considered invalid. +If there is no overlap with the transports which BridgeDB supports, the moat +server will respond with the list of transports which is *does* support:: + + { + "data": { + "id": "1", + "type": "moat-challenge", + "version": "0.1.0", + "transport": [ "TRANSPORT", "TRANSPORT", ... ], + "image": "CAPTCHA", + "challenge": "CHALLENGE", + } + } + Responding to a CAPTCHA challenge """""""""""""""""""""""""""""""""

1 0

[bridgedb/master] Add CHANGELOG entries for 0.6.1.
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit b828d5777f02dc6f544a0ea821952dc1f06e2d7d Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 13 18:55:50 2017 +0000 Add CHANGELOG entries for 0.6.1. --- CHANGELOG | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/CHANGELOG b/CHANGELOG index 7dbe222..effc405 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,3 +1,56 @@ +Changes in version 0.6.1 - 2017-12-13 + + * ADDS a shell script, scripts/test-moat, for testing either a + locally-running moat server, or a remote one through a meek + tunnel. + Thanks to David Fifield for his work on meek, assistance setting + it up, and providing the first version of this script. + + * FIXES #24433 https://bugs.torproject.org/24433 + The test-moat script wasn't sending an X-Forwarded-For header, + which triggered a bug in the moat server, since the CAPTCHA + solution includes an HMAC based on the client's IP (forwarded + through all the several layers of tunnels/proxies). + + * FIXES #24443 https://bugs.torproject.org/24443 + Due to a difference between how booleans are parsed by Python's + json library and normal Python booleans, the moat server was + generating and returning QRCodes… regardless of whether the remote + client application asked for one. This is now fixed. + + * FIXES #24460 https://bugs.torproject.org/24460 + There was an unhandled error when sending certain (what appears to + be possibly malicious? but in a very strange way) requests to + BridgeDB's HTTPS distributor. The robots making the requests were + attempting to request bridges, but were presenting a CAPTCHA + solution without the correct HTML form field parameters present, + which isn't possible through normal usage of the web interface. + Whoever or whatever is doing this is now going to be endlessly + redirected so that they may forever spiral in their own private + internet hell. I reserve the right come up with a worse fate for + them later, should I get bored. + + * FIXES #3015 https://bugs.torproject.org/3015 + BridgeDB has had a partially-implemented concept of "buckets" + since the age of the dinosaurs: write some of the unallocated + bridges to a file which should (somehow) be manually distributed. + In addition to be unused and untested, there were several issues + with the buckets, the most significant of which were the inability + to request pluggable transports in a bucket and the fact that + buckets were not persistent in any way (e.g. if i request a bucket + of 50 bridges for Gomez and another with 50 for Morticia, they + might end up with some of the same bridges, further, tomorrow + they'll end up with 50 possibly different bridges than those they + received today). All of this code is now removed. + +And includes the following general changes: + + * FIXES issues with JSON quote syntax and a mistaken JSON API + "type" parameter in the specification of the moat server (in the + README). + Thanks to Mark Smith and Kathy Brade for pointing out the issues. + + Changes in version 0.6.0 - 2017-11-15 * ADDS a new JSON API distributor called "moat", which is intended

1 0

[bridgedb/master] Merge branch 'release/0.6.2'
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit 2ae288d6f7e3051b3aa3c85e9a8341c3e21baf4b Merge: 9bb5de6 6ace9a3 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 23:07:12 2017 +0000 Merge branch 'release/0.6.2' README.rst | 34 ++++++------ bridgedb/bridgerequest.py | 9 ++++ bridgedb/distributors/moat/server.py | 73 +++++++++++++++++++++----- bridgedb/test/test_distributors_moat_server.py | 55 +++++++++++++++++-- 4 files changed, 140 insertions(+), 31 deletions(-)

1 0

[bridgedb/master] Clarify the specified moat error if there are no bridges.
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit b0898951e8c1b9a4b3d615e9d9422d5c4ce67576 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 21:36:18 2017 +0000 Clarify the specified moat error if there are no bridges. --- README.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/README.rst b/README.rst index 8481382..7e4f6d1 100644 --- a/README.rst +++ b/README.rst @@ -556,13 +556,14 @@ server is unable to distribute the requested Bridges, the server responds ``200 OK`` with the following JSON:: { - "error": { - "id": "1", + "errors": [{ + "id": "6", + "type": "moat-bridges", + "version": "0.1.0", "code": "404", "status": "Not Found", - "title": "Could not fetch the type of bridges you requested", "detail": "DETAILS", - } + }] } where:

1 0

[bridgedb/master] Change "client-transport" moat resource type to "moat-transports".
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit 50a9321fadcfd0e9a4f30bc1919c1bf46d27d8e1 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 13 18:24:04 2017 +0000 Change "client-transport" moat resource type to "moat-transports". --- bridgedb/distributors/moat/server.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/bridgedb/distributors/moat/server.py b/bridgedb/distributors/moat/server.py index eb356fe..096b8c0 100644 --- a/bridgedb/distributors/moat/server.py +++ b/bridgedb/distributors/moat/server.py @@ -386,12 +386,12 @@ class CaptchaFetchResource(CaptchaResource): :param str request: A JSON blob containing the following fields: * "version": The moat protocol version. - * "type": "client-transports". - * "supported": ['TRANSPORT', … ] + * "type": "moat-transports". + * "supported": ["TRANSPORT", … ] where: - * TRANSPORT is a string identifying a transport, e.g. "obfs3" or - "obfs4". Currently supported transport identifiers are: - "vanilla", "fte", "obfs3", "obfs4", "scramblesuit". + * ``TRANSPORT`` is a string identifying a transport, e.g. + "obfs3" or "obfs4". Currently supported transport identifiers + are: "vanilla", "fte", "obfs3", "obfs4", "scramblesuit". :rtype: list :returns: The list of transports the client supports. """ @@ -401,10 +401,10 @@ class CaptchaFetchResource(CaptchaResource): encoded_data = request.content.read() data = json.loads(encoded_data)["data"][0] - if data["type"] != "client-transports": + if data["type"] != "moat-transports": raise ValueError( "Bad JSON API object type: expected %s got %s" % - ('client-transports', data["type"])) + ('moat-transports', data["type"])) elif data["version"] != MOAT_API_VERSION: raise ValueError( "Client requested protocol version %s, but we're using %s" %

1 0

[bridgedb/master] Merge branch 'release/0.6.1'
by isis＠torproject.org 21 Dec '17

21 Dec '17

commit 9bb5de6b07d33bef6ca7db22a70398018523b6e4 Merge: 659b4ed 6aaa2ed Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 13 20:12:08 2017 +0000 Merge branch 'release/0.6.1' CHANGELOG | 53 +++++ README.rst | 97 ++++----- bridgedb.conf | 6 - bridgedb/Bridges.py | 18 +- bridgedb/Bucket.py | 288 ------------------------- bridgedb/bridgerequest.py | 2 +- bridgedb/distributors/https/server.py | 6 +- bridgedb/distributors/moat/server.py | 2 +- bridgedb/main.py | 51 ++--- bridgedb/parse/options.py | 29 ++- bridgedb/runner.py | 14 -- bridgedb/test/test_Bucket.py | 63 ------ bridgedb/test/test_bridgedb_script.py | 19 -- bridgedb/test/test_distributors_moat_server.py | 2 +- bridgedb/test/test_https_server.py | 14 ++ bridgedb/test/test_main.py | 26 +-- doc/sphinx/source/bridgedb.Bucket.rst | 8 - doc/sphinx/source/bridgedb.rst | 1 - doc/sphinx/source/conf.py | 1 - scripts/test-moat | 56 +++++ 20 files changed, 208 insertions(+), 548 deletions(-)

1 0