December 2017 - tor-commits - lists.torproject.org

[bridgedb/develop] Add CHANGELOG entries for 0.6.2.
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit 1096bf4d9ed1cbded7ce8ad2a183338f969b1166 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 23:22:31 2017 +0000 Add CHANGELOG entries for 0.6.2. --- CHANGELOG | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/CHANGELOG b/CHANGELOG index effc405..82cf769 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,3 +1,23 @@ +Changes in version 0.6.2 - 2017-12-20 + + * FIXES #24636 https://bugs.torproject.org/24636 + The moat API specification included an extra response type which + could be sent if there was no overlap between transports the + client supported and those which the server supported. This has + been removed from the specification, which now describes the + behaviour moat has always exhibited: if there is no overlap, the + server responds with a CAPTCHA image response which includes the + list of transports it does support. + + * FIXES #24637 https://bugs.torproject.org/24637 + The moat server did not respond correctly with the specified JSON + API error type when there were no bridges available. It now + responds correctly with a 404 error whose details describe why the + request could not be fulfilled. The moat server also now logs + messages if there were not the configured + MOAT_BRIDGES_PER_RESPONSE number of bridges available. + + Changes in version 0.6.1 - 2017-12-13 * ADDS a shell script, scripts/test-moat, for testing either a

1 0

[translation/torbutton-torbuttonproperties] Update translations for torbutton-torbuttonproperties
by translation＠torproject.org 20 Dec '17

20 Dec '17

commit deca88701484ffb97af454b1c1b5f09cab5544d7 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Dec 20 23:17:39 2017 +0000 Update translations for torbutton-torbuttonproperties --- mk/torbutton.properties | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/mk/torbutton.properties b/mk/torbutton.properties index d436164dc..e572bf31a 100644 --- a/mk/torbutton.properties +++ b/mk/torbutton.properties @@ -5,7 +5,7 @@ torbutton.circuit_display.this_browser = Овој прелистувач torbutton.circuit_display.relay = јазол torbutton.circuit_display.tor_bridge = Мост / Јазол torbutton.circuit_display.unknown_country = Непозната држава -torbutton.content_sizer.margin_tooltip = Tor Browser adds this margin to make the width and height of your window less distinctive, and thus reduces the ability of people to track you online. +torbutton.content_sizer.margin_tooltip = Тор прелистувачот ја додава ова маргина за ширината и висината на вашиот прозорец да биде постандарден, со што се намалува можноста луѓето да ве следат онлајн. torbutton.panel.tooltip.disabled = Кликнете за да го овозможите Тор torbutton.panel.tooltip.enabled = Кликнете за да го оневозможите Тор torbutton.panel.label.disabled = Тор е оневозможен @@ -28,10 +28,10 @@ torbutton.popup.confirm_plugins = Приклучоците како Flash мож torbutton.popup.never_ask_again = Не ме прашувај повторно torbutton.popup.confirm_newnym = Тор прелистувачот ќе ги затвори сите прозорци и табови. Сите веб сесии ќе бидат изгубени.\n\nПовторно стартувајте го Тор прелистувачот сега да го промените вашиот идентитет.\n\n -torbutton.slider_notification = The green onion menu now has a security slider which lets you adjust your security level. Check it out! -torbutton.slider_notification_button = Open security settings +torbutton.slider_notification = Зеленото кромид мени сега има безбедносен лизгач кој ви дозволува да го прилагодите вашето ниво на безбедност. Погледнете го! +torbutton.slider_notification_button = Отвори ги безбедносните поставки -torbutton.maximize_warning = Maximizing Tor Browser can allow websites to determine your monitor size, which can be used to track you. We recommend that you leave Tor Browser windows in their original default size. +torbutton.maximize_warning = Максимизирањето на Тор прелистувачот дозволува на веб страните да ја утврдат големината на вашиот монитор, што пак може да се користи за ваше следење. Ви препорачуваме да го оставите прозорецот на Тор прелистувачот својата оригинална големина. # Canvas permission prompt. Strings are kept here for ease of translation. canvas.siteprompt=Оваа веб страна (%S) пробува да извлече HTML5 податоци, преку кои може уникатно да се идентифукува вашиот компјутер.\n\nТреба ли Тор прелистувачот да дозволи оваа веб страна да ги извлече HTML5 податоците? @@ -46,6 +46,6 @@ canvas.neverAccessKey=e # LOCALIZATION NOTE: %S is the application name. profileProblemTitle=%S Профил проблем profileReadOnly=Не можете да го стартувате %S од систем без запишувачки привилегии. Копирајте %S на друга локација пред проба за користење. -profileReadOnlyMac=You cannot run %S from a read-only file system. Please copy %S to your Desktop or Applications folder before trying to use it. -profileAccessDenied=%S does not have permission to access the profile. Please adjust your file system permissions and try again. -profileMigrationFailed=Migration of your existing %S profile failed.\nNew settings will be used. +profileReadOnlyMac=Не можете да го стартувате %S од систем без запишувачки привилегии. Копирајте го %S на Работната површина или во Апликативната папка / Program Files. +profileAccessDenied=%S нема дозвола да пристапи на профилот. Прилагодете ги системските привилегии на датотеката и обидете се повторно. +profileMigrationFailed=Миграцијата на вашиот постоечки %S профил не успеа.\nНови поставки ќе бидат користени.

1 0

[bridgedb/develop] Add a human-readable __str__() method to BridgeRequest classes.
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit f37c22c6284f13fb9f26f78926f0d115b468fb71 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 21:38:12 2017 +0000 Add a human-readable __str__() method to BridgeRequest classes. --- bridgedb/bridgerequest.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/bridgedb/bridgerequest.py b/bridgedb/bridgerequest.py index e555e0a..d5af05f 100644 --- a/bridgedb/bridgerequest.py +++ b/bridgedb/bridgerequest.py @@ -117,6 +117,15 @@ class BridgeRequestBase(object): self.client = 'default' self.valid = False + def __str__(self): + """Return a human-readable string describing this bridge request.""" + return "%s(ipVersion=%d, transports=%s, notBlockedIn=%s, valid=%s)" % \ + (self.__class__.__name__, + self.ipVersion, + self.transports, + self.notBlockedIn, + self.valid) + @property def ipVersion(self): """The IP version of bridge addresses to distribute to the client.

1 0

[bridgedb/master] Send correct X-Forwarded-For headers in test-moat script.
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit a1597da01eb30e7222dc153a81cd7575a889f8ff Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon Nov 27 19:30:03 2017 +0000 Send correct X-Forwarded-For headers in test-moat script. --- bridgedb/bridgerequest.py | 2 +- scripts/test-moat | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/bridgedb/bridgerequest.py b/bridgedb/bridgerequest.py index 560b704..e555e0a 100644 --- a/bridgedb/bridgerequest.py +++ b/bridgedb/bridgerequest.py @@ -149,7 +149,7 @@ class BridgeRequestBase(object): mod the number of nodes in the hashring. """ if not client: - client = self.client + client = type('')(self.client) # Get an HMAC with the key of the client identifier: digest = getHMACFunc(key)(client) diff --git a/scripts/test-moat b/scripts/test-moat index 490edc6..13b835e 100755 --- a/scripts/test-moat +++ b/scripts/test-moat @@ -15,6 +15,7 @@ fi function do_fetch() { curl -H 'Content-Type: application/vnd.api+json' \ -H 'Accept: application/vnd.api+json' \ + -H 'X-Forwarded-For: 1.2.3.4' \ --data '{"data": [{"supported": ["obfs4"], "version": "0.1.0", "type": "client-transports"}]}' \ http://127.0.0.1:6790/meek/moat/fetch echo @@ -23,6 +24,7 @@ function do_fetch() { function do_check() { curl -H 'Content-Type: application/vnd.api+json' \ -H 'Accept: application/vnd.api+json' \ + -H 'X-Forwarded-For: 1.2.3.4' \ --data '{"data": [{"challenge": "'$CHALLENGE'", "solution": "'$SOLUTION'", "version": "0.1.0", "qrcode": "false", "type": "moat-solution", "id": 2, "transport": "obfs4"}]}' \ http://127.0.0.1:6790/meek/moat/check echo

1 0

[bridgedb/master] Fix failing unittest after #24433 merge.
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit a316083e1a3a9eedcc6aa5d27c053f4974f8cf39 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Nov 28 02:33:05 2017 +0000 Fix failing unittest after #24433 merge. --- bridgedb/test/test_distributors_moat_server.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bridgedb/test/test_distributors_moat_server.py b/bridgedb/test/test_distributors_moat_server.py index 2145541..95796d1 100644 --- a/bridgedb/test/test_distributors_moat_server.py +++ b/bridgedb/test/test_distributors_moat_server.py @@ -882,7 +882,7 @@ class CaptchaCheckResourceTests(unittest.TestCase): 'transport': 'obfs4', 'challenge': challenge, 'solution': self.solution, - 'qrcode': True, + 'qrcode': 'true', }] } encoded_data = json.dumps(data)

1 0

[bridgedb/master] Fix error in usage of redirectTo() with a malicious request.
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit 77776deaad380d9a48524c69533bf566a7a1406b Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Nov 28 19:43:10 2017 +0000 Fix error in usage of redirectTo() with a malicious request. * FIXES #24460: https://bugs.torproject.org/24460 --- bridgedb/distributors/https/server.py | 6 ++++-- bridgedb/test/test_https_server.py | 14 ++++++++++++++ 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/bridgedb/distributors/https/server.py b/bridgedb/distributors/https/server.py index 352a838..d4771a6 100644 --- a/bridgedb/distributors/https/server.py +++ b/bridgedb/distributors/https/server.py @@ -410,8 +410,10 @@ class CaptchaProtectedResource(CustomErrorHandlingResource, CSPResource): try: challenge = request.args['captcha_challenge_field'][0] response = request.args['captcha_response_field'][0] - except Exception: # pragma: no cover - return redirectTo(request.URLPath(), request) + except Exception as error: + logging.debug(("Client CAPTCHA solution to HTTPS distributor server" + "didn't include correct HTTP arguments: %s" % error)) + return redirectTo(type(b'')(request.URLPath()), request) return (challenge, response) def checkSolution(self, request): diff --git a/bridgedb/test/test_https_server.py b/bridgedb/test/test_https_server.py index dbd177f..13ec20e 100644 --- a/bridgedb/test/test_https_server.py +++ b/bridgedb/test/test_https_server.py @@ -372,6 +372,20 @@ class GimpCaptchaProtectedResourceTests(unittest.TestCase): self.assertEqual(challenge, expectedChallenge) self.assertEqual(response, expectedResponse) + def test_extractClientSolution_missing_arguments(self): + """A solution with missing arguments (the solution field) should + return a very agressive redirect to the originally requested, + CAPTCHA-protected page. + """ + expectedChallenge = '23232323232323232323' + + self.request.method = b'POST' + self.request.addArg('captcha_challenge_field', expectedChallenge) + + response = self.captchaResource.extractClientSolution(self.request) + + self.assertIn("click here", response) + def test_checkSolution(self): """checkSolution() should return False is the solution is invalid.""" expectedChallenge = '23232323232323232323'

1 0

[bridgedb/develop] Change moat API specification to remove "moat-transports" response.
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit da84c257cc600a1aeb6bafb70bd3fc0878ed212f Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 22:51:38 2017 +0000 Change moat API specification to remove "moat-transports" response. * FIXES #24636: https://bugs.torproject.org/24636 --- README.rst | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/README.rst b/README.rst index 8481382..e2d17a5 100644 --- a/README.rst +++ b/README.rst @@ -452,17 +452,6 @@ Receiving a CAPTCHA challenge The moat server will respond with ``200 OK``. -If there is no overlap with the transports which BridgeDB supports, the moat -server will respond with the list of transports which is *does* support:: - - { - "data": { - "version": "0.1.0", - "type": "moat-transports", - "supported": [ "TRANSPORT", "TRANSPORT", ... ], - } - } - If there is an overlap with what BridgeDB supports, the moat server will select the "best" transport from the list of supported transports, and respond with the following JSON containing a CAPTCHA challenge:: @@ -490,6 +479,20 @@ The challenge contains an encrypted-then-HMACed timestamp, and solutions submitted more than 30 minutes after requesting the CAPTCHA are considered invalid. +If there is no overlap with the transports which BridgeDB supports, the moat +server will respond with the list of transports which is *does* support:: + + { + "data": { + "id": "1", + "type": "moat-challenge", + "version": "0.1.0", + "transport": [ "TRANSPORT", "TRANSPORT", ... ], + "image": "CAPTCHA", + "challenge": "CHALLENGE", + } + } + Responding to a CAPTCHA challenge """""""""""""""""""""""""""""""""

1 0

[bridgedb/develop] Implement moat error handling when no bridges are available.
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit 4c799735b5b86beea486ee3b1cd0ea8a4f5cb4b5 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 22:36:06 2017 +0000 Implement moat error handling when no bridges are available. * FIXES #24637: https://bugs.torproject.org/24637 --- bridgedb/distributors/moat/server.py | 73 +++++++++++++++++++++----- bridgedb/test/test_distributors_moat_server.py | 55 +++++++++++++++++-- 2 files changed, 112 insertions(+), 16 deletions(-) diff --git a/bridgedb/distributors/moat/server.py b/bridgedb/distributors/moat/server.py index eb356fe..a669ee5 100644 --- a/bridgedb/distributors/moat/server.py +++ b/bridgedb/distributors/moat/server.py @@ -209,6 +209,7 @@ class JsonAPIErrorResource(JsonAPIResource): resource403 = JsonAPIErrorResource(code=403, status="Forbidden") +resource404 = JsonAPIErrorResource(code=404, status="Not Found") resource406 = JsonAPIErrorResource(code=406, status="Not Acceptable") resource415 = JsonAPIErrorResource(code=415, status="Unsupported Media Type") resource419 = JsonAPIErrorResource(code=419, status="No You're A Teapot") @@ -497,21 +498,21 @@ class CaptchaCheckResource(CaptchaResource): self.nBridgesToGive = N self.useForwardedHeader = useForwardedHeader - def getBridgeLines(self, ip, data): - """Get bridge lines for a client's HTTP request. + def createBridgeRequest(self, ip, data): + """Create an appropriate :class:`MoatBridgeRequest` from the ``data`` + of a client's request. :param str ip: The client's IP address. :param dict data: The decoded JSON API data from the client's request. - :rtype: list or None - :returns: A list of bridge lines. + :rtype: :class:`MoatBridgeRequest` + :returns: An object which specifies the filters for retreiving + the type of bridges that the client requested. """ - bridgeLines = None - interval = self.schedule.intervalStart(time.time()) + logging.debug("Creating moat bridge request object for %s." % ip) - logging.debug("Replying to JSON API request from %s." % ip) + bridgeRequest = MoatBridgeRequest() if ip and data: - bridgeRequest = MoatBridgeRequest() bridgeRequest.client = IPAddress(ip) bridgeRequest.isValid(True) bridgeRequest.withIPversion() @@ -519,6 +520,23 @@ class CaptchaCheckResource(CaptchaResource): bridgeRequest.withoutBlockInCountry(data) bridgeRequest.generateFilters() + return bridgeRequest + + def getBridgeLines(self, bridgeRequest): + """Get bridge lines for a client's HTTP request. + + :type bridgeRequest: :class:`MoatBridgeRequest` + :param bridgeRequest: A valid bridge request object with pre-generated + filters (as returned by :meth:`createBridgeRequest`). + :rtype: list + :returns: A list of bridge lines. + """ + bridgeLines = list() + interval = self.schedule.intervalStart(time.time()) + + logging.debug("Replying to JSON API request from %s." % bridgeRequest.client) + + if bridgeRequest.isValid(): bridges = self.distributor.getBridges(bridgeRequest, interval) bridgeLines = [replaceControlChars(bridge.getBridgeLine(bridgeRequest)) for bridge in bridges] @@ -599,17 +617,34 @@ class CaptchaCheckResource(CaptchaResource): return valid - def failureResponse(self, id, request): - """Respond with status code "419 No You're A Teapot".""" - error_response = resource419 - error_response.type = 'moat-bridges' + def failureResponse(self, id, request, bridgeRequest=None): + """Respond with status code "419 No You're A Teapot" if the captcha + verification failed, or status code "404 Not Found" if there + were none of the type of bridges requested. + :param int id: The JSON API "id" field of the + ``JsonAPIErrorResource`` which should be returned. + :type request: :api:`twisted.web.http.Request` + :param request: The current request we're handling. + :type bridgeRequest: :class:`MoatBridgeRequest` + :param bridgeRequest: A valid bridge request object with pre-generated + filters (as returned by :meth:`createBridgeRequest`). + """ if id == 4: + error_response = resource419 error_response.id = 4 error_response.detail = "The CAPTCHA solution was incorrect." elif id == 5: + error_response = resource419 error_response.id = 5 error_response.detail = "The CAPTCHA challenge timed out." + elif id == 6: + error_response = resource404 + error_response.id = 6 + error_response.detail = ("No bridges available to fulfill " + "request: %s.") % bridgeRequest + + error_response.type = 'moat-bridges' return error_response.render(request) @@ -665,7 +700,19 @@ class CaptchaCheckResource(CaptchaResource): if valid: qrcode = None - bridgeLines = self.getBridgeLines(clientIP, client_data) + bridgeRequest = self.createBridgeRequest(clientIP, client_data) + bridgeLines = self.getBridgeLines(bridgeRequest) + + # If we can only return less than the configured + # MOAT_BRIDGES_PER_ANSWER then log a warning. + if len(bridgeLines) < self.nBridgesToGive: + logging.warn(("Not enough bridges of the type specified to " + "fulfill the following request: %s") % bridgeRequest) + + # If we have no bridges at all to give to the client, then + # return a JSON API 404 error. + if not bridgeLines: + return self.failureResponse(6, request) if include_qrcode: qrjpeg = generateQR(bridgeLines) diff --git a/bridgedb/test/test_distributors_moat_server.py b/bridgedb/test/test_distributors_moat_server.py index 95796d1..2f84557 100644 --- a/bridgedb/test/test_distributors_moat_server.py +++ b/bridgedb/test/test_distributors_moat_server.py @@ -581,6 +581,14 @@ class CaptchaFetchResourceTests(unittest.TestCase): self.assert_data_is_ok(decoded) +class MockCaptchaCheckResource(server.CaptchaCheckResource): + """A mocked :class:`server.CaptchaCheckResource` whose + ``getBridgeLines`` method returns no bridges. + """ + def getBridgeLines(self, bridgeRequest): + return list() + + class CaptchaCheckResourceTests(unittest.TestCase): """Tests for :class:`bridgedb.distributors.moat.server.CaptchaCheckResource`.""" @@ -611,6 +619,14 @@ class CaptchaCheckResourceTests(unittest.TestCase): "iz_PdOD2GIGPeclwiHAWM1pOS4cQVsTQR_z4ojZbpLiSp35n4Qbb11YOoreovZzlbS" "7W38rAsTirkdeugcNq82AxKP3phEkyRcw--CzV") + def mock_getBridgeLines(self): + self.resource = MockCaptchaCheckResource(self.distributor, + self.schedule, 3, + self.hmacKey, + self.publicKey, + self.secretKey, + useForwardedHeader=False) + def create_POST_with_data(self, data): request = DummyRequest([self.pagename]) request.requestHeaders.addRawHeader('Content-Type', 'application/vnd.api+json') @@ -826,13 +842,24 @@ class CaptchaCheckResourceTests(unittest.TestCase): self.assertEqual(error['type'], "moat-bridges") self.assertEqual(error['id'], 4) + def test_createBridgeRequest(self): + request = self.create_valid_POST_with_challenge(self.expiredChallenge) + request.client = requesthelper.IPv4Address('TCP', '3.3.3.3', 443) + encoded_content = request.content.read() + content = json.loads(encoded_content)['data'][0] + + bridgeRequest = self.resource.createBridgeRequest('3.3.3.3', content) + + self.assertTrue(bridgeRequest.isValid()) + def test_getBridgeLines(self): request = self.create_valid_POST_with_challenge(self.expiredChallenge) request.client = requesthelper.IPv4Address('TCP', '3.3.3.3', 443) encoded_content = request.content.read() content = json.loads(encoded_content)['data'][0] - bridgelines = self.resource.getBridgeLines('3.3.3.3', content) + bridgeRequest = self.resource.createBridgeRequest('3.3.3.3', content) + bridgelines = self.resource.getBridgeLines(bridgeRequest) self.assertTrue(bridgelines) @@ -840,9 +867,31 @@ class CaptchaCheckResourceTests(unittest.TestCase): request = self.create_valid_POST_with_challenge(self.expiredChallenge) request.client = requesthelper.IPv4Address('TCP', '3.3.3.3', 443) - bridgelines = self.resource.getBridgeLines('3.3.3.3', None) + bridgeRequest = self.resource.createBridgeRequest('3.3.3.3', None) + bridgelines = self.resource.getBridgeLines(bridgeRequest) + + self.assertFalse(bridgeRequest.isValid()) + self.assertEqual(len(bridgelines), 0) + + def test_failureResponse_no_bridges(self): + request = self.create_valid_POST_with_challenge(self.expiredChallenge) + request.client = requesthelper.IPv4Address('TCP', '3.3.3.3', 443) + encoded_content = request.content.read() + content = json.loads(encoded_content)['data'][0] + + bridgeRequest = self.resource.createBridgeRequest('3.3.3.3', content) + + response = self.resource.failureResponse(6, request, bridgeRequest) + + self.assertIn("No bridges available", response) + + def test_render_POST_no_bridges(self): + self.mock_getBridgeLines() + + request = self.create_valid_POST_make_new_challenge() + response = self.resource.render(request) - self.assertIsNone(bridgelines) + self.assertIn("No bridges available", response) def test_render_POST_unexpired(self): request = self.create_valid_POST_make_new_challenge()

1 0

[bridgedb/develop] Merge branch 'fix/24637' into develop
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit 6ace9a350f5f700d9f1f6ff88c6a275c1fb18414 Merge: 91305fa 4c79973 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 22:59:15 2017 +0000 Merge branch 'fix/24637' into develop README.rst | 9 ++-- bridgedb/bridgerequest.py | 9 ++++ bridgedb/distributors/moat/server.py | 73 +++++++++++++++++++++----- bridgedb/test/test_distributors_moat_server.py | 55 +++++++++++++++++-- 4 files changed, 126 insertions(+), 20 deletions(-)

1 0

[bridgedb/develop] Merge branch 'master' into develop
by isis＠torproject.org 20 Dec '17

20 Dec '17

commit 05ca3dbd35a392e8661c7288dc42c99a3443b110 Merge: 6ace9a3 2ae288d Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Dec 20 23:07:24 2017 +0000 Merge branch 'master' into develop

1 0