February 2014 - tor-commits - lists.torproject.org

[bridgedb/develop] Don't safelog randomly generated IPs.
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit bf852f9e95162f42c02f50554853e3075bedbd9f Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Jan 25 17:38:31 2014 +0000 Don't safelog randomly generated IPs. * REMOVE calls to `Util.safelog` which masked the IP addresses which are randomly generated in `HTTPServer.CaptchaProtectedResource.render_POST()`. These random IPs are sent to the ReCaptcha server in order to not reveal the client's real IP, and they are no more revealing/identifying of a particular client than the `twisted.web.http.Request.args` which are logged. --- lib/bridgedb/HTTPServer.py | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/lib/bridgedb/HTTPServer.py b/lib/bridgedb/HTTPServer.py index 32399b3..eac1fb9 100644 --- a/lib/bridgedb/HTTPServer.py +++ b/lib/bridgedb/HTTPServer.py @@ -114,13 +114,14 @@ class CaptchaProtectedResource(twisted.web.resource.Resource): recaptcha_response = captcha.submit(challenge, response, self.recaptchaPrivKey, remote_ip) if recaptcha_response.is_valid: - logging.info("Valid recaptcha from %s. Parameters were %r", - Util.logSafely(remote_ip), request.args) + logging.info("Valid recaptcha from %s. Parameters were %r" + % (remote_ip, request.args)) return self.resource.render(request) else: - logging.info("Invalid recaptcha from %s. Parameters were %r", - Util.logSafely(remote_ip), request.args) - logging.info("Recaptcha error code: %s", recaptcha_response.error_code) + logging.info("Invalid recaptcha from %s. Parameters were %r" + % (remote_ip, request.args)) + logging.info("Recaptcha error code: %r" + % recaptcha_response.error_code) return redirectTo(request.URLPath(), request) class WebResource(twisted.web.resource.Resource):

1 0

[bridgedb/develop] Preserve HTTP POST args across redirects.
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit 458e490881e96069642ceaffe9f2ce8c5f251cd8 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Jan 26 04:26:06 2014 +0000 Preserve HTTP POST args across redirects. * FIXES #10737, an issue where if a CAPTCHA was entered incorrectly, the client would be redirected back to /bridges.html without the original HTTP POST arguments (which specify the requested pluggable transports, IP version, etc.). --- lib/bridgedb/HTTPServer.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/bridgedb/HTTPServer.py b/lib/bridgedb/HTTPServer.py index 5ce8b72..ddb1812 100644 --- a/lib/bridgedb/HTTPServer.py +++ b/lib/bridgedb/HTTPServer.py @@ -145,7 +145,10 @@ class CaptchaProtectedResource(twisted.web.resource.Resource): % (remote_ip, request.args)) logging.info("Recaptcha error code: %r" % recaptcha_response.error_code) - return redirectTo(request.URLPath(), request) + + logging.debug("Client failed a recaptcha; returning redirect to %s" + % request.uri) + return redirectTo(request.uri, request) class WebResource(twisted.web.resource.Resource): """This resource is used by Twisted Web to give a web page with some

1 0

[bridgedb/develop] Catch error case in HTTPServer where ReCaptcha image is None.
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit c47420c63b010ed66d286c8e1bb5665424aec729 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Jan 26 02:47:23 2014 +0000 Catch error case in HTTPServer where ReCaptcha image is None. --- lib/bridgedb/HTTPServer.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/lib/bridgedb/HTTPServer.py b/lib/bridgedb/HTTPServer.py index 26bc5ad..9b4c515 100644 --- a/lib/bridgedb/HTTPServer.py +++ b/lib/bridgedb/HTTPServer.py @@ -96,6 +96,15 @@ class CaptchaProtectedResource(twisted.web.resource.Resource): except Exception as error: logging.fatal("Connection to Recaptcha server failed: %s" % error) + if c.image is None: + # TODO: We should have a general "Something went wrong!" page + # which displays here, rather than displaying the img alt text + # (which says "Upgrade your browser to Firefox"), so that users + # don't think the problem is on their end when it's actually a + # problem with ReCaptcha or BridgeDB. + logging.warn("No CAPTCHA image received from ReCaptcha server!") + c.image = '' + # TODO: this does not work for versions of IE < 8.0 imgstr = 'data:image/jpeg;base64,%s' % base64.b64encode(c.image) template = lookup.get_template('captcha.html')

1 0

[bridgedb/develop] Fix indentation level in method docstring.
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit 2bd2c85ef5fee1381a469839498c4d8384d78067 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sun Jan 26 05:42:05 2014 +0000 Fix indentation level in method docstring. --- lib/bridgedb/HTTPServer.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bridgedb/HTTPServer.py b/lib/bridgedb/HTTPServer.py index ddb1812..3bffe81 100644 --- a/lib/bridgedb/HTTPServer.py +++ b/lib/bridgedb/HTTPServer.py @@ -112,7 +112,7 @@ class CaptchaProtectedResource(twisted.web.resource.Resource): return rendered def render_POST(self, request): - """Process a client CAPTCHA by sending it to the ReCaptcha server. + """Process a client CAPTCHA by sending it to the ReCaptcha server. The client's IP address is not sent to the ReCaptcha server; instead, a completely random IP is generated and sent instead.

1 0

[bridgedb/develop] Merge branch 'fix/6127-simple-error-page' into develop
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit 8ec29d7a7f09c979de79faa15302d25e7e95fe13 Merge: b8e307f 7e21184 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Jan 28 17:24:12 2014 +0000 Merge branch 'fix/6127-simple-error-page' into develop lib/bridgedb/HTTPServer.py | 87 +++++++++++++++++++++++++++++++++----------- 1 file changed, 66 insertions(+), 21 deletions(-)

1 0

[bridgedb/develop] Merge branch 'fix/10737-post-args-redirects' into develop
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit b8e307f7189a64b1db01888655a81b16e3c67ba3 Merge: b52786f ca1544c Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Jan 28 16:53:07 2014 +0000 Merge branch 'fix/10737-post-args-redirects' into develop lib/bridgedb/HTTPServer.py | 58 ++++++++++++++++++++++++++++++++------------ 1 file changed, 42 insertions(+), 16 deletions(-)

1 0

[bridgedb/develop] Add Sphinx docstring to CaptchaProtectedResource.render_GET().
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit 16d084e4392f9f845452c58dfaa9138ff4c3bd28 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Jan 28 17:24:51 2014 +0000 Add Sphinx docstring to CaptchaProtectedResource.render_GET(). --- lib/bridgedb/HTTPServer.py | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/lib/bridgedb/HTTPServer.py b/lib/bridgedb/HTTPServer.py index 5794be7..ddac918 100644 --- a/lib/bridgedb/HTTPServer.py +++ b/lib/bridgedb/HTTPServer.py @@ -123,8 +123,16 @@ class CaptchaProtectedResource(twisted.web.resource.Resource): return ip def render_GET(self, request): - # get a captcha + """Retrieve a ReCaptcha from the API server and serve it to the client. + + :type request: :api:`twisted.web.http.Request` + :param request: A ``Request`` object for 'bridges.html'. + :rtype: str + :returns: A rendered HTML page containing a ReCaptcha challenge image + for the client to solve. + """ c = Raptcha(self.recaptchaPubKey, self.recaptchaPrivKey) + try: c.get() except Exception as error:

1 0

[bridgedb/develop] Implement a simple error page to replace all webserver tracebacks.
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit 7e21184d4432b4c25f775eb27aa8b9de68d650b5 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Jan 28 17:01:57 2014 +0000 Implement a simple error page to replace all webserver tracebacks. * ADD HTTPServer.replaceErrorPage() function for catching templating errors and sending them to the logger. The user is served a very simply "Somthing went wrong page". * CHANGE all calls to mako.templates.Template.render() to be wrapped in a try/except block which directs to the new HTTPServer.replaceErrorPage() function. --- lib/bridgedb/HTTPServer.py | 87 +++++++++++++++++++++++++++++++++----------- 1 file changed, 66 insertions(+), 21 deletions(-) diff --git a/lib/bridgedb/HTTPServer.py b/lib/bridgedb/HTTPServer.py index 104e924..5794be7 100644 --- a/lib/bridgedb/HTTPServer.py +++ b/lib/bridgedb/HTTPServer.py @@ -34,6 +34,7 @@ from bridgedb.Filters import filterBridgesByNotBlockedIn from bridgedb.parse import headers from ipaddr import IPv4Address, IPv6Address from random import randint +import mako.exceptions from mako.template import Template from mako.lookup import TemplateLookup from zope.interface import Interface, Attribute, implements @@ -66,6 +67,39 @@ else: logging.info("GeoIP database loaded") +def replaceErrorPage(error, template_name=None): + """Create a general error page for displaying in place of tracebacks. + + Log the error to BridgeDB's logger, and then display a very plain "Sorry! + Something went wrong!" page to the client. + + :type error: :exc:`Exception` + :param error: Any exeption which has occurred while attempting to retrieve + a template, render a page, or retrieve a resource. + :param str template_name: A string describing which template/page/resource + was being used when the exception occurred, + i.e. ``'index.html'``. + :returns: A string containing HTML to serve to the client (rather than + serving a traceback). + """ + logging.error("Error while attempting to render %s: %s" + % (template_name or 'template', + mako.exceptions.text_error_template().render())) + + errmsg = _("Sorry! Something went wrong with your request.") + rendered = """<html> + <head> + <link href="/assets/bootstrap.min.css" rel="stylesheet"> + <link href="/assets/custom.css" rel="stylesheet"> + </head> + <body> + <p>{0}</p> + </body> + </html>""".format(errmsg) + + return rendered + + class CaptchaProtectedResource(twisted.web.resource.Resource): def __init__(self, useRecaptcha=False, recaptchaPrivKey='', recaptchaPubKey='', useForwardedHeader=False, resource=None): @@ -97,18 +131,17 @@ class CaptchaProtectedResource(twisted.web.resource.Resource): logging.fatal("Connection to Recaptcha server failed: %s" % error) if c.image is None: - # TODO: We should have a general "Something went wrong!" page - # which displays here, rather than displaying the img alt text - # (which says "Upgrade your browser to Firefox"), so that users - # don't think the problem is on their end when it's actually a - # problem with ReCaptcha or BridgeDB. logging.warn("No CAPTCHA image received from ReCaptcha server!") - c.image = '' - # TODO: this does not work for versions of IE < 8.0 - imgstr = 'data:image/jpeg;base64,%s' % base64.b64encode(c.image) - template = lookup.get_template('captcha.html') - rendered = template.render(imgstr=imgstr, challenge_field=c.challenge) + try: + # TODO: this does not work for versions of IE < 8.0 + imgstr = 'data:image/jpeg;base64,%s' % base64.b64encode(c.image) + template = lookup.get_template('captcha.html') + rendered = template.render(imgstr=imgstr, + challenge_field=c.challenge) + except Exception as err: + rendered = replaceErrorPage(err, 'captcha.html') + return rendered def render_POST(self, request): @@ -135,21 +168,27 @@ class CaptchaProtectedResource(twisted.web.resource.Resource): randint(1,255),randint(1,255)) recaptcha_response = captcha.submit(challenge, response, - self.recaptchaPrivKey, remote_ip) + self.recaptchaPrivKey, remote_ip) + logging.debug("Captcha from client with masked IP %r. Parameters:\n%r" + % (remote_ip, request.args)) + if recaptcha_response.is_valid: - logging.info("Valid recaptcha from %s. Parameters were %r" - % (remote_ip, request.args)) - return self.resource.render(request) + logging.info("Valid recaptcha from client with masked IP %r." + % remote_ip) + try: + rendered = self.resource.render(request) + except Exception as err: + rendered = replaceErrorPage(err) + return rendered else: - logging.info("Invalid recaptcha from %s. Parameters were %r" - % (remote_ip, request.args)) - logging.info("Recaptcha error code: %r" - % recaptcha_response.error_code) + logging.info("Invalid recaptcha from client with masked IP %r: %r" + % (remote_ip, recaptcha_response.error_code)) logging.debug("Client failed a recaptcha; returning redirect to %s" % request.uri) return redirectTo(request.uri, request) + class WebResource(twisted.web.resource.Resource): """This resource is used by Twisted Web to give a web page with some bridges in response to a request.""" @@ -312,11 +351,17 @@ class WebResource(twisted.web.resource.Resource): """ if format == 'plain': request.setHeader("Content-Type", "text/plain") - return answer + rendered = bridgeLines else: request.setHeader("Content-Type", "text/html; charset=utf-8") - return lookup.get_template('bridges.html').render(answer=bridgeLines, - rtl=rtl) + try: + template = lookup.get_template('bridges.html') + rendered = template.render(answer=bridgeLines, rtl=rtl) + except Exception as err: + rendered = replaceErrorPage(err) + + return rendered + class WebRoot(twisted.web.resource.Resource): """The parent resource of all other documents hosted by the webserver."""

1 0

[bridgedb/develop] Change Mako template lookups to ignore atime checks.
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit 3c2100f4be0952b1e1e9fcfc0bfab7b90acf1549 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Jan 28 16:16:53 2014 +0000 Change Mako template lookups to ignore atime checks. * CHANGE Mako template lookups to ignore atime filesystem checks, using only compiled template and not reloading from file. --- lib/bridgedb/HTTPServer.py | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/lib/bridgedb/HTTPServer.py b/lib/bridgedb/HTTPServer.py index 65702ba..2668e31 100644 --- a/lib/bridgedb/HTTPServer.py +++ b/lib/bridgedb/HTTPServer.py @@ -40,10 +40,18 @@ from mako.lookup import TemplateLookup from zope.interface import Interface, Attribute, implements template_root = os.path.join(os.path.dirname(__file__),'templates') -lookup = TemplateLookup(directories=[template_root], - output_encoding='utf-8') rtl_langs = ('ar', 'he', 'fa', 'gu_IN', 'ku') +# Setting `filesystem_checks` to False is recommended for production servers, +# due to potential speed increases. This means that the atimes of the Mako +# template files aren't rechecked every time the template is requested +# (otherwise, if they are checked, and the atime is newer, the template is +# recompiled). See: +# http://docs.makotemplates.org/en/latest/usage.html#setting-filesystem-checks +lookup = TemplateLookup(directories=[template_root], + output_encoding='utf-8', + filesystem_checks=False) + logging.debug("Set template root to %s" % template_root) try:

1 0

[bridgedb/develop] Merge branch 'fix/mako-template-caching' into develop
by isis＠torproject.org 01 Feb '14

01 Feb '14

commit dd681babf7505b6cbcab7d3e76b6feebcc2accc3 Merge: 6b585fc b6be171 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Jan 28 17:46:51 2014 +0000 Merge branch 'fix/mako-template-caching' into develop lib/bridgedb/HTTPServer.py | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)

1 0