tor-commits May 2015

tor-commits@lists.torproject.org

22 participants
990 discussions

[bridgedb/master] Merge branch 'master' into develop
by isis＠torproject.org 01 May '15

01 May '15

commit 5912b3ca830b238e56d6e2a78fe359024379596d Merge: 9ec5a96 3a1aa0d Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Mar 25 19:02:12 2015 +0000 Merge branch 'master' into develop

1 0

[bridgedb/master] Fix bug in Levenshtein fuzzy matching for EMAIL_BLACKLIST.
by isis＠torproject.org 01 May '15

01 May '15

commit b5b3d3c8e7038b45e87b91e97cae1d2658b939c7 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Mar 25 21:49:41 2015 +0000 Fix bug in Levenshtein fuzzy matching for EMAIL_BLACKLIST. See https://bugs.torproject.org/9385#comment:14. --- lib/bridgedb/email/autoresponder.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/bridgedb/email/autoresponder.py b/lib/bridgedb/email/autoresponder.py index 80ad83d..aba92ac 100644 --- a/lib/bridgedb/email/autoresponder.py +++ b/lib/bridgedb/email/autoresponder.py @@ -672,8 +672,7 @@ class SMTPAutoresponder(smtp.SMTPClient): # :func:`~bridgedb.util.levenshteinDistance`): if self.incoming.context.fuzzyMatch != 0: for blacklistedAddress in self.incoming.context.blacklist: - distance = levenshteinDistance(self.incoming.canonicalFromEmail, - blacklistedAddress) + distance = levenshteinDistance(str(client), blacklistedAddress) if distance <= self.incoming.context.fuzzyMatch: logging.info("Fuzzy-matched %s to blacklisted address %s!" % (self.incoming.canonicalFromEmail,

1 0

[bridgedb/master] Merge branch 'hotfix/0.3.1-9385-levenshtein-distance' into develop
by isis＠torproject.org 01 May '15

01 May '15

commit 0fd8810b1e57e6ae7a16f9c3672693442444641d Merge: 5912b3c b5b3d3c Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Mar 25 21:51:46 2015 +0000 Merge branch 'hotfix/0.3.1-9385-levenshtein-distance' into develop lib/bridgedb/email/autoresponder.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)

1 0

[bridgedb/master] Log when canonicalFromSMTP doesn't match canonicalFromEmail.
by isis＠torproject.org 01 May '15

01 May '15

commit 43302d0946a13678b4c61300f3646e9f24f03d38 Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Mar 25 21:54:45 2015 +0000 Log when canonicalFromSMTP doesn't match canonicalFromEmail. * ADDS more logging, in order to debug #12627. https://bugs.torproject.org/12627 --- lib/bridgedb/email/autoresponder.py | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/lib/bridgedb/email/autoresponder.py b/lib/bridgedb/email/autoresponder.py index 4fba40d..0eeb14a 100644 --- a/lib/bridgedb/email/autoresponder.py +++ b/lib/bridgedb/email/autoresponder.py @@ -656,12 +656,12 @@ class SMTPAutoresponder(smtp.SMTPClient): # The canonical domains from the SMTP ``MAIL FROM:`` and the email # ``From:`` header should match: - #if self.incoming.canonicalFromSMTP != self.incoming.canonicalFromEmail: - # logging.error("SMTP/Email canonical domain mismatch!") - # logging.debug("Canonical domain mismatch: %s != %s" - # % (self.incoming.canonicalFromSMTP, - # self.incoming.canonicalFromEmail)) - # return False + if self.incoming.canonicalFromSMTP != self.incoming.canonicalFromEmail: + logging.error("SMTP/Email canonical domain mismatch!") + logging.debug("Canonical domain mismatch: %s != %s" + % (self.incoming.canonicalFromSMTP, + self.incoming.canonicalFromEmail)) + #return False self.incoming.domainRules = self.incoming.context.domainRules.get( self.incoming.canonicalFromEmail, list())

1 0

[bridgedb/master] Straight up reject EMAIL_BLACKLIST addresses without further processing.
by isis＠torproject.org 01 May '15

01 May '15

commit d81b47626be69a3370ae687494ce5782057ecefd Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Mar 25 21:53:45 2015 +0000 Straight up reject EMAIL_BLACKLIST addresses without further processing. --- lib/bridgedb/email/autoresponder.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/bridgedb/email/autoresponder.py b/lib/bridgedb/email/autoresponder.py index aba92ac..4fba40d 100644 --- a/lib/bridgedb/email/autoresponder.py +++ b/lib/bridgedb/email/autoresponder.py @@ -634,6 +634,10 @@ class SMTPAutoresponder(smtp.SMTPClient): self.incoming.canonicalFromEmail = client.domain logging.info("'From:' header contained whitelisted address: %s" % str(client)) + # Straight up reject addresses in the EMAIL_BLACKLIST config option: + elif str(client) in self.incoming.context.blacklist: + logging.info("'From:' header contained blacklisted address: %s") + return False else: logging.debug("Canonicalizing client email domain...") try:

1 0

[bridgedb/master] Make a note in README.rst that we have to install libjpeg.dev.
by isis＠torproject.org 01 May '15

01 May '15

commit 3d3be783be1bc2885e5890dc191ca73c0a4c44eb Author: Isis Lovecruft <isis(a)torproject.org> Date: Wed Mar 25 21:52:44 2015 +0000 Make a note in README.rst that we have to install libjpeg.dev. Otherwise the QRCode generator won't work. --- README.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/README.rst b/README.rst index 3c8c1e3..5715a88 100644 --- a/README.rst +++ b/README.rst @@ -87,6 +87,7 @@ BridgeDB requires the following OS-level dependencies: - libgeoip-dev - geoip-database - `python-setuptools <https://pypi.python.org/pypi/setuptools>`__ +- libjpeg-dev As well as any Python dependencies in the ``requirements.txt`` file.

1 0

[bridgedb/master] Add install-dependencies script and use it for .travis.yml install.
by isis＠torproject.org 01 May '15

01 May '15

commit dc30f7572d8da06377ec8029ccc58f3bde54ea48 Author: Isis Lovecruft <isis(a)torproject.org> Date: Thu Mar 26 05:26:06 2015 +0000 Add install-dependencies script and use it for .travis.yml install. --- .travis.yml | 4 +--- scripts/install-dependencies | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 75f8857..7cda131 100644 --- a/.travis.yml +++ b/.travis.yml @@ -59,9 +59,7 @@ before_install: - chmod -R og-w $PYTHON_EGG_CACHE install: - - sudo apt-get install -qq --no-install-suggests --no-install-recommends build-essential openssl sqlite3 python-dev python-setuptools libgeoip-dev geoip-database - - pip install -q --no-use-wheel -r .travis.requirements.txt - - pip install -q --no-use-wheel Twisted==$TWISTED_VERSION pyOpenSSL==$PYOPENSSL_VERSION + - TWISTED_VERSION=$TWISTED_VERSION PYOPENSSL_VERSION=$PYOPENSSL_VERSION ./scripts/install-dependencies - make install # Start a BridgeDB instance before running the tests: diff --git a/scripts/install-dependencies b/scripts/install-dependencies new file mode 100755 index 0000000..0093700 --- /dev/null +++ b/scripts/install-dependencies @@ -0,0 +1,40 @@ +#!/usr/bin/env bash + +set -x -- + +SUDO= +APT_GET=$(which apt-get) +APT_FLAGS='-q --no-install-suggests --no-install-recommends' +PIP=$(which pip) +PIP_FLAGS='--no-use-wheel' +DEPENDS="build-essential openssl sqlite3 python-dev python-setuptools" +DEPENDS="${DEPENDS} libgeoip-dev geoip-database libjpeg-dev" +HERE=$(dirname $0) + +if [ "$EUID" != "0" ] ; then SUDO=$(which sudo); fi +# pip is pre-installed on Travis-CI machines in their Python environment, so +# we should avoid reinstalling it (as that might do odd things to the +# virtualenv that we're already inside): +if [ "$TRAVIS" == "true" ] ; then + DEPENDS="${DEPENDS} realpath" +else + DEPENDS="${DEPENDS} python-pip" +fi + +MISSING="" +for dep in $DEPENDS ; do + if ! test "$(dpkg -l $dep | grep \"ii $dep\")" ; then + MISSING="${MISSING} $dep" + fi +done + +$SUDO $APT_GET install ${APT_FLAGS} ${MISSING} + +# When running on Travis, be sure that environment variables are passed from +# .travis.yml to this script: +if [ "$TRAVIS" == "true" ] ; then + $PIP install $PIP_FLAGS -r "$HERE/../.travis.requirements.txt" + $PIP install $PIP_FLAGS "Twisted==$TWISTED_VERSION" "pyOpenSSL==$PYOPENSSL_VERSION" +else + $PIP install $PIP_FLAGS -r "$HERE/../requirements.txt" +fi

1 0

[bridgedb/master] Merge branch 'fix/12504-config-pts' into develop
by isis＠torproject.org 01 May '15

01 May '15

commit 25e654384243ae281c843b282f83ac0bf570f6ec Merge: 43302d0 0a6bea1 Author: Isis Lovecruft <isis(a)torproject.org> Date: Thu Mar 26 04:38:10 2015 +0000 Merge branch 'fix/12504-config-pts' into develop bridgedb.conf | 18 +++++++ lib/bridgedb/configure.py | 14 ++++++ lib/bridgedb/email/templates.py | 2 +- lib/bridgedb/strings.py | 90 +++++++++++++++++++++++++++++++---- lib/bridgedb/templates/options.html | 13 +++-- 5 files changed, 121 insertions(+), 16 deletions(-)

1 0

[bridgedb/master] Make which pluggable transports are distributed easily configurable.
by isis＠torproject.org 01 May '15

01 May '15

commit 0a6bea1ddae7e9a3885b2d4486e625c2b3f7f7ac Author: Isis Lovecruft <isis(a)torproject.org> Date: Thu Mar 26 03:03:15 2015 +0000 Make which pluggable transports are distributed easily configurable. * REMOVE bridgedb.strings.CURRENT_TRANSPORTS. * ADD b.strings.SUPPORTED_TRANSPORTS, which can be get/set with b.strings._getSupportedTransports() and _setSupportedTransports(). There is also a new SUPPORTED_TRANSPORTS option in bridgedb.conf; the setting at bridgedb.strings.SUPPORTED_TRANSPORTS is now automatically configured via the bridgedb.conf file. * ADD b.strings.DEFAULT_TRANSPORT, which can be get/set with b.strings._getDefaultTransport() and _setDefaultTransport(). There is also a new DEFAULT_TRANSPORT option in bridgedb.conf; the setting at bridgedb.strings.DEFAULT_TRANSPORT is now automatically configured via the bridgedb.conf file. * CHANGE options.html template to automatically populate the list of available transports. The DEFAULT_TRANSPORT is automatically the selected default. * FIXES #12504: https://bugs.torproject.org/12504 --- bridgedb.conf | 18 +++++++ lib/bridgedb/configure.py | 14 ++++++ lib/bridgedb/email/templates.py | 2 +- lib/bridgedb/strings.py | 90 +++++++++++++++++++++++++++++++---- lib/bridgedb/templates/options.html | 13 +++-- 5 files changed, 121 insertions(+), 16 deletions(-) diff --git a/bridgedb.conf b/bridgedb.conf index e0b25f0..c91cc73 100644 --- a/bridgedb.conf +++ b/bridgedb.conf @@ -20,6 +20,9 @@ # # CHANGELOG: # ~~~~~~~~~~ +# Changed in version 0.0.15 - 2015-03-26 +# * ADD new SUPPORTED_TRANSPORTS and DEFAULT_TRANSPORT settings. +# # Changes in version 0.0.14 - 2015-02-22 # * ADD new OpenPGP-related options: # - EMAIL_GPG_HOMEDIR @@ -237,6 +240,21 @@ TASKS = { 'GET_TOR_EXIT_LIST': 3 * 60 * 60, } +# SUPPORTED_TRANSPORTS is a dictionary mapping Pluggable Transport methodnames +# to booleans. If ``True``, the PT is distributed; if ``False``, it isn't. +SUPPORTED_TRANSPORTS = { + 'obfs2': True, + 'obfs3': True, + 'obfs4': True, + 'scramblesuit': True, + 'fte': True, +} + +# DEFAULT_TRANSPORT is a string. It should be the PT methodname of the +# transport which is selected by default (e.g. in the webserver dropdown +# menu). +DEFAULT_TRANSPORT = 'obfs4' + #------------------------------- # HTTP(S) Distribution Options \ #------------------------------------------------------------------------------ diff --git a/lib/bridgedb/configure.py b/lib/bridgedb/configure.py index 90c5e2f..4fb6cc0 100644 --- a/lib/bridgedb/configure.py +++ b/lib/bridgedb/configure.py @@ -14,6 +14,9 @@ import logging import os +# Used to set the SUPPORTED_TRANSPORTS: +from bridgedb import strings + def loadConfig(configFile=None, configCls=None): """Load configuration settings on top of the current settings. @@ -116,6 +119,17 @@ def loadConfig(configFile=None, configCls=None): setting = getattr(config, attr, []) # Default to empty lists setattr(config, attr, setting) + for attr in ["SUPPORTED_TRANSPORTS"]: + setting = getattr(config, attr, {}) # Default to emtpy dicts + setattr(config, attr, setting) + + # Set the SUPPORTED_TRANSPORTS to populate the webserver and email options: + strings._setSupportedTransports(getattr(config, "SUPPORTED_TRANSPORTS", {})) + strings._setDefaultTransport(getattr(config, "DEFAULT_TRANSPORT", "")) + logging.info("Currently supported transports: %s" % + " ".join(strings._getSupportedTransports())) + logging.info("Default transport: %s" % strings._getDefaultTransport()) + for domain in config.EMAIL_DOMAINS: config.EMAIL_DOMAIN_MAP[domain] = domain diff --git a/lib/bridgedb/email/templates.py b/lib/bridgedb/email/templates.py index e81283e..ae85e8f 100644 --- a/lib/bridgedb/email/templates.py +++ b/lib/bridgedb/email/templates.py @@ -53,7 +53,7 @@ def addCommands(template): # And include the currently supported transports: commands += template.gettext(strings.EMAIL_MISC_TEXT.get(5)) commands += "\n" - for pt in strings.CURRENT_TRANSPORTS: + for pt in strings._getSupportedTransports(): commands += ' ' + pt + "\n" return commands diff --git a/lib/bridgedb/strings.py b/lib/bridgedb/strings.py index 5a9c528..1f38e4b 100644 --- a/lib/bridgedb/strings.py +++ b/lib/bridgedb/strings.py @@ -1,4 +1,4 @@ -# -*- coding: utf-8 -*- +# -*- coding: utf-8 ; test-case-name: bridgedb.test.test_strings ; -*- # # This file is part of BridgeDB, a Tor bridge distribution system. # @@ -10,6 +10,12 @@ from __future__ import unicode_literals +# This won't work on Python2.6, however +# 1) We don't use Python2.6, and +# 2) We don't care about supporting Python2.6, because Python 2.6 (and, +# honestly, all of Python2) should die. +from collections import OrderedDict + def _(text): """This is necessary because strings are translated when they're imported. @@ -166,21 +172,85 @@ EMAIL_COMMANDS = { # All of the following containers are untranslated! #----------------------------------------------------------------------------- -#: A list of all currently available pluggable transports. By "currently -#: available" we mean: +#: SUPPORTED TRANSPORTS is dictionary mapping all Pluggable Transports +#: methodname to whether or not we actively distribute them. The ones which we +#: distribute SHOULD have the following properties: #: #: 1. The PT is in a widely accepted, usable state for most Tor users. #: 2. The PT is currently publicly deployed *en masse*". #: 3. The PT is included within the transports which Tor Browser offers in #: the stable releases. #: -CURRENT_TRANSPORTS = [ - "obfs2", - "obfs3", - "obfs4", - "scramblesuit", - "fte", -] +#: These will be sorted by methodname in alphabetical order. +#: +#: ***Don't change this setting here; change it in :file:`bridgedb.conf`.*** +SUPPORTED_TRANSPORTS = {} + +#: DEFAULT_TRANSPORT is a string. It should be the PT methodname of the +#: transport which is selected by default (e.g. in the webserver dropdown +#: menu). +#: +#: ***Don't change this setting here; change it in :file:`bridgedb.conf`.*** +DEFAULT_TRANSPORT = '' + +def _getSupportedTransports(): + """Get the list of currently supported transports. + + :rtype: list + :returns: A list of strings, one for each supported Pluggable Transport + methodname, sorted in alphabetical order. + """ + supported = [name.lower() for name,w00t in SUPPORTED_TRANSPORTS.items() if w00t] + supported.sort() + return supported + +def _setDefaultTransport(transport): + global DEFAULT_TRANSPORT + DEFAULT_TRANSPORT = transport + +def _getDefaultTransport(): + return DEFAULT_TRANSPORT + +def _setSupportedTransports(transports): + """Set the list of currently supported transports. + + .. note: You shouldn't need to touch this. This is used by the config file + parser. You should change the SUPPORTED_TRANSPORTS dictionary in + :file:`bridgedb.conf`. + + :param dict transports: A mapping of Pluggable Transport methodnames + (strings) to booleans. If the boolean is ``True``, then the Pluggable + Transport is one which we will (more easily) distribute to clients. + If ``False``, then we (sort of) don't distribute it. + """ + global SUPPORTED_TRANSPORTS + SUPPORTED_TRANSPORTS = transports + +def _getSupportedAndDefaultTransports(): + """Get a dictionary of currently supported transports, along with a boolean + marking which transport is the default. + + It is returned as a :class:`collections.OrderedDict`, because if it is a + regular dict, then the dropdown menu would populated in random order each + time the page is rendered. It is sorted in alphabetical order. + + :rtype: :class:`collections.OrderedDict` + :returns: An :class:`~collections.OrderedDict` of the Pluggable Transport + methodnames from :data:`SUPPORTED_TRANSPORTS` whose value in + ``SUPPORTED_TRANSPORTS`` is ``True``. If :data:`DEFAULT_TRANSPORT` is + set, then the PT methodname in the ``DEFAULT_TRANSPORT`` setting is + added to the :class:`~collections.OrderedDict`, with the value + ``True``. Every other transport in the returned ``OrderedDict`` has + its value set to ``False``, so that only the one which should be the + default PT is ``True``. + """ + supported = _getSupportedTransports() + transports = OrderedDict(zip(supported, [False for _ in range(len(supported))])) + + if DEFAULT_TRANSPORT: + transports[DEFAULT_TRANSPORT] = True + + return transports EMAIL_SPRINTF = { # Goes into the "%s types of Pluggable Transports %s" part of ``WELCOME[0]`` diff --git a/lib/bridgedb/templates/options.html b/lib/bridgedb/templates/options.html index e765ce7..9486199 100644 --- a/lib/bridgedb/templates/options.html +++ b/lib/bridgedb/templates/options.html @@ -87,11 +87,14 @@ accesskey="t"> ${_("""No""")} <option label="none" value="0" >${_("none")}</option> -<option label="obfs2" value="obfs2" >obfs2</option> -<option label="obfs3" value="obfs3" selected >obfs3</option> -<option label="obfs4" value="obfs4" >obfs4</option> -<option label="scramblesuit" value="scramblesuit">scramblesuit</option> -<option label="fte" value="fte" >fteproxy</option> +% for methodname, default in strings._getSupportedAndDefaultTransports().items(): + <option label=${methodname} + value=${methodname} + % if default: + selected + % endif + > ${methodname} </option> +% endfor </select> </div> </div>

1 0

[bridgedb/master] Fix duplicated unittest.TestCase method name in test_bridges.
by isis＠torproject.org 01 May '15

01 May '15

commit f913ab9f911ba9f79c7d25c35a88870f2882b755 Author: Isis Lovecruft <isis(a)torproject.org> Date: Thu Mar 26 22:43:36 2015 +0000 Fix duplicated unittest.TestCase method name in test_bridges. --- lib/bridgedb/test/test_bridges.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bridgedb/test/test_bridges.py b/lib/bridgedb/test/test_bridges.py index 4388437..907cc28 100644 --- a/lib/bridgedb/test/test_bridges.py +++ b/lib/bridgedb/test/test_bridges.py @@ -692,7 +692,7 @@ class BridgeBackwardsCompatibilityTests(unittest.TestCase): self.assertTrue(bridge.running) self.assertTrue(bridge.flags.running) - def test_BridgeBackwardsCompatibility_setStatus_running(self): + def test_BridgeBackwardsCompatibility_setStatus_running_stable(self): """Using setStatus() to set the Running and Stable flags should set Bridge.running, Bridge.flags.running, Bridge.stable, and Bridge.flags.stable.

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits May 2015