July 2015 - tor-commits - lists.torproject.org

[tor/master] changes file for bug16631
by nickm＠torproject.org 21 Jul '15

21 Jul '15

commit 6e1eaa936a8cd42da24fe7a8b1f698402f3714a7 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jul 21 13:59:25 2015 -0400 changes file for bug16631 --- changes/bug16631 | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/changes/bug16631 b/changes/bug16631 new file mode 100644 index 0000000..24c5747 --- /dev/null +++ b/changes/bug16631 @@ -0,0 +1,4 @@ + o Minor bugfixes (relay): + - Fix a rarely-encountered memory leak when failing to initialize + the thread pool. Fixes bug 16631; bugfix on 0.2.6.3-alpha. Patch + from "cypherpunks".

1 0

[webwml/master] add our latest funder
by arma＠torproject.org 21 Jul '15

21 Jul '15

commit 9cf698ab151ef9a0b39682b27e82d959d03ed177 Author: Roger Dingledine <arma(a)torproject.org> Date: Tue Jul 21 02:54:59 2015 -0400 add our latest funder (they're subsidizing some of the upcoming dev meeting) --- about/en/sponsors.wml | 1 + 1 file changed, 1 insertion(+) diff --git a/about/en/sponsors.wml b/about/en/sponsors.wml index 0c2cfdd..90e3f4f 100644 --- a/about/en/sponsors.wml +++ b/about/en/sponsors.wml @@ -30,6 +30,7 @@ <li><a href="http://www.rfa.org/english/">Radio Free Asia</a> (2012-2016)</li> <li><a href="http://nsf.gov/">National Science Foundation</a> joint with Georgia Tech and Princeton University (2012-2016)</li> <li><a href="http://nsf.gov/">National Science Foundation</a> via University of Minnesota (2013-2017)</li> + <li><a href="http://www.auswaertiges-amt.de/">Federal Foreign Office of Germany</a> (2015)</li> <li><a href="https://digitaldefenders.org/">Hivos/The Digital Defenders Partnership</a> (2014-2015)</li> <li><a href="http://www.sri.com/">SRI International</a> (2011-2015)</li> <li><a href="http://www.state.gov/j/drl/">US Department of State Bureau of Democracy, Human Rights, and Labor</a> (2013-2016)</li>

1 0

[bridgedb/develop] Tiny documentation fixes for how to use the Sphinx traclinks extension.
by isis＠torproject.org 21 Jul '15

21 Jul '15

commit c134d3eeddd5f1ae99c8b09fd63891c15f7cc3b9 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon Jul 20 21:23:30 2015 +0000 Tiny documentation fixes for how to use the Sphinx traclinks extension. --- bridgedb/parse/descriptors.py | 2 +- doc/sphinx/ext/traclinks.py | 9 +++++---- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/bridgedb/parse/descriptors.py b/bridgedb/parse/descriptors.py index 1b0108e..2986fd3 100644 --- a/bridgedb/parse/descriptors.py +++ b/bridgedb/parse/descriptors.py @@ -92,7 +92,7 @@ def parseNetworkStatusFile(filename, validate=True, skipAnnotations=True, descriptorClass=RouterStatusEntryV3): """Parse a file which contains an ``@type bridge-networkstatus`` document. - See :trac:`#12254` for why networkstatus-bridges documents don't look + See :trac:`12254` for why networkstatus-bridges documents don't look anything like the networkstatus v2 documents that they are purported to look like. They are missing all headers, and the entire footer including authority signatures. diff --git a/doc/sphinx/ext/traclinks.py b/doc/sphinx/ext/traclinks.py index fd2d9b3..75abce4 100644 --- a/doc/sphinx/ext/traclinks.py +++ b/doc/sphinx/ext/traclinks.py @@ -3,13 +3,14 @@ RestructuredText interpreted text role that looks like this:: :trac:`trac_link_text` -for example:: +For example:: - :trac:`#2015` + :trac:`2015` -creates a link to ticket number 2015. +would create a link to ticket number #2015 (e.g. the link URI would be +https://bugs.torproject.org/2015). -adapted from recipe here_. +Adapted from recipe here_. .. _here: http://stackoverflow.com/a/2111327/13564 """

1 0

[bridgedb/develop] Merge branch 'hotfix/0.3.2-remove-lib-dir' into develop
by isis＠torproject.org 21 Jul '15

21 Jul '15

commit 16ff46eab29dede9cedccf6c385be5a69617eced Merge: c134d3e 0a26984 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon Jul 20 21:32:26 2015 +0000 Merge branch 'hotfix/0.3.2-remove-lib-dir' into develop doc/sphinx/source/conf.py | 1 - 1 file changed, 1 deletion(-)

1 0

[bridgedb/develop] Add unittests for HSDir flags in bridge networkstatus documents.
by isis＠torproject.org 21 Jul '15

21 Jul '15

commit 5f3910b02dfa19a70cb8ba0cee7bbc3a2c82b8ed Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon Jul 20 19:58:18 2015 +0000 Add unittests for HSDir flags in bridge networkstatus documents. * FIXES #16616: https://bugs.torproject.org/16616 --- test/test_bridges.py | 19 +++++++++++++++++++ test/test_parse_descriptors.py | 24 ++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/test/test_bridges.py b/test/test_bridges.py index 9448e3f..3c5fe0d 100644 --- a/test/test_bridges.py +++ b/test/test_bridges.py @@ -403,6 +403,25 @@ class FlagsTests(unittest.TestCase): self.assertFalse(self.flags.fast) self.assertTrue(self.flags.stable) + def test_update_HSDir(self): + """Test adding the HSDir flag with the update() method. + + See also: :trac:`16616`. + """ + self.flags.update(["Fast", "Stable", "HSDir"]) + self.assertTrue(self.flags.fast) + self.assertTrue(self.flags.stable) + # We don't care about the HSDir flag: + self.assertIsNone(getattr(self.flags, "hsdir", None)) + + def test_update_Unicorn(self): + """Test adding a completely made-up flag, "Unicorn", with the update() + method. (It shouldn't get added.) + """ + self.flags.update(["Unicorn"]) + # We don't care about the make-believe Unicorn flag: + self.assertIsNone(getattr(self.flags, "unicorn", None)) + class BridgeAddressBaseTests(unittest.TestCase): """Tests for :class:`bridgedb.bridges.BridgeAddressBase`.""" diff --git a/test/test_parse_descriptors.py b/test/test_parse_descriptors.py index 5104ccd..bd3c3d5 100644 --- a/test/test_parse_descriptors.py +++ b/test/test_parse_descriptors.py @@ -391,6 +391,30 @@ class ParseDescriptorsTests(unittest.TestCase): descriptors.parseNetworkStatusFile, descFile) + def test_parse_descriptors_parseNetworkStatusFile_HSDir_flag(self): + """A Bridge networkstatus descriptor with the HSDir flag should be + possible to parse (without errors), however, the flag should be ignored + (since the :class:`bridgedb.bridges.Flags` class doesn't care about it). + + See also: :trac:`16616` + """ + unparseable = BRIDGE_NETWORKSTATUS_0.replace( + 's Fast Guard Running Stable Valid', + 's Fast Guard Running Stable Valid HSDir') + # Write the descriptor to a file for testing. This is necessary + # because the function opens the networkstatus file to read it. + descFile = self.writeTestDescriptorsToFile('networkstatus-bridges', + unparseable) + routers = descriptors.parseNetworkStatusFile(descFile) + bridge = routers[0] + + for flag in [u'Fast', u'Guard', u'Running', + u'Stable', u'Valid', u'HSDir']: + self.assertTrue(flag in bridge.flags, + ("Expected to parse the %r flag from a bridge " + "networkstatus document, but the flag was not " + "found!")) + def test_parse_descriptors_parseNetworkStatusFile_IPv6_ORAddress(self): """A Bridge can't have its primary ORAddress be IPv6 without raising a ValueError.

1 0

[bridgedb/develop] Remove import of bridgedb.test for Sphinx documentation builds.
by isis＠torproject.org 21 Jul '15

21 Jul '15

commit 0a2698469b678adce9e1057e57a16bfcb68d98e4 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon Jul 20 21:31:58 2015 +0000 Remove import of bridgedb.test for Sphinx documentation builds. --- doc/sphinx/source/conf.py | 1 - 1 file changed, 1 deletion(-) diff --git a/doc/sphinx/source/conf.py b/doc/sphinx/source/conf.py index cab614f..a4dba2c 100644 --- a/doc/sphinx/source/conf.py +++ b/doc/sphinx/source/conf.py @@ -64,7 +64,6 @@ import bridgedb.schedule import bridgedb.Stability import bridgedb.Storage import bridgedb.strings -import bridgedb.test import bridgedb.translations import bridgedb.txrecaptcha import bridgedb.util

1 0

[bridgedb/develop] Merge branch 'fix/16616-hsdir-flag' into develop
by isis＠torproject.org 21 Jul '15

21 Jul '15

commit 5e97d681d201c1d078aa0dbf23475841f1f0eb9d Merge: 40334fa 5f3910b Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Jul 21 04:46:53 2015 +0000 Merge branch 'fix/16616-hsdir-flag' into develop test/test_bridges.py | 19 +++++++++++++++++++ test/test_parse_descriptors.py | 24 ++++++++++++++++++++++++ 2 files changed, 43 insertions(+)

1 0

[bridgedb/develop] Numerous Sphinx documentation fixes.
by isis＠torproject.org 21 Jul '15

21 Jul '15

commit 40334fa3c80844b9e14d80c8f28bfbc4dc328aa2 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Jul 21 04:42:11 2015 +0000 Numerous Sphinx documentation fixes. * ADDS inheritance diagrams. * FIXES links to Stem's documentation. * FIXES problems with cross-referencing :ivar:s. * FIXES a few typos and problems with pluralisation of referenced objects. --- bridgedb/bridgerequest.py | 72 +++++++++--- bridgedb/bridges.py | 178 +++++++++++++++++------------ bridgedb/captcha.py | 98 +++++++++------- bridgedb/configure.py | 38 +++--- bridgedb/crypto.py | 37 +++--- bridgedb/distribute.py | 167 ++++++++++++++------------- bridgedb/email/autoresponder.py | 117 ++++++++++--------- bridgedb/email/distributor.py | 17 ++- bridgedb/email/dkim.py | 1 + bridgedb/email/request.py | 8 +- bridgedb/email/server.py | 25 ++-- bridgedb/filters.py | 69 ++++++----- bridgedb/https/distributor.py | 12 +- bridgedb/https/request.py | 3 + bridgedb/https/server.py | 31 ++--- bridgedb/interfaces.py | 10 +- bridgedb/parse/descriptors.py | 49 ++++---- bridgedb/parse/fingerprint.py | 12 +- bridgedb/proxy.py | 5 +- bridgedb/safelog.py | 63 +++++----- bridgedb/schedule.py | 39 ++++--- bridgedb/txrecaptcha.py | 39 ++++--- doc/sphinx/ext/traclinks.py | 21 +++- doc/sphinx/source/bridgedb.distribute.rst | 8 ++ doc/sphinx/source/bridgedb.rst | 5 +- doc/sphinx/source/conf.py | 42 ++++++- doc/sphinx/source/index.rst | 53 +++++---- 27 files changed, 736 insertions(+), 483 deletions(-) diff --git a/bridgedb/bridgerequest.py b/bridgedb/bridgerequest.py index 1266145..d24948c 100644 --- a/bridgedb/bridgerequest.py +++ b/bridgedb/bridgerequest.py @@ -10,11 +10,17 @@ # :license: see LICENSE for licensing information #_____________________________________________________________________________ +"""API for creating classes which store information on the type of bridges +requested by a client. + +.. inheritance-diagram:: BridgeRequestBase + :parts: 1 +""" import ipaddr import logging -from zope.interface import implements +from zope.interface import implementer from zope.interface import Attribute from zope.interface import Interface @@ -41,9 +47,9 @@ class IRequestBridges(Interface): client = Attribute( "This should be some information unique to the client making the " "request for bridges, such that we are able to HMAC this unique " - "data, via getHashringPlacement(), in order to place the client " - "into a hashring (determining which bridge addresses they get in " - "the request response).") + "data, via :meth:`getHashringPlacement()`, in order to place the " + "client into a hashring (determining which bridge addresses they get " + "in the request response).") def addFilter(): """Add a filter to the list of ``filters``.""" @@ -82,27 +88,33 @@ class IRequestBridges(Interface): """ +@implementer(IRequestBridges) class BridgeRequestBase(object): - """A generic base class for storing options of a client bridge request.""" - - implements(IRequestBridges) + """A generic base class for storing options of a client bridge request. + + :vartype filters: list + :ivar filters: A list of callables used to filter bridges from a hashring. + :vartype transports: list + :ivar transports: A list of strings of Pluggable Transport types requested. + :vartype notBlockedIn: list + :ivar notBlockedIn: A list of two-character country codes. The distributed + bridges should not be blocked in these countries. + :vartype client: str + :ivar client: This should be some information unique to the client making + the request for bridges, such that we are able to HMAC this unique + data in order to place the client into a hashring (determining which + bridge addresses they get in the request response). It defaults to the + string ``'default'``. + :vartype valid: bool + :ivar valid: Should be ``True`` if the client's request was valid. + """ def __init__(self, ipVersion=None): self.ipVersion = ipVersion - #: (list) A list of callables used to filter bridges from a hashring. self.filters = list() - #: (list) A list of strings of Pluggable Transport types requested. self.transports = list() - #: (list) A list of two-character country codes. The distributed bridges - #: should not be blocked in these countries. self.notBlockedIn = list() - #: This should be some information unique to the client making the - #: request for bridges, such that we are able to HMAC this unique data - #: in order to place the client into a hashring (determining which - #: bridge addresses they get in the request response). It defaults to - #: the string ``'default'``. self.client = 'default' - #: (bool) Should be ``True`` if the client's request was valid. self.valid = False @property @@ -160,25 +172,45 @@ class BridgeRequestBase(object): return self.valid def withIPv4(self): + """Set the ``ipVersion`` to IPv4.""" self.ipVersion = 4 def withIPv6(self): + """Set the ``ipVersion`` to IPv6.""" self.ipVersion = 6 def withoutBlockInCountry(self, country): + """Add this **countryCode** to the list of countries which distributed + bridges should not be blocked in (``notBlockedIn``). + """ self.notBlockedIn.append(country.lower()) def withPluggableTransportType(self, pt): + """Add this **pt** to the list of requested ``transports``. + + :param str pt: A :class:`~bridgedb.bridges.PluggableTransport`. + :data:`methodname <bridgedb.bridges.PluggableTransport.methodname>`. + """ self.transports.append(pt) def addFilter(self, filtre): + """Add a **filtre** to the list of ``filters``. + + :type filter: callable + :param filter: A filter function, e.g. one generated via + :mod:`bridgedb.filters`. + """ self.filters.append(filtre) def clearFilters(self): + """Clear the list of ``filters``.""" self.filters = [] def justOnePTType(self): - """Get just one bridge PT type at a time!""" + """Get just one bridge type (e.g. a + :data:`methodname <bridgedb.bridges.PluggableTransport.methodname>` of + :class:`~bridgedb.bridges.PluggableTransport`) at a time! + """ ptType = None try: ptType = self.transports[-1] # Use the last PT requested @@ -187,6 +219,10 @@ class BridgeRequestBase(object): return ptType def generateFilters(self): + """Build the list of callables, ``filters``, according to the current + contents of the lists of ``transports``, ``notBlockedIn``, and the + ``ipVersion``. + """ self.clearFilters() pt = self.justOnePTType() diff --git a/bridgedb/bridges.py b/bridgedb/bridges.py index 747863a..56e55cc 100644 --- a/bridgedb/bridges.py +++ b/bridgedb/bridges.py @@ -7,7 +7,25 @@ # (c) 2007-2015, The Tor Project, Inc. # :license: see LICENSE for licensing information -"""Classes for manipulating and storing Bridges and their attributes.""" +"""Classes for manipulating and storing Bridges and their attributes. + +.. inheritance-diagram:: PluggableTransportUnavailable MalformedBridgeInfo MalformedPluggableTransport InvalidPluggableTransportIP MissingServerDescriptorDigest ServerDescriptorDigestMismatch ServerDescriptorWithoutNetworkstatus InvalidExtraInfoSignature Flags PluggableTransport Bridge + :parts: 1 + +------------ + +**Glossary Terms** + +.. glossary:: + + Bridge Line + A "Bridge Line" is how BridgeDB refers to lines in a ``torrc`` + file which should begin with the word ``"Bridge"``, and it is how + a client tells their Tor process that they would like to use a + particular bridge. + +------------ +""" from __future__ import print_function @@ -306,7 +324,7 @@ class PluggableTransport(BridgeAddressBase): .. _pt-spec.txt: https://gitweb.torproject.org/torspec.git/tree/pt-spec.txt - :type fingerprint: str + :vartype fingerprint: str :ivar fingerprint: The uppercased, hexadecimal fingerprint of the identity key of the parent bridge running this pluggable transport instance, i.e. the main ORPort bridge whose ``@type bridge-server-descriptor`` @@ -314,24 +332,24 @@ class PluggableTransport(BridgeAddressBase): latter of which contains the parameter of this pluggable transport in its ``transport`` line. - :type methodname: str + :vartype methodname: str :ivar methodname: The canonical "name" for this pluggable transport, i.e. the one which would be specified in a torrc file. For example, ``"obfs2"``, ``"obfs3"``, ``"scramblesuit"`` would all be pluggable transport method names. - :type address: ``ipaddr.IPv4Address`` or ``ipaddr.IPv6Address`` + :vartype address: ``ipaddr.IPv4Address`` or ``ipaddr.IPv6Address`` :ivar address: The IP address of the transport. Currently (as of 20 March 2014), there are no known, widely-deployed pluggable transports which support IPv6. Ergo, this is very likely going to be an IPv4 address. - :type port: int + :vartype port: int :ivar port: A integer specifying the port which this pluggable transport is listening on. (This should likely be whatever port the bridge specified in its ``ServerTransportPlugin`` torrc line, unless the pluggable transport is running in "managed" mode.) - :type arguments: dict + :vartype arguments: dict :ivar arguments: Some PTs can take additional arguments, which must be distributed to the client out-of-band. These are present in the ``@type bridge-extrainfo-document``, in the ``transport`` line like @@ -420,14 +438,14 @@ class PluggableTransport(BridgeAddressBase): We currently check that: - 1. The :data:`port` is an integer, and that it is between the values - of ``0`` and ``65535`` (inclusive). + 1. The :data:`port` is an integer, and that it is between the values + of ``0`` and ``65535`` (inclusive). - 2. The :data:`arguments` is a dictionary. + 2. The :data:`arguments` is a dictionary. - 3. The :data:`arguments` do not contain non-ASCII or control - characters or double quotes or backslashes, in keys or - in values. + 3. The :data:`arguments` do not contain non-ASCII or control + characters or double quotes or backslashes, in keys or + in values. :raises MalformedPluggableTransport: if any of the above checks fails. """ @@ -468,8 +486,9 @@ class PluggableTransport(BridgeAddressBase): def _checkArguments(self): """This method is a temporary fix for PTs with missing arguments - (see `#13202 <https://bugs.torproject.org/13202`_) This method can - be removed after Tor-0.2.4.x is deprecated. + (see :trac:`13202`). + + .. todo: This method can be removed after Tor-0.2.4.x is deprecated. """ # obfs4 requires (iat-mode && (cert || (node-id && public-key))): if self.methodname == 'obfs4': @@ -513,22 +532,14 @@ class PluggableTransport(BridgeAddressBase): def getTransportLine(self, includeFingerprint=True, bridgePrefix=False): """Get a Bridge Line for this :class:`PluggableTransport`. - .. glossary:: - - Bridge Line - A "Bridge Line" is how BridgeDB refers to lines in a ``torrc`` - file which should begin with the word ``"Bridge"``, and it is how - a client tells their Tor process that they would like to use a - particular bridge. - .. note:: If **bridgePrefix** is ``False``, this method does not - return lines which are prefixed with the word 'bridge', as they - would be in a torrc file. Instead, lines returned look like this:: + return lines which are prefixed with the word ``'bridge'``, as they + would be in a torrc file. Instead, lines returned look like:: obfs3 245.102.100.252:23619 59ca743e89b508e16b8c7c6d2290efdfd14eea98 This was made configurable to fix Vidalia being a brain-damaged - piece of shit (#5851_). TorLaucher replaced Vidalia soon after, + piece of shit (:trac:`5851`). TorLaucher replaced Vidalia soon after, and TorLauncher is intelligent enough to understand :term:`Bridge Line`s regardless of whether or not they are prefixed with the word "Bridge". @@ -571,7 +582,7 @@ class PluggableTransport(BridgeAddressBase): which Stem uses. Stem's - :api:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor` + :class:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor` parses extrainfo ``transport`` lines into a dictionary with the following structure:: @@ -678,15 +689,16 @@ class BridgeBackwardsCompatibility(BridgeBase): def __init__(self, nickname=None, ip=None, orport=None, fingerprint=None, id_digest=None, or_addresses=None): - """Create a Bridge which is backwards compatible with the old Bridge class - implementation. - - .. info: For backwards compatibility, `nickname`, `ip`, and `orport` - must be the first, second, and third arguments, respectively. The - `fingerprint` and `id_digest` were previously kwargs, and are also - provided for backwards compatibility. New calls to - :meth:`__init__` *should* avoid using these kwargs, and instead - use the methods :meth:`updateFromNetworkStatus`, + """Create a :class:`Bridge <bridgedb.bridges.IBridge>` which is + backwards compatible with the legacy Bridge class implementation. + + .. note: For backwards compatibility, **nickname**, **ip**, and + **orport** must be the first, second, and third arguments, + respectively. The **fingerprint** and **id_digest** were + previously kwargs, and are also provided for backwards + compatibility. New calls to :meth:`__init__` *should* avoid using + these kwargs, and instead use the methods + :meth:`updateFromNetworkStatus`, :meth:`updateFromServerDescriptor`, and :meth:`updateFromExtraInfoDescriptor`. """ @@ -866,50 +878,65 @@ class BridgeBackwardsCompatibility(BridgeBase): class Bridge(BridgeBackwardsCompatibility): """A single bridge, and all the information we have for it. - :type fingerprint: str or ``None`` + :vartype fingerprint: :any:`str` or ``None`` :ivar fingerprint: This ``Bridge``'s fingerprint, in lowercased hexadecimal format. - :type nickname: str or ``None`` + + :vartype nickname: :any:`str` or ``None`` :ivar nickname: This ``Bridge``'s router nickname. - :type socksPort: int + + :vartype socksPort: int :ivar socksPort: This ``Bridge``'s SOCKSPort. Should always be ``0``. - :type dirPort: int + + :vartype dirPort: int :ivar dirPort: This ``Bridge``'s DirPort. Should always be ``0``. - :type orAddresses: list + + :vartype orAddresses: list :ivar orAddresses: A list of 3-tuples in the form:: + (ADDRESS, PORT, IP_VERSION) + where: * ADDRESS is an :class:`ipaddr.IPAddress`, * PORT is an ``int``, * IP_VERSION is either ``4`` or ``6``. - :type transports: list - :ivar transports: A list of :class:`PluggableTransport`s, one for each + + :vartype transports: list + :ivar transports: A list of :class:`PluggableTransport`, one for each transport that this :class:`Bridge` currently supports. - :type flags: :class:`~bridgedb.bridges.Flags` + + :vartype flags: :class:`~bridgedb.bridges.Flags` :ivar flags: All flags assigned by the BridgeAuthority to this :class:`Bridge`. - :type hibernating: bool + + :vartype hibernating: bool :ivar hibernating: ``True`` if this :class:`Bridge` is hibernating and not currently serving clients (e.g. if the Bridge hit its configured ``RelayBandwidthLimit``); ``False`` otherwise. - :type _blockedIn: dict + + :vartype _blockedIn: dict :ivar _blockedIn: A dictionary of ``ADDRESS:PORT`` pairs to lists of lowercased, two-letter country codes (e.g. ``"us"``, ``"gb"``, ``"cn"``, etc.) which that ``ADDRESS:PORT`` pair is blocked in. - :type contact: str or ``None`` + + :vartype contact: :any:`str` or ``None`` :ivar contact: The contact information for the this Bridge's operator. - :type family: set or ``None`` + + :vartype family: :any:`set` or ``None`` :ivar family: The fingerprints of other Bridges related to this one. - :type platform: str or ``None`` + + :vartype platform: :any:`str` or ``None`` :ivar platform: The ``platform`` line taken from the ``@type bridge-server-descriptor``, e.g. ``'Tor 0.2.5.4-alpha on Linux'``. - :type software: :api:`stem.version.Version` or ``None`` + + :vartype software: :class:`stem.version.Version` or ``None`` :ivar software: The OR version portion of the ``platform`` line. - :type os: str or None + + :vartype os: :any:`str` or ``None`` :ivar os: The OS portion of the ``platform`` line. """ - #: (bool) If ``True``, check that the signature of the bridge's + #: (:any:`bool`) If ``True``, check that the signature of the bridge's #: ``@type bridge-server-descriptor`` is valid and that the signature was #: created with the ``signing-key`` contained in that descriptor. _checkServerDescriptorSignature = True @@ -917,12 +944,13 @@ class Bridge(BridgeBackwardsCompatibility): def __init__(self, *args, **kwargs): """Create and store information for a new ``Bridge``. - .. info: For backwards compatibility, `nickname`, `ip`, and `orport` - must be the first, second, and third arguments, respectively. The - `fingerprint` and `id_digest` were previously kwargs, and are also - provided for backwards compatibility. New calls to - :meth:`__init__` *should* avoid using these kwargs, and instead - use the methods :meth:`updateFromNetworkStatus`, + .. note: For backwards compatibility, **nickname**, **ip**, and + **orport** must be the first, second, and third arguments, + respectively. The **fingerprint** and **id_digest** were + previously kwargs, and are also provided for backwards + compatibility. New calls to :meth:`__init__` *should* avoid using + these kwargs, and instead use the methods + :meth:`updateFromNetworkStatus`, :meth:`updateFromServerDescriptor`, and :meth:`updateFromExtraInfoDescriptor`. """ @@ -1245,19 +1273,19 @@ class Bridge(BridgeBackwardsCompatibility): We require that: - 1. Any IP addresses contained in :data:`orAddresses` are valid, - according to :func:`~bridgedb.parse.addr.isValidIP`. + 1. Any IP addresses contained in :data:`orAddresses` are valid, + according to :func:`~bridgedb.parse.addr.isValidIP`. - 2. Any ports in :data:`orAddresses` are between ``1`` and ``65535`` - (inclusive). + 2. Any ports in :data:`orAddresses` are between ``1`` and ``65535`` + (inclusive). - 3. All IP version numbers given in :data:`orAddresses` are either - ``4`` or ``6``. + 3. All IP version numbers given in :data:`orAddresses` are either + ``4`` or ``6``. .. todo:: This should probably be reimplemented as a property that automatically sanitises the values for each ORAddress, as is done - for :property:`bridgedb.bridges.BridgeAddressBase.address` and - :property:`bridgedb.bridges.BridgeBase.orPort`. + for :data:`bridgedb.bridges.BridgeAddressBase.address` and + :data:`bridgedb.bridges.BridgeBase.orPort`. :raises MalformedBridgeInfo: if something was found to be malformed or invalid. @@ -1281,8 +1309,8 @@ class Bridge(BridgeBackwardsCompatibility): Launcher or paste directly into their ``torrc``. This is a helper method to call either :meth:`_getTransportForRequest` - or :meth:`_getVanillaForRequest` depending on whether or not any - :class:`PluggableTransport`s were requested in the + or :meth:`_getVanillaForRequest` depending on whether or not a + :class:`PluggableTransport` was requested in the :class:`bridgeRequest <bridgedb bridgerequest.BridgeRequestBase>`, and then construct the :term:`Bridge Line` accordingly. @@ -1319,7 +1347,7 @@ class Bridge(BridgeBackwardsCompatibility): :param str key: The key to lookup in the :data:`Bridge._blockedIn` dictionary. This should be in the form returned by - :classmethod:`_getBlockKey`. + :meth:`_getBlockKey`. :param str countryCode: A two-character country code specifier. """ if self._blockedIn.has_key(key): @@ -1499,7 +1527,7 @@ class Bridge(BridgeBackwardsCompatibility): document. :type descriptor: - :api:`stem.descriptors.router_status_entry.RouterStatusEntry` + :class:`stem.descriptors.router_status_entry.RouterStatusEntry` :param descriptor: The networkstatus document for this bridge. :param bool ignoreNetworkstatus: If ``True``, then ignore most of the information in the networkstatus document. @@ -1528,7 +1556,7 @@ class Bridge(BridgeBackwardsCompatibility): def updateFromServerDescriptor(self, descriptor, ignoreNetworkstatus=False): """Update this bridge's info from an ``@type bridge-server-descriptor``. - .. info:: + .. note:: If :func:`~bridgedb.parse.descriptor.parseServerDescriptorFile` is called with ``validate=True``, then Stem will handle checking that the ``signing-key`` hashes to the ``fingerprint``. Stem will also @@ -1539,7 +1567,7 @@ class Bridge(BridgeBackwardsCompatibility): actual digest match). :type descriptor: - :api:`stem.descriptor.server_descriptor.RelayDescriptor` + :class:`stem.descriptor.server_descriptor.RelayDescriptor` :param descriptor: The bridge's server descriptor to gather data from. :raises MalformedBridgeInfo: If this Bridge has no corresponding networkstatus entry, or its **descriptor** digest didn't match the @@ -1589,7 +1617,7 @@ class Bridge(BridgeBackwardsCompatibility): ``@type bridge-extrainfo`` descriptor. :type descriptor: - :api:`stem.descriptor.extrainfo_descriptor.RelayExtraInfoDescriptor` + :class:`stem.descriptor.extrainfo_descriptor.RelayExtraInfoDescriptor` :param descriptor: An ``@type bridge-extrainfo`` descriptor for this :class:`Bridge`, parsed with Stem. :raises InvalidExtraInfoSignature: if the signature was invalid, @@ -1672,7 +1700,7 @@ class Bridge(BridgeBackwardsCompatibility): """Update this bridge's information from an extrainfo descriptor. Stem's - :api:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor` + :class:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor` parses extrainfo ``transport`` lines into a dictionary with the following structure:: @@ -1688,12 +1716,12 @@ class Bridge(BridgeBackwardsCompatibility): .. todo:: The ``transport`` attribute of Stem's ``BridgeExtraInfoDescriptor`` class is a dictionary that uses the - Pluggable Transport's eype as the keys. Meaning that if a bridge + Pluggable Transport's type as the keys… meaning that if a bridge were to offer four instances of ``obfs3``, only one of them would get to us through Stem. This might pose a problem someday. :type descriptor: - :api:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor` + :class:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor` :param descriptor: DOCDOC :param bool verify: If ``True``, check that the ``router-signature`` on the extrainfo **descriptor** is a valid signature from diff --git a/bridgedb/captcha.py b/bridgedb/captcha.py index d2fbcc1..bdcd90c 100644 --- a/bridgedb/captcha.py +++ b/bridgedb/captcha.py @@ -15,11 +15,14 @@ """This module implements various methods for obtaining or creating CAPTCHAs. +.. inheritance-diagram:: CaptchaExpired CaptchaKeyError GimpCaptchaError Captcha ReCaptcha GimpCaptcha + :parts: 1 + **Module Overview:** :: - captcha + bridgedb.captcha |- CaptchaExpired - Raised if a solution is given for a stale CAPTCHA. |- CaptchaKeyError - Raised if a CAPTCHA system's keys are invalid/missing. |- GimpCaptchaError - Raised when a Gimp CAPTCHA can't be retrieved. @@ -101,12 +104,16 @@ class ICaptcha(Interface): class Captcha(object): """A generic CAPTCHA base class. - :ivar str image: The CAPTCHA image. - :ivar str challenge: A challenge string which should permit checking of + :vartype image: str + :ivar image: The CAPTCHA image. + :vartype challenge: str + :ivar challenge: A challenge string which should permit checking of the client's CAPTCHA solution in some manner. In stateless protocols such as HTTP, this should be passed along to the client with the CAPTCHA image. + :vartype publicKey: str :ivar publicKey: A public key used for encrypting CAPTCHA challenge strings. + :vartype secretKey: str :ivar secretKey: A private key used for decrypting challenge strings during CAPTCHA solution verification. """ @@ -122,8 +129,9 @@ class Captcha(object): def get(self): """Retrieve a new CAPTCHA image and its associated challenge string. - The image and challenge will be stored as :ivar:`image` and - :ivar:`challenge, respectively. + The image and challenge will be stored as + :attr:`image <bridgedb.captcha.Captcha.image>` and + :attr:`challenge <bridgedb.captcha.Captcha.challenge>`, respectively. """ self.image = None self.challenge = None @@ -132,13 +140,17 @@ class Captcha(object): class ReCaptcha(Captcha): """A CAPTCHA obtained from a remote reCaptcha_ API server. - :ivar str image: The CAPTCHA image. - :ivar str challenge: The ``'recaptcha_challenge_response'`` HTTP form + :vartype image: str + :ivar image: The CAPTCHA image. + :vartype challenge: str + :ivar challenge: The ``'recaptcha_challenge_response'`` HTTP form field to pass to the client, along with the CAPTCHA image. See :doc:`BridgeDB's captcha.html <templates/captcha.html>` Mako_ template for an example usage. - :ivar str publicKey: The public reCaptcha API key. - :ivar str secretKey: The private reCaptcha API key. + :vartype publicKey: str + :ivar publicKey: The public reCaptcha API key. + :vartype secretKey: str + :ivar secretKey: The private reCaptcha API key. .. _reCaptcha: https://code.google.com/p/recaptcha/ .. _Mako: http://docs.makotemplates.org/en/latest/syntax.html#page @@ -162,8 +174,8 @@ class ReCaptcha(Captcha): stored at ``ReCaptcha.image`` and the challenge string at ``ReCaptcha.challenge``. - :raises CaptchaKeyError: If either the :ivar:`publicKey` or - :ivar:`secretKey` are missing. + :raises CaptchaKeyError: If either the :attr:`publicKey` or + :attr:`secretKey` are missing. :raises HTTPError: If the server returned any HTTP error status code. """ if not self.publicKey or not self.secretKey: @@ -186,20 +198,24 @@ class ReCaptcha(Captcha): class GimpCaptcha(Captcha): """A locally cached CAPTCHA image which was created with gimp-captcha_. - :ivar str secretKey: A PKCS#1 OAEP-padded, private RSA key, used for - verifying the client's solution to the CAPTCHA. - :ivar str publickey: A PKCS#1 OAEP-padded, public RSA key. This is used to + :vartype publicKey: str + :ivar publicKey: A PKCS#1 OAEP-padded, public RSA key. This is used to hide the correct CAPTCHA solution within the ``captcha_challenge_field`` HTML form field. That form field is given - to the a client along with the :ivar:`image` during the initial + to the a client along with the :attr:`image` during the initial CAPTCHA request, and the client *should* give it back to us later during the CAPTCHA solution verification step. - :ivar bytes hmacKey: A client-specific HMAC secret key. - :ivar str cacheDir: The local directory which pre-generated CAPTCHA images + :vartype secretKey: str + :ivar secretKey: A PKCS#1 OAEP-padded, private RSA key, used for + verifying the client's solution to the CAPTCHA. + :vartype hmacKey: bytes + :ivar hmacKey: A client-specific HMAC secret key. + :vartype cacheDir: str + :ivar cacheDir: The local directory which pre-generated CAPTCHA images have been stored in. This can be set via the ``GIMP_CAPTCHA_DIR`` setting in the config file. - :type sched: :class:`bridgedb.schedule.ScheduledInterval` - :ivar sched: An time interval. After this much time has passed, the + :vartype sched: :class:`bridgedb.schedule.ScheduledInterval` + :ivar sched: A time interval. After this amount time has passed, the CAPTCHA is considered stale, and all solutions are considered invalid regardless of their correctness. @@ -212,7 +228,7 @@ class GimpCaptcha(Captcha): cacheDir=None): """Create a ``GimpCaptcha`` which retrieves images from **cacheDir**. - :param str publickey: A PKCS#1 OAEP-padded, public RSA key, used for + :param str publicKey: A PKCS#1 OAEP-padded, public RSA key, used for creating the ``captcha_challenge_field`` string to give to a client. :param str secretKey: A PKCS#1 OAEP-padded, private RSA key, used for @@ -221,9 +237,9 @@ class GimpCaptcha(Captcha): :param str cacheDir: The local directory which pre-generated CAPTCHA images have been stored in. This can be set via the ``GIMP_CAPTCHA_DIR`` setting in the config file. - :raises GimpCaptchaError: if :ivar:`cacheDir` is not a directory. - :raises CaptchaKeyError: if any of :ivar:`secretKey`, - :ivar:`publicKey`, or :ivar:`hmacKey` are invalid or missing. + :raises GimpCaptchaError: if :attr:`cacheDir` is not a directory. + :raises CaptchaKeyError: if any of :attr:`secretKey`, + :attr:`publicKey`, or :attr:`hmacKey` are invalid or missing. """ if not cacheDir or not os.path.isdir(cacheDir): raise GimpCaptchaError("Gimp captcha cache isn't a directory: %r" @@ -314,12 +330,12 @@ class GimpCaptcha(Captcha): | Field | Description | Length | +=============+============================================+==========+ | HMAC | An HMAC of the ``ENC_BLOB``, created with | 20 bytes | - | | the client-specific :ivar:`hmacKey`, by | | + | | the client-specific :attr:`hmacKey`, by | | | | applying :func:`~crypto.getHMAC` to the | | | | ``ENC_BLOB``. | | +-------------+--------------------------------------------+----------+ | ENC_BLOB | An encrypted ``ANSWER_BLOB``, created with | varies | - | | a PKCS#1 OAEP-padded RSA :ivar:`publicKey`.| | + | | a PKCS#1 OAEP-padded RSA :attr:`publicKey`.| | +-------------+--------------------------------------------+----------+ | ANSWER_BLOB | Contains the concatenated ``TIMESTAMP`` | varies | | | and ``ANSWER``. | | @@ -328,27 +344,23 @@ class GimpCaptcha(Captcha): | | left-padded with "0"s. | | +-------------+--------------------------------------------+----------+ | ANSWER | A string containing answer to this | 8 bytes | - | | CAPTCHA :ivar:`image`. | | + | | CAPTCHA :attr:`image`. | | +-------------+--------------------------------------------+----------+ The steps taken to produce a ``CHALLENGE`` are then: - 1. Create a ``TIMESTAMP``, and pad it on the left with ``0``s to 12 - bytes in length. - - 2. Next, take the **answer** to this CAPTCHA :ivar:`image: and - concatenate the padded ``TIMESTAMP`` and the ``ANSWER``, forming - an ``ANSWER_BLOB``. - - 3. Encrypt the resulting ``ANSWER_BLOB`` to :ivar:`publicKey` to - create the ``ENC_BLOB``. - - 4. Use the client-specific :ivar:`hmacKey` to apply the - :func:`~crypto.getHMAC` function to the ``ENC_BLOB``, obtaining - an ``HMAC``. - - 5. Create the final ``CHALLENGE`` string by concatenating the - ``HMAC`` and ``ENC_BLOB``, then base64-encoding the result. + 1. Create a ``TIMESTAMP``, and pad it on the left with ``0``s to 12 + bytes in length. + 2. Next, take the **answer** to this CAPTCHA :data:`image` and + concatenate the padded ``TIMESTAMP`` and the ``ANSWER``, forming + an ``ANSWER_BLOB``. + 3. Encrypt the resulting ``ANSWER_BLOB`` to :data:`publicKey` to + create the ``ENC_BLOB``. + 4. Use the client-specific :data:`hmacKey` to apply the + :func:`~crypto.getHMAC` function to the ``ENC_BLOB``, obtaining + an ``HMAC``. + 5. Create the final ``CHALLENGE`` string by concatenating the + ``HMAC`` and ``ENC_BLOB``, then base64-encoding the result. :param str answer: The answer to a CAPTCHA. :rtype: str @@ -369,7 +381,7 @@ class GimpCaptcha(Captcha): challenge string for the CAPTCHA, via :meth:`createChallenge`. :raises GimpCaptchaError: if the chosen CAPTCHA image file could not - be read, or if the **cacheDir** is empty. + be read, or if the :attr:`cacheDir` is empty. :rtype: tuple :returns: A 2-tuple containing the image file contents as a string, and a challenge string (used for checking the client's solution). diff --git a/bridgedb/configure.py b/bridgedb/configure.py index 0056107..bceb002 100644 --- a/bridgedb/configure.py +++ b/bridgedb/configure.py @@ -23,38 +23,42 @@ def loadConfig(configFile=None, configCls=None): All pathnames and filenames within settings in the ``configFile`` will be expanded, and their expanded values will be stored in the returned - :class:`config <Conf>` object. + :class:`configuration <bridgedb.configure.Conf>` object. - ** Note: ** - On the strange-looking use of - ``exec compile(open(configFile).read(), '<string>', 'exec') in dict()`` - in this function: + **Note:** - The contents of the config file should be compiled first, and then - ``exec``ed -- not ``execfile``! -- in order to get the contents of the - config file to exist within the scope of the configuration dictionary. + On the strange-looking use of:: + + exec compile(open(configFile).read(), '<string>', 'exec') in dict() + + in this function… + + The contents of the config file should be compiled first, and then passed + to ``exec()`` -- not ``execfile()`` ! -- in order to get the contents of + the config file to exist within the scope of the configuration dictionary. Otherwise, Python *will* default_ to executing the config file directly within the ``globals()`` scope. - Additionally, it's roughly 20-30 times faster_ to use the ``compile`` - builtin on a string (the contents of the file) before ``exec``ing it, than - using ``execfile`` directly on the file. + Additionally, it's roughly 20-30 times faster_ to use the ``compile()`` + builtin on a string (the contents of the file) before passing it to + ``exec()``, than using ``execfile()`` directly on the file. .. _default: http://stackoverflow.com/q/17470193 .. _faster: http://lucumr.pocoo.org/2011/2/1/exec-in-python/ - :ivar boolean itsSafeToUseLogging: This is called in - :func:`~bridgedb.Main.run` before + :ivar bool itsSafeToUseLogging: This is called in + :func:`bridgedb.Main.run` before :func:`bridgedb.safelog.configureLogging`. When called from :func:`~bridgedb.Main.run`, the **configCls** parameter is not given, - because that is the first time that a :class:`Conf` is created. If a + because that is the first time that a + :class:`config <bridgedb.configure.Conf>` has been created. If a :class:`logging.Logger` is created in this function, then logging will not be correctly configured, therefore, if the **configCls** parameter is not given, then it's the first time this function has been called - and it is therefore not safe to make calls to the logging module. - :type: configFile: string or None + and it is therefore *not* safe to make calls to the logging module. + :type configFile: :any:`str` or ``None`` :param configFile: If given, the filename of the config file to load. - :type configCls: :class:`Conf` or None + :type configCls: :class:`bridgedb.configure.Conf` or ``None`` :param configCls: The current configuration instance, if one already exists. :returns: A new :class:`configuration <bridgedb.configure.Conf>`, with the diff --git a/bridgedb/crypto.py b/bridgedb/crypto.py index 7785073..3854602 100644 --- a/bridgedb/crypto.py +++ b/bridgedb/crypto.py @@ -8,18 +8,17 @@ # (c) 2007-2015, all entities within the AUTHORS file # :license: 3-clause BSD, see included LICENSE for information -"""BridgeDB general cryptographic utilities. +"""This module contains general utilities for working with external +cryptographic tools and libraries, including OpenSSL and GnuPG. It also +includes utilities for creating callable HMAC functions, generating HMACs for +data, and generating and/or storing key material. .. py:module:: bridgedb.crypto - :synopsis: This module contains general utilities for working with external - cryptographic tools and libraries, including OpenSSL and GnuPG. It also - includes utilities for creating callable HMAC functions, generating - HMACs for data, and generating and/or storing key material. + :synopsis: BridgeDB general cryptographic utilities. -Module Overview -~~~~~~~~~~~~~~~ :: - crypto + + bridgedb.crypto |_getGPGContext() - Get a pre-configured GPGME context. |_getHMAC() - Compute an HMAC with some key for some data. |_getHMACFunc() - Get a callable for producing HMACs with the given key. @@ -35,7 +34,7 @@ Module Overview |_getHostnameFromURL() - Parses the hostname from the request URL. \_verifyHostname() - Check that the cert CN matches the request hostname. -:: +.. """ from __future__ import absolute_import @@ -67,8 +66,9 @@ DIGESTMOD = hashlib.sha1 # # TypeError: 'buffer' does not have the buffer interface # -#: ``True`` if we have the new-style -#: `buffer <https://docs.python.org/2/c-api/buffer.html>` interface. +#: ``True`` if we have the new-style `buffer`_ interface; ``False`` otherwise. +#: +#: .. _buffer: https://docs.python.org/2/c-api/buffer.html NEW_BUFFER_INTERFACE = False try: io.BytesIO(buffer('test')) @@ -233,17 +233,22 @@ def getHMACFunc(key, hex=True): def removePKCS1Padding(message): """Remove PKCS#1 padding from a **message**. - (PKCS#1 v1.0? see https://bugs.torproject.org/13042) + (PKCS#1 v1.0? See :trac:`13042`.) Each block is 128 bytes total in size: - * 2 bytes for the type info ('\x00\x01') - * 1 byte for the separator ('\x00') - * variable length padding ('\xFF') + * 2 bytes for the type info (``'\\x00\\x01'``) + * 1 byte for the separator (``'\\x00'``) + * variable length padding (``'\\xFF'``) * variable length for the **message** + .. Note that the above strings are double escaped, due to the way that + Sphinx renders escaped strings in docstrings. + For more information on the structure of PKCS#1 padding, see :rfc:`2313`, - particularly the notes in §8.1. + particularly `the notes in §8.1`__. + + .. __: https://tools.ietf.org/html/rfc2313#section-8.1 :param str message: A message which is PKCS#1 padded. :raises PKCS1PaddingError: if there is an issue parsing the **message**. diff --git a/bridgedb/distribute.py b/bridgedb/distribute.py index f48cbb6..ac8df5f 100644 --- a/bridgedb/distribute.py +++ b/bridgedb/distribute.py @@ -13,87 +13,94 @@ """Classes for creating bridge distribution systems. -DEFINITELY ----------- - -Distributor { - name property - bridgesPerResponse() property FORMERLY getNumBridgesPerAnswer() - hashring struct FORMERLY KNOWN AS splitter - rotate bool - rotationGroups - rotationSchedule - key str - subrings list - - Subring - clear() - export() FORMERLY KNOWN AS dumpAssignments() - insert() - getBridges() FORMERLY KNOWN AS getBridgesForEmail() and getBridgesForIP() - handleBridgeRequest() - handleIncomingBridges() -} - -DistributionContext { # should go in bridgedb.py - distributors { - name: DistributorContext +.. inheritance-diagram:: Distributor + :parts: 1 + +.. + (These are design notes. Please ignore.) + + DEFINITELY + ---------- + + Distributor { + name property + bridgesPerResponse() property FORMERLY getNumBridgesPerAnswer() + hashring struct FORMERLY KNOWN AS splitter + rotate bool + rotationGroups + rotationSchedule + key str + subrings list + - Subring + clear() + export() FORMERLY KNOWN AS dumpAssignments() + insert() + getBridges() FORMERLY KNOWN AS getBridgesForEmail() and getBridgesForIP() + handleBridgeRequest() + handleIncomingBridges() } -} - -DistributorContext { # should go in bridgedb.py - name str - allocationPercentage property - publicKey -} - -Hashring { - assignBridgesToSubrings() FORMERLY bridgedb.filters.assignBridgesToSubring() - + filters bridges uniformly into subrings - clear() / __del__() - isEmpty property -} - -MAYBE ------ -mapClientToHashring() FORMERLY KNOWN AS areaMapper AND -mapClientToSubhashring() -authenticateToBridgeDB() -maintainACL() for proxylists - -- need a way for BridgeDB to decide global parameters to be followed - by all distributors. - - BridgeAnswerParameters? - maybe call it DistributionContext? - then have DistributorContexts? - - requiredFlags AnswerParameters? - requireFlag() - requiredPorts - requirePorts() - - THINGS NEEDED FOR COMMUNICATION BETWEEN DISTRIBUTORS AND BRIDGEDB - ----------------------------------------------------------------- - * distributorCredential (for authenticating to the DB) - * metrics? - * total clients seen - * total clients served - - unique clients seen - - unique clients served - * total requests for TRANSPORT - * total times TRANSPORT was served - - THINGS DISTRIBUTORS SHOULD KEEP TRACK OF, BUT NOT REPORT - -------------------------------------------------------- - - approximate bridge bandwidth - - approximate bandwidth per client - - approximate bridge bandwidth already distributed - - NAMES FOR CHOOSING "GET ME WHATEVER TRANSPORTS" - ----------------------------------------------- - chocolate box, russian roulette - - * How much of a bad idea would it be to store bridges allocated to a - distributor as diffs over the last time the Distributor asked? + + DistributionContext { # should go in bridgedb.py + distributors { + name: DistributorContext + } + } + + DistributorContext { # should go in bridgedb.py + name str + allocationPercentage property + publicKey + } + + Hashring { + assignBridgesToSubrings() FORMERLY bridgedb.filters.assignBridgesToSubring() + + filters bridges uniformly into subrings + clear() / __del__() + isEmpty property + } + + MAYBE + ----- + mapClientToHashring() FORMERLY KNOWN AS areaMapper AND + mapClientToSubhashring() + authenticateToBridgeDB() + maintainACL() for proxylists + + - need a way for BridgeDB to decide global parameters to be followed + by all distributors. + - BridgeAnswerParameters? + maybe call it DistributionContext? + then have DistributorContexts? + + requiredFlags AnswerParameters? + requireFlag() + requiredPorts + requirePorts() + + THINGS NEEDED FOR COMMUNICATION BETWEEN DISTRIBUTORS AND BRIDGEDB + ----------------------------------------------------------------- + * distributorCredential (for authenticating to the DB) + * metrics? + * total clients seen + * total clients served + - unique clients seen + - unique clients served + * total requests for TRANSPORT + * total times TRANSPORT was served + + THINGS DISTRIBUTORS SHOULD KEEP TRACK OF, BUT NOT REPORT + -------------------------------------------------------- + - approximate bridge bandwidth + - approximate bandwidth per client + - approximate bridge bandwidth already distributed + + NAMES FOR CHOOSING "GET ME WHATEVER TRANSPORTS" + ----------------------------------------------- + chocolate box, russian roulette + + * How much of a bad idea would it be to store bridges allocated to a + distributor as diffs over the last time the Distributor asked? + """ import logging diff --git a/bridgedb/email/autoresponder.py b/bridgedb/email/autoresponder.py index ad63bfd..f4a93e3 100644 --- a/bridgedb/email/autoresponder.py +++ b/bridgedb/email/autoresponder.py @@ -21,6 +21,9 @@ bridgedb.email.autoresponder Functionality for autoresponding to incoming emails. +.. inheritance-diagram:: EmailResponse SMTPAutoresponder + :parts: 1 + :: bridgedb.email.autoresponder @@ -66,7 +69,7 @@ def createResponseBody(lines, context, client, lang='en'): :param list lines: The list of lines from the original request sent by the client. - :type context: class:`bridgedb.email.server.MailServerContext` + :type context: :class:`bridgedb.email.server.MailServerContext` :param context: The context which contains settings for the email server. :type client: :api:`twisted.mail.smtp.Address` :param client: The client's email address which should be in the @@ -77,11 +80,11 @@ def createResponseBody(lines, context, client, lang='en'): email to `bridges+fa(a)torproject.org <mailto:bridges+fa@torproject.org>`__, the client should receive a response in Farsi. - :rtype: None or str - :returns: None if we shouldn't respond to the client (i.e., if they have - already received a rate-limiting warning email). Otherwise, returns a - string containing the (optionally translated) body for the email - response which we should send out. + :rtype: str + :returns: ``None`` if we shouldn't respond to the client (i.e., if they + have already received a rate-limiting warning email). Otherwise, + returns a string containing the (optionally translated) body for the + email response which we should send out. """ translator = translations.installTranslations(lang) bridges = None @@ -122,26 +125,26 @@ def createResponseBody(lines, context, client, lang='en'): def generateResponse(fromAddress, client, body, subject=None, messageID=None, gpgSignFunc=None): - """Create a :class:`EmailResponse`, which acts like an in-memory - ``io.StringIO`` file, by creating and writing all headers and the email - body into the file-like ``EmailResponse.mailfile``. + """Create an :class:`EmailResponse`, which acts like an + :class:`io.StringIO` instance, by creating and writing all headers and the + email body into the file-like :attr:`EmailResponse.mailfile`. - :param str fromAddress: The rfc:`2821` email address which should be in - the :header:`From:` header. + :param str fromAddress: The :rfc:`2821` email address which should be in + the ``'From:'`` header. :type client: :api:`twisted.mail.smtp.Address` :param client: The client's email address which should be in the ``'To:'`` header of the response email. - :param str subject: The string to write to the ``Subject:'`` header. + :param str subject: The string to write to the ``'Subject:'`` header. :param str body: The body of the email. If a **gpgSignFunc** is also given, then :meth:`EmailResponse.writeBody` will generate and include an ascii-armored OpenPGP signature in the **body**. - :type messageID: None or str + :type messageID: ``None`` or :any:`str` :param messageID: The :rfc:`2822` specifier for the ``'Message-ID:'`` header, if including one is desirable. - :type gpgSignFunc: ``None`` or callable - :param gpgSignFunc: A function for signing messages. See - :func:`bridgedb.crypto.initializeGnuPG` for obtaining a pre-configured - **gpgSignFunc**. + :type gpgSignFunc: callable + :param gpgSignFunc: If given, this should be a callable function for + signing messages. See :func:`bridgedb.crypto.initializeGnuPG` for + obtaining a pre-configured **gpgSignFunc**. :returns: An :class:`EmailResponse` which contains the entire email. To obtain the contents of the email, including all headers, simply use :meth:`EmailResponse.readContents`. @@ -173,17 +176,20 @@ class EmailResponse(object): keyfile, for example, rather than simply pasting it into the body of the email.) - :var _buff: (unicode or buffer) Used internally to write lines for the - response email into the ``_mailfile``. The reason why both of these - attributes have two possible types is for the same Python-buggy - reasons which require :data:`~bridgedb.crypto.NEW_BUFFER_INTERFACE`. - :var mailfile: (:class:`io.StringIO` or :class:`io.BytesIO`) An in-memory - file for storing the formatted headers and body of the response email. + + :vartype _buff: :any:`unicode` or :any:`buffer` + :var _buff: Used internally to write lines for the response email into the + ``_mailfile``. The reason why both of these attributes have two + possible types is for the same Python-buggy reasons which require + :data:`~bridgedb.crypto.NEW_BUFFER_INTERFACE`. + :vartype mailfile: :class:`io.StringIO` or :class:`io.BytesIO` + :var mailfile: An in-memory file-like object for storing the formatted + headers and body of the response email. :var str delimiter: Delimiter between lines written to the :data:`mailfile`. :var bool closed: ``True`` if :meth:`close` has been called. - :var to: An :api:`twisted.mail.smtp.Address` for the client's email address - which this response should be sent to. + :vartype to: :api:`twisted.mail.smtp.Address` + :var to: The client's email address, to which this response should be sent. """ _buff = buffer if NEW_BUFFER_INTERFACE else unicode mailfile = io.BytesIO if NEW_BUFFER_INTERFACE else io.StringIO @@ -194,10 +200,10 @@ class EmailResponse(object): This class deals with correctly formatting text for the response email headers and the response body into an instance of :data:`mailfile`. - :type gpgSignFunc: ``None`` or callable - :param gpgSignFunc: A function for signing messages. See - :func:`bridgedb.crypto.initializeGnuPG` for obtaining a - pre-configured **gpgSignFunc**. + :type gpgSignFunc: callable + :param gpgSignFunc: If given, this should be a callable function for + signing messages. See :func:`bridgedb.crypto.initializeGnuPG` for + obtaining a pre-configured **gpgSignFunc**. """ self.gpgSign = gpgSignFunc self.mailfile = self.mailfile() @@ -279,8 +285,8 @@ class EmailResponse(object): (i.e. ``'\\n'``). See :api:`twisted.mail.smtp.SMTPClient.getMailData` for the reason. - :type lines: basestring or list - :param lines: The lines to write to the :ivar:`mailfile`. + :type lines: :any:`basestring` or :any:`list` + :param lines: The lines to write to the :attr:`mailfile`. """ if isinstance(lines, basestring): lines = lines.replace('\r\n', '\n') @@ -297,9 +303,9 @@ class EmailResponse(object): :param str fromAddress: The email address for the ``'From:'`` header. :param str toAddress: The email address for the ``'To:'`` header. - :type subject: None or str + :type subject: ``None`` or :any:`str` :param subject: The ``'Subject:'`` header. - :type inReplyTo: None or str + :type inReplyTo: ``None`` or :any:`str` :param inReplyTo: If set, an ``'In-Reply-To:'`` header will be generated. This should be set to the ``'Message-ID:'`` header from the client's original request email. @@ -307,7 +313,7 @@ class EmailResponse(object): ``'Message-ID:'`` header for the response. :param str contentType: The ``'Content-Type:'`` header. :kwargs: If given, the key will become the name of the header, and the - value will become the Contents of that header. + value will become the contents of that header. """ self.write("From: %s" % fromAddress) self.write("To: %s" % toAddress) @@ -336,11 +342,11 @@ class EmailResponse(object): self.write(self.delimiter) def writeBody(self, body): - """Write the response body into the :cvar:`mailfile`. + """Write the response body into the :attr:`mailfile`. - If ``EmailResponse.gpgSignFunc`` is set, and signing is configured, the - **body** will be automatically signed before writing its contents into - the ``mailfile``. + If :attr:`EmailResponse.gpgSignFunc` is set, and signing is configured, + the **body** will be automatically signed before writing its contents + into the :attr:`mailfile`. :param str body: The body of the response email. """ @@ -361,12 +367,17 @@ class SMTPAutoresponder(smtp.SMTPClient): create a :class:`EmailResponse` email message in reply to it, and then, finally, send it out. - :ivar log: A :api:`twisted.python.util.LineLog` cache of messages. - :ivar debug: If ``True``, enable logging (accessible via :ivar:`log`). + :vartype log: :api:`twisted.python.util.LineLog` + :ivar log: A cache of debug log messages. + :vartype debug: bool + :ivar debug: If ``True``, enable logging (accessible via :attr:`log`). :ivar str identity: Our FQDN which will be sent during client ``HELO``. - :ivar incoming: An incoming - :api:`Message <twisted.mail.smtp.rfc822.Message>`, i.e. as returned - from :meth:`SMTPMessage.getIncomingMessage`. + :vartype incoming: :api:`Message <twisted.mail.smtp.rfc822.Message>` + :ivar incoming: An incoming message, i.e. as returned from + :meth:`SMTPMessage.getIncomingMessage`. + + :vartype deferred: :api:`twisted.internet.defer.Deferred` + :ivar deferred: A :api:`Deferred <twisted.internet.defer.Deferred>` with registered callbacks, :meth:`sentMail` and :meth:`sendError`, which will be given to the reactor in order to process the sending of the @@ -546,9 +557,10 @@ class SMTPAutoresponder(smtp.SMTPClient): def sendError(self, fail): """Errback for a :api:`twisted.mail.smtp.SMTPSenderFactory`. - :param fail: A :api:`twisted.python.failure.Failure` or a - :api:`twisted.mail.smtp.SMTPClientError` which occurred during the - transaction to send the outgoing email. + :type fail: :api:`twisted.python.failure.Failure` or + :api:`twisted.mail.smtp.SMTPClientError` + :param fail: An exception which occurred during the transaction to + send the outgoing email. """ logging.debug("called with %r" % fail) @@ -604,13 +616,14 @@ class SMTPAutoresponder(smtp.SMTPClient): 4. If the incoming message is from a domain which supports DKIM signing, then run :func:`bridgedb.email.dkim.checkDKIM` as well. - .. note:: Calling this method sets the ``canonicalFromEmail`` and - :data:``canonicalDomainRules`` attributes of the :data:`incoming` - message. + .. note:: Calling this method sets the + :attr:`incoming.canonicalFromEmail` and + :attr:`incoming.canonicalDomainRules` attributes of the + :attr:`incoming` message. - :param client: An :api:`twisted.mail.smtp.Address`, which contains - the client's email address, extracted from the ``'From:'`` header - from the incoming email. + :type client: :api:`twisted.mail.smtp.Address` + :param client: The client's email address, extracted from the + ``'From:'`` header from the incoming email. :rtype: bool :returns: ``False`` if the checks didn't pass, ``True`` otherwise. """ diff --git a/bridgedb/email/distributor.py b/bridgedb/email/distributor.py index d8ea9bf..8f60b0f 100644 --- a/bridgedb/email/distributor.py +++ b/bridgedb/email/distributor.py @@ -10,8 +10,18 @@ # (c) 2007-2015, The Tor Project, Inc. # :license: see LICENSE for licensing information -"""A :class:`~bridgedb.distribute.Distributor` which hands out -:class:`bridges <bridgedb.bridges.Bridge>` to clients via an email interface. +""" +.. py:module:: bridgedb.email.distributor + :synopsis: A Distributor which hands out Bridges via an email interface. + +bridgedb.email.autoresponder +============================ + +A :class:`~bridgedb.distribute.Distributor` which hands out :class:`bridges +<bridgedb.bridges.Bridge>` to clients via an email interface. + +.. inheritance-diagram:: IgnoreEmail TooSoonEmail EmailRequestedHelp EmailRequestedKey EmailDistributor + :parts: 1 """ import logging @@ -117,13 +127,14 @@ class EmailDistributor(Distributor): :data:`~bridgedb.bridgerequest.BridgeRequestBase.client` attribute set to a string containing the client's full, canonicalized email address. + :type interval: str :param interval: The time period when we got this request. This can be any string, so long as it changes with every period. :type clock: :api:`twisted.internet.task.Clock` :param clock: If given, use the clock to ask what time it is, rather than :api:`time.time`. This should likely only be used for testing. - :rtype: list or ``None`` + :rtype: :any:`list` or ``None`` :returns: A list of :class:`~bridgedb.bridges.Bridges` for the ``bridgeRequest.client``, if allowed. Otherwise, returns ``None``. """ diff --git a/bridgedb/email/dkim.py b/bridgedb/email/dkim.py index d1075aa..7b6adfa 100644 --- a/bridgedb/email/dkim.py +++ b/bridgedb/email/dkim.py @@ -26,6 +26,7 @@ Functions for checking DKIM verification results in email headers. bridgedb.email.dkim |_ checkDKIM - Check the DKIM verification results header. + .. """ diff --git a/bridgedb/email/request.py b/bridgedb/email/request.py index 50fd32c..5721fa3 100644 --- a/bridgedb/email/request.py +++ b/bridgedb/email/request.py @@ -23,6 +23,9 @@ bridgedb.email.request Classes for parsing and storing information about requests for bridges which are sent to the email distributor. +.. inheritance-diagram:: EmailBridgeRequest + :parts: 1 + :: bridgedb.email.request @@ -30,6 +33,7 @@ which are sent to the email distributor. | offer help. |_ EmailBridgeRequest - A request for bridges which was received through the email distributor. + .. """ @@ -56,7 +60,7 @@ UNBLOCKED_PATTERN = re.compile(UNBLOCKED_REGEXP) def determineBridgeRequestOptions(lines): - """Figure out which :class:`Bridges.BridgeFilter`s to apply, or offer help. + """Figure out which :mod:`~bridgedb.filters` to apply, or offer help. .. note:: If any ``'transport TYPE'`` was requested, or bridges not blocked in a specific CC (``'unblocked CC'``), then the ``TYPE`` @@ -66,7 +70,7 @@ def determineBridgeRequestOptions(lines): :raises EmailRequestedHelp: if the client requested help. :raises EmailRequestedKey: if the client requested our GnuPG key. :rtype: :class:`EmailBridgeRequest` - :returns: A :class:`~bridgerequst.BridgeRequest` with all of the requested + :returns: A :class:`~bridgerequest.BridgeRequest` with all of the requested parameters set. The returned ``BridgeRequest`` will have already had its filters generated via :meth:`~EmailBridgeRequest.generateFilters`. """ diff --git a/bridgedb/email/server.py b/bridgedb/email/server.py index 736c3f6..010648f 100644 --- a/bridgedb/email/server.py +++ b/bridgedb/email/server.py @@ -23,6 +23,9 @@ bridgedb.email.server Servers which interface with clients and distribute bridges over SMTP. +.. inheritance-diagram:: MailServerContext SMTPMessage SMTPIncomingDelivery SMTPIncomingDeliveryFactory SMTPIncomingServerFactory + :parts: 1 + :: bridgedb.email.server @@ -40,6 +43,7 @@ Servers which interface with clients and distribute bridges over SMTP. creates a new SMTPMessageDelivery, which handles response email automation, whenever we get a incoming connection on the SMTP port. + .. """ @@ -97,6 +101,8 @@ class MailServerContext(object): initialize GnuPG for some reason. :ivar gpgSignFunc: A callable which signs a message, e.g. the one returned from :func:`~bridgedb.crypto.initialiseGnuPG`. + + .. _interface: https://pythonhosted.org/gnupg/gnupg.html#gnupg-module """ def __init__(self, config, distributor, schedule): @@ -313,7 +319,7 @@ class SMTPIncomingDelivery(smtp.SMTP): :param origin: The email address we received this message from. :raises: :api:`twisted.mail.smtp.SMTPBadSender` if the ``origin.domain`` was neither our local hostname, nor one of the - canonical domains listed in :ivar:`context.canon`. + canonical domains listed in :attr:`context.canon`. :rtype: :api:`twisted.mail.smtp.Address` :returns: The ``origin``. We *must* return some non-``None`` data from this method, or else Twisted will reply to the sender with a 503 @@ -425,18 +431,23 @@ class SMTPIncomingServerFactory(smtp.SMTPFactory): :class:`SMTPIncomingDeliveryFactory`, which handles response email automation whenever we get a incoming connection on the SMTP port. - .. warning:: My :data:`context` isn't an OpenSSL context, as is used for - the :api:`twisted.mail.smtp.ESMTPSender`. + .. warning:: + My :attr:`~bridgedb.email.server.SMTPIncomingServerFactory.context` + isn't an OpenSSL context, as is used for the + :api:`twisted.mail.smtp.ESMTPSender`. - :ivar context: A :class:`MailServerContext` for storing configuration settings. - :ivar deliveryFactory: A :class:`SMTPIncomingDeliveryFactory` for - producing :class:`SMTPIncomingDelivery`s. + :vartype context: :class:`MailServerContext` + :ivar context: A context for storing server configuration settings. + :vartype deliveryFactory: :class:`SMTPIncomingDeliveryFactory` + :ivar deliveryFactory: A factory for producing + :class:`SMTPIncomingDelivery` instances. :ivar domain: :api:`Our FQDN <twisted.mail.smtp.DNSNAME>`. :ivar int timeout: The number of seconds to wait, after the last chunk of data was received, before raising a :api:`SMTPTimeoutError <twisted.mail.smtp.SMTPTimeoutError>` for an incoming connection. - :ivar protocol: :api:`SMTP <twisted.mail.smtp.SMTP>` + :vartype protocol: :api:`twisted.internet.protocol.Protocol` + :ivar protocol: :api:`twisted.mail.smtp.SMTP` """ context = None diff --git a/bridgedb/filters.py b/bridgedb/filters.py index ca9b673..c11c7ce 100644 --- a/bridgedb/filters.py +++ b/bridgedb/filters.py @@ -11,6 +11,8 @@ # :license: see LICENSE for licensing information #_____________________________________________________________________________ +"""Functions for filtering :class:`Bridges <bridgedb.bridges.Bridge>`.""" + import logging from ipaddr import IPv4Address @@ -36,7 +38,7 @@ def bySubring(hmac, assigned, total): also be assigned to subring 2of3. :param int total: The total number of subrings. :rtype: callable - :returns: A filter function for :class:`~bridgedb.bridges.Bridge`s. + :returns: A filter function for :class:`Bridges <bridgedb.bridges.Bridge>`. """ logging.debug(("Creating a filter for assigning bridges to subhashring " "%s-of-%s...") % (assigned, total)) @@ -61,11 +63,11 @@ def bySubring(hmac, assigned, total): def byFilters(filtres): """Returns a filter which filters by multiple **filtres**. - :type filtres: list - :param filtres: A list (or other iterable) of callables which some - :class:`~bridgedb.bridges.Bridge`s should be filtered according to. + :param list filtres: A list (or other iterable) of callables which some + :class:`Bridges <bridgedb.bridges.Bridge>` should be filtered + according to. :rtype: callable - :returns: A filter function for :class:`~bridgedb.bridges.Bridge`s. + :returns: A filter function for :class:`Bridges <bridgedb.bridges.Bridge>`. """ name = [] for filtre in filtres: @@ -100,6 +102,14 @@ def byIPv(ipVersion=None): return _cache[name] except KeyError: def _byIPv(bridge): + """Determine if the **bridge** has an IPv{0} address. + + :type bridge: :class:`bridgedb.bridges.Bridge` + :param bridge: A bridge to filter. + :rtype: bool + :returns: ``True`` if the **bridge** has an address with the + correct IP version; ``False`` otherwise. + """ if isIPv(ipVersion, bridge.address): return True else: @@ -109,6 +119,7 @@ def byIPv(ipVersion=None): return False setattr(_byIPv, "description", "ip=%d" % ipVersion) _byIPv.__name__ = "byIPv%d()" % ipVersion + _byIPv.func_doc = _byIPv.func_doc.format(ipVersion) _byIPv.name = name _cache[name] = _byIPv return _byIPv @@ -117,25 +128,27 @@ byIPv4 = byIPv(4) byIPv6 = byIPv(6) def byTransport(methodname=None, ipVersion=None): - """Returns a filter function for :class:`~bridgedb.bridges.Bridge`s. + """Returns a filter function for a :class:`~bridgedb.bridges.Bridge`. The returned filter function should be called on a - :class:`~bridgedb.bridges.Bridge`. It returns ``True`` if the ``Bridge`` - has a :class:`~bridgedb.bridges.PluggableTransport` such that: + :class:`~bridgedb.bridges.Bridge`. It returns ``True`` if the + :class:`~bridgedb.bridges.Bridge` has a + :class:`~bridgedb.bridges.PluggableTransport` such that: - 1. The :data:`~bridge.bridges.PluggableTransport.methodname` matches - **methodname**, and + 1. The :data:`methodname <bridgedb.bridges.PluggableTransport.methodname>` + matches **methodname**, and, - 2. The :data:`~bridgedb.bridges.PluggableTransport.address`` version - matches the **ipVersion**. + 2. The :attr:`bridgedb.bridges.PluggableTransport.address.version` + equals the **ipVersion**. :param str methodname: A Pluggable Transport - :data:`~bridge.bridges.PluggableTransport.methodname`. + :data:`~bridgedb.bridges.PluggableTransport.methodname`. :param int ipVersion: Either ``4`` or ``6``. The IP version that the ``Bridge``'s ``PluggableTransport`` - :data:`~bridgedb.bridges.PluggableTransport.address`` should have. + :attr:`address <bridgedb.bridges.PluggableTransport.address>` should + have. :rtype: callable - :returns: A filter function for :class:`~bridgedb.bridges.Bridge`s. + :returns: A filter function for :class:`Bridges <bridgedb.bridges.Bridge>`. """ if not ipVersion in (4, 6): ipVersion = 4 @@ -161,7 +174,7 @@ def byTransport(methodname=None, ipVersion=None): return _byTransport def byNotBlockedIn(countryCode=None, methodname=None, ipVersion=4): - """Returns a filter function for :class:`~bridgedb.bridges.Bridge`s. + """Returns a filter function for :class:`Bridges <bridgedb.bridges.Bridge>`. If a Pluggable Transport **methodname** was not specified, the returned filter function returns ``True`` if any of the ``Bridge``'s addresses or @@ -169,25 +182,29 @@ def byNotBlockedIn(countryCode=None, methodname=None, ipVersion=4): **countryCode**. See :meth:`~bridgedb.bridges.Bridge.isBlockedIn`. Otherwise, if a Pluggable Transport **methodname** was specified, it - returns ``True`` if the ``Bridge`` has a + returns ``True`` if the :class:`~bridgedb.bridges.Bridge` has a :class:`~bridgedb.bridges.PluggableTransport` such that: - 1. The :data:`~bridge.bridges.PluggableTransport.methodname` matches - **methodname**, + 1. The :data:`methodname <bridgedb.bridges.PluggableTransport.methodname>` + matches **methodname**, + + 2. The :attr:`bridgedb.bridges.PluggableTransport.address.version` + equals the **ipVersion**, and, - 2. The :data:`~bridgedb.bridges.PluggableTransport.address.version`` - equals the **ipVersion**, and isn't known to be blocked in - **countryCode**. + 3. The :class:`~bridgedb.bridges.PluggableTransport`. + :attr:`address <bridgedb.bridges.PluggableTransport.address>` isn't + known to be blocked in **countryCode**. :type countryCode: str or ``None`` :param countryCode: A two-letter country code which the filtered - :class:`PluggableTransport`s should not be blocked in. + :class:`PluggableTransports <bridgedb.bridges.PluggableTransport>` + should not be blocked in. :param str methodname: A Pluggable Transport - :data:`~bridge.bridges.PluggableTransport.methodname`. + :data:`methodname <bridgedb.bridges.PluggableTransport.methodname>`. :param int ipVersion: Either ``4`` or ``6``. The IP version that the - ``Bridge``'s addresses should have. + ``PluggableTransports``'s addresses should have. :rtype: callable - :returns: A filter function for :class:`~bridgedb.bridges.Bridge`s. + :returns: A filter function for :class:`Bridges <bridgedb.bridges.Bridge>`. """ if not ipVersion in (4, 6): ipVersion = 4 diff --git a/bridgedb/https/distributor.py b/bridgedb/https/distributor.py index f8cf09d..7d9a293 100644 --- a/bridgedb/https/distributor.py +++ b/bridgedb/https/distributor.py @@ -10,7 +10,15 @@ # (c) 2007-2015, The Tor Project, Inc. # :license: see LICENSE for licensing information -"""A Distributor that hands out bridges through a web interface.""" +""" +bridgedb.https.distributor +========================== + +A Distributor that hands out bridges through a web interface. + +.. inheritance-diagram:: HTTPSDistributor + :parts: 1 +""" import ipaddr import logging @@ -216,7 +224,7 @@ class HTTPSDistributor(Distributor): :data:`proxies`. Thus, the resulting hashring-subhashring structure would look like: - +------------------+---------------------------------------------------+-------------- + +------------------+---------------------------------------------------+-------------+ | | Directly connecting users | Tor / known | | | | proxy users | +------------------+------------+------------+------------+------------+-------------+ diff --git a/bridgedb/https/request.py b/bridgedb/https/request.py index b106a13..e438044 100644 --- a/bridgedb/https/request.py +++ b/bridgedb/https/request.py @@ -20,12 +20,15 @@ bridgedb.https.request Classes for parsing and storing information about requests for bridges which are sent to the HTTPS distributor. +.. inheritance-diagram:: HTTPSBridgeRequest + :: bridgedb.https.request | |_ HTTPSBridgeRequest - A request for bridges which was received through the HTTPS distributor. + .. """ diff --git a/bridgedb/https/server.py b/bridgedb/https/server.py index 2106dbf..81ce09f 100644 --- a/bridgedb/https/server.py +++ b/bridgedb/https/server.py @@ -16,6 +16,9 @@ bridgedb.https.server ===================== Servers which interface with clients and distribute bridges over HTTP(S). + +.. inheritance-diagram:: TranslatedTemplateResource IndexResource OptionsResource HowtoResource CaptchaProtectedResource GimpCaptchaProtectedResource ReCaptchaProtectedResource BridgesResource + :parts: 1 """ import base64 @@ -75,15 +78,14 @@ logging.debug("Set template root to %s" % TEMPLATE_DIR) def getClientIP(request, useForwardedHeader=False): - """Get the client's IP address from the :header:`X-Forwarded-For` + """Get the client's IP address from the ``'X-Forwarded-For:'`` header, or from the :api:`request <twisted.web.server.Request>`. :type request: :api:`twisted.web.http.Request` - :param request: A ``Request`` object for a - :api:`twisted.web.resource.Resource`. + :param request: A ``Request`` for a :api:`twisted.web.resource.Resource`. :param bool useForwardedHeader: If ``True``, attempt to get the client's - IP address from the :header:`X-Forwarded-For` header. - :rtype: None or str + IP address from the ``'X-Forwarded-For:'`` header. + :rtype: ``None`` or :any:`str` :returns: The client's IP address, if it was obtainable. """ ip = None @@ -153,7 +155,7 @@ class TranslatedTemplateResource(resource.Resource): isLeaf = True def __init__(self, template=None): - """Create a new :api:`~twisted.web.resource.Resource` for a + """Create a new :api:`Resource <twisted.web.resource.Resource>` for a Mako-templated webpage. """ gettext.install("bridgedb", unicode=True) @@ -214,13 +216,13 @@ class CaptchaProtectedResource(resource.Resource): self.resource = protectedResource def getClientIP(self, request): - """Get the client's IP address from the :header:`X-Forwarded-For` + """Get the client's IP address from the ``'X-Forwarded-For:'`` header, or from the :api:`request <twisted.web.server.Request>`. :type request: :api:`twisted.web.http.Request` - :param request: A ``Request`` object for a + :param request: A ``Request`` for a :api:`twisted.web.resource.Resource`. - :rtype: None or str + :rtype: ``None`` or :any:`str` :returns: The client's IP address, if it was obtainable. """ return getClientIP(request, self.useForwardedHeader) @@ -228,10 +230,11 @@ class CaptchaProtectedResource(resource.Resource): def getCaptchaImage(self, request=None): """Get a CAPTCHA image. + :rtype: tuple :returns: A 2-tuple of ``(image, challenge)``, where ``image`` is a - binary, JPEG-encoded image, and ``challenge`` is a unique - string. If unable to retrieve a CAPTCHA, returns a tuple - containing two empty strings. + binary, JPEG-encoded image, and ``challenge`` is a unique + string. If unable to retrieve a CAPTCHA, returns a tuple + containing two empty strings. """ return ('', '') @@ -672,13 +675,13 @@ class BridgesResource(resource.Resource): return response def getClientIP(self, request): - """Get the client's IP address from the :header:`X-Forwarded-For` + """Get the client's IP address from the ``'X-Forwarded-For:'`` header, or from the :api:`request <twisted.web.server.Request>`. :type request: :api:`twisted.web.http.Request` :param request: A ``Request`` object for a :api:`twisted.web.resource.Resource`. - :rtype: None or str + :rtype: ``None`` or :any:`str` :returns: The client's IP address, if it was obtainable. """ return getClientIP(request, self.useForwardedHeader) diff --git a/bridgedb/interfaces.py b/bridgedb/interfaces.py index 89ddb12..3ec01b6 100644 --- a/bridgedb/interfaces.py +++ b/bridgedb/interfaces.py @@ -11,7 +11,10 @@ #_____________________________________________________________________________ -"""All available ``zope.interface``s in BridgeDB.""" +"""All available `Zope`_ interfaces in BridgeDB. + +.. _Zope: http://docs.zope.org/zope.interface/index.html +""" from zope.interface import Interface from zope.interface import Attribute @@ -26,9 +29,10 @@ class IName(Interface): @implementer(IName) class Named(object): - """A named object""" + """A named object.""" - #: The characters used to join child Named object's names with our name. + #: The character(s) used to join child :class:`Named` object's names with + #: our name. separator = ' ' def __init__(self): diff --git a/bridgedb/parse/descriptors.py b/bridgedb/parse/descriptors.py index 2986fd3..d1c4a6c 100644 --- a/bridgedb/parse/descriptors.py +++ b/bridgedb/parse/descriptors.py @@ -94,8 +94,8 @@ def parseNetworkStatusFile(filename, validate=True, skipAnnotations=True, See :trac:`12254` for why networkstatus-bridges documents don't look anything like the networkstatus v2 documents that they are purported to - look like. They are missing all headers, and the entire footer including - authority signatures. + look like. They are missing all headers, and the entire footer (including + authority signatures). :param str filename: The location of the file containing bridge networkstatus descriptors. @@ -105,8 +105,10 @@ def parseNetworkStatusFile(filename, validate=True, skipAnnotations=True, :param bool skipAnnotations: If ``True``, skip parsing everything before the first ``r`` line. :param descriptorClass: A class (probably from - :api:`stem.descriptors.router_status_entry`) which Stem will parse - each descriptor it reads from **filename** into. + :mod:`stem.descriptors.router_status_entry`, i.e. + :class:`stem.descriptor.router_status_entry.RouterStatusEntryV2` or + :class:`stem.descriptor.router_status_entry.RouterStatusEntryV3`) + which Stem will parse each descriptor it reads from **filename** into. :raises InvalidRouterNickname: if one of the routers in the networkstatus file had a nickname which does not conform to Tor's nickname specification. @@ -115,7 +117,7 @@ def parseNetworkStatusFile(filename, validate=True, skipAnnotations=True, :raises IOError: if the file at **filename** can't be read. :rtype: list :returns: A list of - :api:`stem.descriptor.router_status_entry.RouterStatusEntryV`s. + :class:`stem.descriptor.router_status_entry.RouterStatusEntry`. """ routers = [] @@ -142,24 +144,19 @@ def parseNetworkStatusFile(filename, validate=True, skipAnnotations=True, return routers def parseServerDescriptorsFile(filename, validate=True): - """Parse a file which contains ``@type bridge-server-descriptor``s. - - .. note:: ``validate`` defaults to ``False`` because there appears to be a - bug in Leekspin, the fake descriptor generator, where Stem thinks the - fingerprint doesn't match the key… + """Open and parse **filename**, which should contain + ``@type bridge-server-descriptor``. .. note:: We have to lie to Stem, pretending that these are - ``@type server-descriptor``s, **not** - ``@type bridge-server-descriptor``s. See ticket #`11257`_. - - .. _`11257`: https://bugs.torproject.org/11257 + ``@type server-descriptor``, **not** + ``@type bridge-server-descriptor``. See :trac:`11257`. :param str filename: The file to parse descriptors from. :param bool validate: Whether or not to validate descriptor - contents. (default: ``False``) + contents. (default: ``True``) :rtype: list :returns: A list of - :api:`stem.descriptor.server_descriptor.RelayDescriptor`s. + :class:`stem.descriptor.server_descriptor.RelayDescriptor`s. """ logging.info("Parsing server descriptors with Stem: %s" % filename) descriptorType = 'server-descriptor 1.0' @@ -198,9 +195,9 @@ def deduplicate(descriptors, statistics=False): to be broken or malicious. :param list descriptors: A list of - :api:`stem.descriptor.server_descriptor.RelayDescriptor`s, - :api:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor`s, - or :api:`stem.descriptor.router_status_entry.RouterStatusEntryV2`s. + :class:`stem.descriptor.server_descriptor.RelayDescriptor`, + :class:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor`, + or :class:`stem.descriptor.router_status_entry.RouterStatusEntry`. :param bool statistics: If ``True``, log some extra statistics about the number of duplicates. :rtype: dict @@ -241,26 +238,30 @@ def deduplicate(descriptors, statistics=False): return newest def parseExtraInfoFiles(*filenames, **kwargs): - """Parse files which contain ``@type bridge-extrainfo-descriptor``s. + """Open **filenames** and parse any ``@type bridge-extrainfo-descriptor`` + contained within. .. warning:: This function will *not* check that the ``router-signature`` at the end of the extrainfo descriptor is valid. See ``bridgedb.bridges.Bridge._verifyExtraInfoSignature`` for a method for - checking the signature. + checking the signature. The signature cannot be checked here, because + to do so, we would need the latest, valid, corresponding + ``signing-key`` for the Bridge. .. note:: This function will call :func:`deduplicate` to deduplicate the extrainfo descriptors parsed from all **filenames**. :kwargs validate: If there is a ``'validate'`` keyword argument, its value will be passed along as the ``'validate'`` argument to - :api:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor`. + :class:`stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor`. The ``'validate'`` keyword argument defaults to ``True``, meaning that the hash digest stored in the ``router-digest`` line will be checked against the actual contents of the descriptor and the extrainfo document's signature will be verified. :rtype: dict - :returns: A dictionary mapping bridge fingerprints to deduplicated - :api:`stem.descriptor.extrainfo_descriptor.RelayExtraInfoDescriptor`s. + :returns: A dictionary mapping bridge fingerprints to their corresponding, + deduplicated + :class:`stem.descriptor.extrainfo_descriptor.RelayExtraInfoDescriptor`. """ descriptors = [] diff --git a/bridgedb/parse/fingerprint.py b/bridgedb/parse/fingerprint.py index bf12ee5..4e3ff02 100644 --- a/bridgedb/parse/fingerprint.py +++ b/bridgedb/parse/fingerprint.py @@ -10,17 +10,19 @@ # :license: see LICENSE for licensing information #_____________________________________________________________________________ -"""Utility functions for converting between various relay fingerprint formats, -and checking their validity. - -.. py:module:: bridgedb.parse.fingerprints +""" +.. py:module:: bridgedb.parse.fingerprint :synopsis: Parsers for Tor Bridge fingerprints. .. todo: This module is very small; it could possibly be combined with another module, e.g. :mod:`bridgedb.parse.descriptors`. -bridgedb.parse.fingerprints +bridgedb.parse.fingerprint ============================ + +Utility functions for converting between various relay fingerprint formats, +and checking their validity. + :: toHex - Convert a fingerprint from its binary representation to hexadecimal. diff --git a/bridgedb/proxy.py b/bridgedb/proxy.py index 086f3bd..5b881fa 100644 --- a/bridgedb/proxy.py +++ b/bridgedb/proxy.py @@ -116,8 +116,11 @@ def loadProxiesFromFile(filename, proxySet=None, removeStale=False): class ProxySet(MutableSet): - """A :class:`collections.MutableSet` for storing validated IP addresses.""" + """A :class:`collections.MutableSet` for storing validated IP addresses. + .. inheritance-diagram:: ProxySet + :parts: 1 + """ #: A tag to apply to IP addresses within this ``ProxySet`` which are known #: Tor exit relays. _exitTag = 'exit_relay' diff --git a/bridgedb/safelog.py b/bridgedb/safelog.py index 8130be6..24c9768 100644 --- a/bridgedb/safelog.py +++ b/bridgedb/safelog.py @@ -9,6 +9,9 @@ """Filters for log sanitisation. +.. inheritance-diagram:: BaseSafelogFilter SafelogEmailFilter SafelogIPv4Filter SafelogIPv6Filter + :parts: 1 + The ``Safelog*Filter`` classes within this module can be instantiated and adding to any :class:`logging.Handler`, in order to transparently filter substrings within log messages which match the given ``pattern``. Matching @@ -27,22 +30,24 @@ with the ``replacement`` string. For example:: .. -Module Overview: -~~~~~~~~~~~~~~~~ +**Module Overview:** + :: - safelog + + bridgedb.safelog | - |_setSafeLogging - Enable or disable safelogging globally. - |_logSafely - Utility for manually sanitising a portion of a log message + |_ setSafeLogging - Enable or disable safelogging globally. + |_ logSafely - Utility for manually sanitising a portion of a log message | - |_BaseSafelogFilter - Base class for log message sanitisation filters - | |_doubleCheck - Optional stricter validation on matching substrings - | |_filter - Determine if some part of a log message should be filtered + \_ BaseSafelogFilter - Base class for log message sanitisation filters + | |_ doubleCheck - Optional stricter validation on matching substrings + | \_ filter - Determine if some part of a log message should be filtered | - |_SafelogEmailFilter - Filter for removing email addresses from logs - |_SafelogIPv6Filter - Filter for removing IPv4 addresses from logs - |_SafelogIPv6Filter - Filter for removing IPv6 addresses from logs -:: + |_ SafelogEmailFilter - Filter for removing email addresses from logs + |_ SafelogIPv4Filter - Filter for removing IPv4 addresses from logs + |_ SafelogIPv6Filter - Filter for removing IPv6 addresses from logs + +.. """ import functools @@ -81,24 +86,27 @@ def logSafely(string): class BaseSafelogFilter(logging.Filter): """Base class for creating log message sanitisation filters. - A :class:`BaseSafelogFilter` uses a compiled regex :cvar:`pattern` to + A :class:`BaseSafelogFilter` uses a compiled regex :attr:`pattern` to match particular items of data in log messages which should be sanitised (if ``SAFELOGGING`` is enabled in :file:`bridgedb.conf`). - .. note:: The ``pattern`` is used only for string *matching* purposes, and - *not* for validation. In other words, a ``pattern`` which matches email - addresses should simply match something which appears to be an email - address, even though that matching string might not technically be a - valid email address vis-á-vis :rfc:`5321`. + .. note:: + The :attr:`pattern` is used only for string *matching* purposes, and + *not* for validation. In other words, a :attr:`pattern` which matches + email addresses should simply match something which appears to be an + email address, even though that matching string might not technically + be a valid email address vis-á-vis :rfc:`5321`. - In addition, a ``BaseSafelogFilter`` uses a :cvar:`easyFind`, which is + In addition, a ``BaseSafelogFilter`` uses a :attr:`easyFind`, which is simply a string or character to search for before running checking against the regular expression, to attempt to avoid regexing *everything* which passes through the logger. :cvar pattern: A compiled regular expression, whose matches will be - scrubbed from log messages and replaced with :cvar:`replacement`. - :cvar easyFind: A simpler string to search for before regex matching. + scrubbed from log messages and replaced with :attr:`replacement`. + :vartype easyFind: str + :cvar easyFind: A simpler string to search for before to match by regex. + :vartype replacement: str :cvar replacement: The string to replace ``pattern`` matches with. (default: ``"[scrubbed]"``) """ @@ -110,9 +118,9 @@ class BaseSafelogFilter(logging.Filter): """Subclasses should override this function to implement any additional substring filtering to decrease the false positive rate, i.e. any additional filtering or validation which is *more* costly than - checking against the regular expression, :cvar:`pattern`. + checking against the regular expression, :attr:`pattern`. - To use only the :cvar:`pattern` matching in :meth:`filter`, and not + To use only the :attr:`pattern` matching in :meth:`filter`, and not use this method, simply do:: return True @@ -131,13 +139,12 @@ class BaseSafelogFilter(logging.Filter): """Filter a log record. The log **record** is filtered, and thus sanitised by replacing - matching substrings with the :cvar:`replacement` string, if the + matching substrings with the :attr:`replacement` string, if the following checks pass: - 0. ``SAFELOGGING`` is currently enabled. - 1. The ``record.msg`` string contains :cvar:`easyFind`. - 2. The ``record.msg`` matches the regular expression, - :cvar:`pattern`. + 1. ``SAFELOGGING`` is currently enabled. + 2. The ``record.msg`` string contains :attr:`easyFind`. + 3. The ``record.msg`` matches the regular expression, :attr:`pattern`. :type record: :class:`logging.LogRecord` :param record: Basically, anything passed to :func:`logging.log`. diff --git a/bridgedb/schedule.py b/bridgedb/schedule.py index 0adbff8..6cad2dc 100644 --- a/bridgedb/schedule.py +++ b/bridgedb/schedule.py @@ -8,7 +8,11 @@ # (c) 2014-2015, Isis Lovecruft # :license: see LICENSE for licensing information -"""This module implements functions for dividing time into chunks.""" +"""This module implements functions for dividing time into chunks. + +.. inheritance-diagram:: UnknownInterval Unscheduled ScheduledInterval + :parts: 1 +""" import calendar @@ -17,7 +21,7 @@ import math from datetime import datetime from zope import interface -from zope.interface import implements +from zope.interface import implementer from zope.interface import Attribute @@ -32,8 +36,9 @@ class UnknownInterval(ValueError): def toUnixSeconds(timestruct): """Convert a datetime struct to a Unix timestamp in seconds. - :param timestruct: A ``datetime.datetime`` object to convert into a - timestamp in Unix Era seconds. + :type timestruct: :any:`datetime.datetime` + :param timestruct: A ``datetime`` object to convert into a timestamp in + Unix Era seconds. :rtype: int """ return calendar.timegm(timestruct) @@ -42,13 +47,13 @@ def fromUnixSeconds(timestamp): """Convert a Unix timestamp to a datetime struct. :param int timestamp: A timestamp in Unix Era seconds. - :rtype: :type:`datetime.datetime` + :rtype: :any:`datetime.datetime` """ return datetime.fromtimestamp(timestamp) class ISchedule(interface.Interface): - """A ``Interface`` specification for a Schedule.""" + """An ``Interface`` specification for a Schedule.""" intervalPeriod = Attribute( "The type of period which this Schedule's intervals will rotate by.") @@ -65,6 +70,7 @@ class ISchedule(interface.Interface): """Get the start of the interval after the one containing **when**.""" +@implementer(ISchedule) class Unscheduled(object): """A base ``Schedule`` that has only one period that contains all time. @@ -88,12 +94,11 @@ class Unscheduled(object): '9999-12-31 23:59:59' """ - implements(ISchedule) def __init__(self, count=None, period=None): """Create a schedule for dividing time into intervals. - :param int count: The number of **period**s in an interval. + :param int count: The total number of **period** in one interval. :param str period: One of the periods in :data:`KNOWN_INTERVALS`. """ self.intervalCount = count @@ -136,6 +141,7 @@ class Unscheduled(object): return toUnixSeconds(datetime.max.timetuple()) +@implementer(ISchedule) class ScheduledInterval(Unscheduled): """An class that splits time into periods, based on seconds, minutes, hours, days, weeks, or months. @@ -166,16 +172,15 @@ class ScheduledInterval(Unscheduled): '2015-03-31 03:00:00' :ivar str intervalPeriod: One of the :data:`KNOWN_INTERVALS`. - :ivar int intervalCount: The number of times **intervalPeriod** should be - repeated within an interval. + :ivar int intervalCount: The number of times :attr:`intervalPeriod` should + be repeated within an interval. """ - implements(ISchedule) def __init__(self, count=None, period=None): """Create a schedule for dividing time into intervals. - :type count: int or str - :param count: The number of **period**s in an interval. + :type count: :any:`int` or :any:`str` + :param count: The total number of **period** in one interval. :param str period: One of the periods in :data:`KNOWN_INTERVALS`. """ super(ScheduledInterval, self).__init__(count, period) @@ -183,14 +188,14 @@ class ScheduledInterval(Unscheduled): self._setIntervalPeriod(period) def _setIntervalCount(self, count=None): - """Set our :ivar:`intervalCount`. + """Set our :attr:`intervalCount`. - .. attention:: This method should be called _before_ + .. attention:: This method should be called *before* :meth:`_setIntervalPeriod`, because the latter may change the count, if it decides to change the period (for example, to simplify things by changing weeks into days). - :param int count: The number of times the :ivar:`intervalPeriod` + :param int count: The number of times the :attr:`intervalPeriod` should be repeated during the interval. Defaults to ``1``. :raises UnknownInterval: if the specified **count** was invalid. """ @@ -204,7 +209,7 @@ class ScheduledInterval(Unscheduled): self.intervalCount = count def _setIntervalPeriod(self, period=None): - """Set our :ivar:`intervalPeriod`. + """Set our :attr:`intervalPeriod`. :param str period: One of the :data:`KNOWN_INTERVALS`, or its plural. Defaults to ``'hour'``. diff --git a/bridgedb/txrecaptcha.py b/bridgedb/txrecaptcha.py index 3666904..ed5891a 100644 --- a/bridgedb/txrecaptcha.py +++ b/bridgedb/txrecaptcha.py @@ -10,7 +10,7 @@ """Twisted-based reCAPTCHA client. This client *always* uses TLS with strict hostname checking, unlike the -official Google Python recaptcha-client_, which is harcoded_ to use plaintext +official Google Python recaptcha-client_, which is hardcoded_ to use plaintext HTTP. Small portions of this code were taken from the official Google Python @@ -20,6 +20,9 @@ which are copyright the authors of the recaptcha-client_ package. .. _hardcoded: https://code.google.com/p/recaptcha/source/browse/trunk/recaptcha-plugins/p… .. _recaptcha-client: https://pypi.python.org/pypi/recaptcha-client/1.0.6 + +.. inheritance-diagram:: RecaptchaResponseError RecaptchaResponse RecaptchaResponseProtocol + :parts: 1 """ import logging @@ -42,11 +45,11 @@ from zope.interface import implements from bridgedb.crypto import SSLVerifyingContextFactory -#: This was taken from recaptcha.client.captcha.API_SSL_SERVER. +#: This was taken from :data:`recaptcha.client.captcha.API_SSL_SERVER`. API_SSL_SERVER = API_SERVER = "https://www.google.com/recaptcha/api" API_SSL_VERIFY_URL = "%s/verify" % API_SSL_SERVER -#: (type: `OpenSSL.crypto.X509`) Only trust certificate for the reCAPTCHA +#: (:class:`OpenSSL.crypto.X509`) Only trust certificate for the reCAPTCHA #: :data:`API_SSL_SERVER` which were signed by the Google Internet Authority CA. GOOGLE_INTERNET_AUTHORITY_CA_CERT = load_certificate(FILETYPE_PEM, bytes("""\ -----BEGIN CERTIFICATE----- @@ -151,8 +154,9 @@ class RecaptchaResponseError(ValueError): class RecaptchaResponse(object): - """Taken from recaptcha.client.captcha.`RecaptchaResponse`_. - .. RecaptchaResponse: https://code.google.com/p/recaptcha/source/browse/trunk/recaptcha-plugins/p… + """Taken from `recaptcha.client.captcha.RecaptchaResponse`__. + + .. __: https://code.google.com/p/recaptcha/source/browse/trunk/recaptcha-plugins/p… """ def __init__(self, is_valid, error_code=None): self.is_valid = is_valid @@ -163,10 +167,12 @@ class RecaptchaResponseProtocol(protocol.Protocol): """HTML parser which creates a :class:`RecaptchaResponse` from the body of the reCaptcha API server's response. """ + def __init__(self, finished): - """Create a protocol for creating :class:`RecaptchaResponse`s. + """Create a protocol for creating + :class:`RecaptchaResponses <bridgedb.txrecaptcha.RecaptchaResponse>`. - :type finished: :api:`~twisted.internet.defer.Deferred` + :type finished: :api:`twisted.internet.defer.Deferred` :param finished: A deferred which will have its ``callback()`` called with a :class:`RecaptchaResponse`. """ @@ -175,7 +181,7 @@ class RecaptchaResponseProtocol(protocol.Protocol): self.response = '' def dataReceived(self, data): - """Called when some data is received from the connection.""" + """Called when some **data** is received from the connection.""" if self.remaining: received = data[:self.remaining] self.response += received @@ -187,8 +193,6 @@ class RecaptchaResponseProtocol(protocol.Protocol): :type reason: :api:`twisted.python.failure.Failure` :param reason: A string explaning why the connection was closed, wrapped in a ``Failure`` instance. - - :raises: A :api:`twisted.internet.error.ConnectError` if the """ valid = False error = reason.getErrorMessage() @@ -232,7 +236,7 @@ def _cbRequest(response): """Callback for a :api:`twisted.web.client.Agent.request` which delivers the result to a :class:`RecaptchaResponseProtocol`. - :returns: A :api:`~twisted.internet.defer.Deferred` which will callback + :returns: A :api:`twisted.internet.defer.Deferred` which will callback with a ``recaptcha.RecaptchaResponse`` for the request. """ finished = defer.Deferred() @@ -265,19 +269,20 @@ def submit(recaptcha_challenge_field, recaptcha_response_field, 1. It uses Twisted for everything. 2. It uses SSL/TLS for everything. - This function returns a :api:`~twisted.internet.defer.Deferred`. If you + This function returns a :api:`twisted.internet.defer.Deferred`. If you need a ``recaptcha.client.captcha.RecaptchaResponse`` to be returned, use the :func:`submit` function, which is an ``@inlineCallbacks`` wrapper for this function. :param str recaptcha_challenge_field: The value of the HTTP POST ``recaptcha_challenge_field`` argument from the form. - :param recaptcha_response_field: The value of the HTTP POST + :param str recaptcha_response_field: The value of the HTTP POST ``recaptcha_response_field`` argument from the form. - :param private_key: The reCAPTCHA API private key. - :param remoteip: An IP address to give to the reCaptcha API server. - :returns: A :api:`~twisted.internet.defer.Deferred` which will callback - with a ``recaptcha.RecaptchaResponse`` for the request. + :param str private_key: The reCAPTCHA API private key. + :param str remoteip: An IP address to give to the reCaptcha API server. + :rtype: :api:`twisted.internet.defer.Deferred` + :returns: A ``Deferred`` which will callback with a + ``recaptcha.RecaptchaResponse`` for the request. """ if not (recaptcha_response_field and len(recaptcha_response_field) and recaptcha_challenge_field and len(recaptcha_challenge_field)): diff --git a/doc/sphinx/ext/traclinks.py b/doc/sphinx/ext/traclinks.py index 75abce4..732295d 100644 --- a/doc/sphinx/ext/traclinks.py +++ b/doc/sphinx/ext/traclinks.py @@ -20,13 +20,22 @@ from docutils import nodes, utils def make_trac_link(name, rawtext, text, lineno, inliner, options={}, content=[]): + + # quick, dirty, and ugly... + if '<' in text and '>' in text: + full_name, label = text.split('<') + full_name = full_name.strip() + label = label.strip('>').strip() + else: + full_name = text + label = full_name + env = inliner.document.settings.env - trac_url = env.config.traclinks_base_url - ref = trac_url + urllib.quote(text, safe='') - node = nodes.reference(rawtext, - utils.unescape(text), - refuri=ref, - **options) + base_url = env.config.traclinks_base_url + label = utils.unescape('ticket #' + label) + ref = base_url + urllib.quote(full_name, safe='') + node = nodes.reference(rawtext, label, refuri=ref, **options) + return [node],[] diff --git a/doc/sphinx/source/bridgedb.distribute.rst b/doc/sphinx/source/bridgedb.distribute.rst new file mode 100644 index 0000000..aeb93ae --- /dev/null +++ b/doc/sphinx/source/bridgedb.distribute.rst @@ -0,0 +1,8 @@ +bridgedb.distribute +------------------- + +.. automodule:: bridgedb.distribute + :members: + :undoc-members: + :private-members: + :show-inheritance: diff --git a/doc/sphinx/source/bridgedb.rst b/doc/sphinx/source/bridgedb.rst index 207313c..ea04ef4 100644 --- a/doc/sphinx/source/bridgedb.rst +++ b/doc/sphinx/source/bridgedb.rst @@ -1,6 +1,6 @@ -BridgeDB Package and Module Documentation -========================================= +Packages & Modules +==================== .. .. currentmodule:: bridgedb .. autosummary:: @@ -12,6 +12,7 @@ BridgeDB Package and Module Documentation bridgedb.Bucket bridgedb.captcha bridgedb.configure + bridgedb.distribute bridgedb.crypto bridgedb.email bridgedb.filters diff --git a/doc/sphinx/source/conf.py b/doc/sphinx/source/conf.py index a4dba2c..f1f5fa4 100644 --- a/doc/sphinx/source/conf.py +++ b/doc/sphinx/source/conf.py @@ -33,6 +33,7 @@ import bridgedb.captcha import bridgedb.Bridges import bridgedb.Bucket import bridgedb.crypto +import bridgedb.distribute import bridgedb.email import bridgedb.email.autoresponder import bridgedb.email.distributor @@ -109,6 +110,7 @@ needs_sphinx = '1.1' extensions = ['sphinx.ext.autodoc', 'sphinx.ext.autosummary', 'sphinx.ext.doctest', + 'sphinx.ext.inheritance_diagram', 'sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx.ext.coverage', @@ -123,6 +125,39 @@ extensions = ['sphinx.ext.autodoc', todo_include_todos = True +# See http://sphinx-doc.org/ext/inheritance.html for configuring the style of +# inheritance diagrams. An inheritance diagram can be generated via doing: +# +# class Bar(object): +# pass +# +# class Foo(Bar): +# """A foobar. +# +# .. inheritance-diagram:: +# somemodule.Foo +# """ +inheritance_graph_attrs = { + 'rankdir': "LR", + 'size': '"9.0, 12.0"', + 'fontsize': 14, + 'ratio': 'auto', + 'splines': 'ortho', +} +inheritance_node_attrs = { + 'shape': 'ellipse', + 'fontsize': 14, + 'height': 0.75, + 'color': 'mediumaquamarine', + 'style': 'filled', + 'pencolor': 'black', +} +inheritance_edge_attrs = { + 'arrowsize': 1.5, + 'arrowhead': 'open', + 'penwidth': 1.5, +} + # Add any paths that contain templates here, relative to this directory. templates_path = ['source/.templates'] @@ -401,5 +436,8 @@ epub_uid = 'BridgeDB Documentation (ePub)' + 'v' + version # Allow duplicate toc entries. #epub_tocdup = True -# Example configuration for intersphinx: refer to the Python standard library. -intersphinx_mapping = {'http://docs.python.org/': None} +# See http://sphinx-doc.org/ext/intersphinx.html for details of this setting. +intersphinx_mapping = { + 'python': ('https://docs.python.org/', None), + 'stem': ('https://stem.torproject.org/', None), +} diff --git a/doc/sphinx/source/index.rst b/doc/sphinx/source/index.rst index 02f50ad..857aefe 100644 --- a/doc/sphinx/source/index.rst +++ b/doc/sphinx/source/index.rst @@ -3,41 +3,47 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Welcome to BridgeDB's documentation! -==================================== +BridgeDB developer documentation +=============================================== .. image:: _static/bay-bridge.jpg :align: center -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Tor Bridge Descriptor Formats: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^ +BridgeDB's README +^^^^^^^^^^^^^^^^^ .. toctree:: - :maxdepth: 4 + :maxdepth: 3 + + readme - descriptors +^^^^^^^^^^^^^^^^^^^^^ +Packages & Modules +^^^^^^^^^^^^^^^^^^^^^ -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -BridgeDB Package and Module Documentation: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. toctree:: :maxdepth: 3 bridgedb -^^^^^^^^^^^^^^^^^ -BridgeDB's README -^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Tor Bridge Descriptor Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Please also see the `CollecTor`__ documentation on Tor Bridge descriptor formats. + +__ https://collector.torproject.org/formats.html#bridge-descriptors + .. toctree:: - :maxdepth: 3 + :maxdepth: 4 - readme + descriptors -Help Us Develop BridgeDB! +Help Develop BridgeDB! ========================= To see all open tickets for BridgeDB, please `visit the Tor Project's Trac`__ instance. @@ -48,13 +54,14 @@ __ https://trac.torproject.org/projects/tor/query?status=!closed&component=Bri… Todo List ========= -.. todolist:: - +This TODO list is generated in an automated manner via the +`Sphinx TODO extension`__, and it only includes portions documentation string +within BridgeDB's codebase which have been marked with the Sphinx ``.. todo:`` +directive. As such, this should be taken neither as a canonical nor complete +list of tasks planned for future development, but rather a brief list of +small(ish) places within the codebase that could use some attention. -Indices and tables -================== +__ http://sphinx-doc.org/ext/todo.html#directive-todo -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` +.. todolist::

1 0

[bridgedb/develop] Update .gitignore location for bridgedb/_langs.py.
by isis＠torproject.org 21 Jul '15

21 Jul '15

commit 5607297e47263abee5b2e0162e8d6d3ba7fbc9d7 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon Jul 20 19:55:49 2015 +0000 Update .gitignore location for bridgedb/_langs.py. --- .gitignore | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index f208f3e..24b09e6 100644 --- a/.gitignore +++ b/.gitignore @@ -81,9 +81,9 @@ run/* *blocked-bridges *extra-infos -# do not recommit lib/bridgedb/_langs.py: +# do not recommit bridgedb/_langs.py: # (the "supported" variable is supposed to be rewritten at install time) -lib/bridgedb/_langs.py +bridgedb/_langs.py # ignore persistent state files *bridgedb.state*

1 0

[bridgedb/develop] Merge branch 'hotfix/0.3.2-remove-lib-dir' into develop
by isis＠torproject.org 21 Jul '15

21 Jul '15

commit fbd8121449ddefb6daa4910c6779c781d07c3390 Merge: 300bc11 5607297 Author: Isis Lovecruft <isis(a)torproject.org> Date: Mon Jul 20 19:56:51 2015 +0000 Merge branch 'hotfix/0.3.2-remove-lib-dir' into develop .gitignore | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

1 0