January 2014 - tor-commits - lists.torproject.org

[bridgedb/master] Parse bridge descriptors in a separate thread.
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit 8f4cad05500bfc3113713535aa494f8ef5ca878d Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Nov 15 16:13:50 2013 +0000 Parse bridge descriptors in a separate thread. TODO: In the future, we will probably want to add some secondary, temporary HTTPServer which serves a simple "BridgeDB is down for maintenance and will be back shortly!" page while the descriptors are being parsed. * FIXES #5232: Import bridges into BridgeDB in a separate thread and database transaction. --- lib/bridgedb/Main.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/bridgedb/Main.py b/lib/bridgedb/Main.py index 3729a17..5e64599 100644 --- a/lib/bridgedb/Main.py +++ b/lib/bridgedb/Main.py @@ -495,6 +495,8 @@ def startup(options, rundir, configFile): logging.debug("Saving state again before reparsing descriptors...") state.save() + logging.info("Reparsing bridge descriptors...") + reactor.callInThread(load, state, splitter, clear=False) state = persistent.load() logging.info("Bridges loaded: %d" % len(splitter))

1 0

[bridgedb/master] Obtain the fingerprint and add it to the documents/descriptors
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit 33dcd5bed7aad5dd6f37d53cca9afc7baf371b75 Author: Matthew Finkel <Matthew.Finkel(a)gmail.com> Date: Sat Nov 9 17:23:07 2013 +0000 Obtain the fingerprint and add it to the documents/descriptors (cherry picked from commit b4357fe77284d9858a1c71be649cd6cc157002fd) Signed-off-by: Isis Lovecruft <isis(a)torproject.org> Conflicts: scripts/gen_bridge_descriptors Again, I had to change a few things due to Matt and I both trying to fix something, and both simultaneously catching problems that the other person missed. :) --- scripts/gen_bridge_descriptors | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/scripts/gen_bridge_descriptors b/scripts/gen_bridge_descriptors index da25e59..66f07ab 100644 --- a/scripts/gen_bridge_descriptors +++ b/scripts/gen_bridge_descriptors @@ -745,8 +745,6 @@ def generateDescriptors(): idkey_private = getPEMPrivateKey(SIDSKey) idkey_digest = hashlib.sha1(idkey_private).digest() - fpr = convertToSpaceyFingerprint(fingerprint) - idkey_public = OpenSSL.crypto.dump_privatekey(ASN1, SIDPCert.get_pubkey()) idkey_public = re.sub(OPENSSL_BEGIN_KEY, '', idkey_public) @@ -754,11 +752,13 @@ def generateDescriptors(): idkey_public = idkey_public.strip() ident_digest = hashlib.sha1(idkey_public).digest() + fingerprint = hashlib.sha1(idkey_public).hexdigest().upper() + fpr = convertToSpaceyFingerprint(ident_digest) identity = binascii.b2a_base64( hashlib.sha1(idkey_public).digest()).strip().strip('=======') - extrainfo_document = generateExtraInfo(fpr, timestamp, ipv4, port) + extrainfo_document = generateExtraInfo(fingerprint, timestamp, ipv4, port) extrainfo_digest = hashlib.sha1(extrainfo_document).digest() extrainfo_hexdigest = hashlib.sha1(extrainfo_document).hexdigest().upper() extrainfo_sig = signDescriptorDigest(idkey, extrainfo_digest) @@ -782,7 +782,7 @@ def generateDescriptors(): server_desc = '\n'.join(server) server_desc_digest = hashlib.sha1(server_desc).digest() - netstatus_desc = generateNetstatus(idkey_digest, server_desc_digest, + netstatus_desc = generateNetstatus(ident_digest, server_desc_digest, timestamp, ipv4, port, ipv6=ipv6, bandwidth_line=bandwidth) server_desc += signDescriptorDigest(idkey, server_desc_digest)

1 0

[bridgedb/master] Update the docstring for idkey_digest in generateNetstatus
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit c35e5ed498fe2b4808ac0b9a991a759827e128e1 Author: Matthew Finkel <Matthew.Finkel(a)gmail.com> Date: Sat Nov 9 17:15:47 2013 +0000 Update the docstring for idkey_digest in generateNetstatus (cherry picked from commit 65804cd121edc2e140e1e01d6e9762d5d4c9c2d4) Signed-off-by: Isis Lovecruft <isis(a)torproject.org> --- scripts/gen_bridge_descriptors | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/gen_bridge_descriptors b/scripts/gen_bridge_descriptors index 019e1e3..da25e59 100644 --- a/scripts/gen_bridge_descriptors +++ b/scripts/gen_bridge_descriptors @@ -623,7 +623,7 @@ def generateNetstatus(idkey_digest, server_desc_digest, timestamp, DOCDOC - :param XXX idkey_digest: The SHA-1 digest of the router's public identity + :param string idkey_digest: The SHA-1 digest of the router's public identity key. :param XXX server_desc_digest: The SHA-1 digest of the router's ``@type [bridge-]server-descriptor``, before the descriptor is signed.

1 0

[bridgedb/master] I didn't know where to find it, maybe others won't either.
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit b2722a9661812f6cf433441ae659bccdbfbf0174 Author: Matthew Finkel <Matthew.Finkel(a)gmail.com> Date: Sat Nov 9 17:24:36 2013 +0000 I didn't know where to find it, maybe others won't either. (cherry picked from commit fceee92ad07f3390eab7504ba6d86393b57aeda0) Signed-off-by: Isis Lovecruft <isis(a)torproject.org> --- scripts/make-ssl-cert | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/make-ssl-cert b/scripts/make-ssl-cert index 6f4772c..c055fa7 100755 --- a/scripts/make-ssl-cert +++ b/scripts/make-ssl-cert @@ -54,3 +54,4 @@ openssl x509 -req -days 365 -in server.csr -signkey privkey.pem -out cert test -f "privkey.nopasswd" && rm -f privkey.nopasswd test -f "privkey" && rm -f privkey test -f "server.csr" && rm -f server.csr +echo "Done. Please retrieve the files from $REPO_PATH"

1 0

[bridgedb/master] Fix a bug due to misnamed variable RUNDIR → RUN_IN_DIR.
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit 4a3a7b60ddcd3f65b6f2190b7db388ba70ed6830 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Nov 15 17:39:29 2013 +0000 Fix a bug due to misnamed variable RUNDIR → RUN_IN_DIR. --- lib/bridgedb/Main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bridgedb/Main.py b/lib/bridgedb/Main.py index 5e64599..0ed335a 100644 --- a/lib/bridgedb/Main.py +++ b/lib/bridgedb/Main.py @@ -588,7 +588,7 @@ def runSubcommand(options, config): if 'descriptors' in options.subOptions: runner.generateDescriptors(options.subOptions['descriptors'], - config.RUNDIR) + config.RUN_IN_DIR) if options.subCommand == 'test': if options.subOptions['trial']:

1 0

[bridgedb/master] Save config.RUN_IN_DIR, not state.RUN_IN_DIR, so subCommand can access it.
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit 44db6ffd1f9f4d072b80ddb5021ef05c60bca567 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Nov 15 17:48:16 2013 +0000 Save config.RUN_IN_DIR, not state.RUN_IN_DIR, so subCommand can access it. If we save the runtime directory location later in Main.setup() when the persistent.State instance is created, then any options.subCommands which run earlier cannot access it. --- lib/bridgedb/Main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bridgedb/Main.py b/lib/bridgedb/Main.py index 0ed335a..fa2e3bf 100644 --- a/lib/bridgedb/Main.py +++ b/lib/bridgedb/Main.py @@ -351,6 +351,7 @@ def startup(options, rundir, configFile): os.chdir(rundir) config = loadConfig(configFile) + config.RUN_IN_DIR = rundir # Set up logging as early as possible. We cannot import from the bridgedb # package any of our modules which import :mod:`logging` and start using @@ -376,7 +377,6 @@ def startup(options, rundir, configFile): from bridgedb import persistent state = persistent.State(config=config) - state.RUN_IN_DIR = rundir from bridgedb import EmailServer from bridgedb import HTTPServer

1 0

[bridgedb/master] Don't add the ORAddresses from a networkstatus if an exception was raised.
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit fcaeca998f4a5c5b98a5bd38eb1b66924dbc64ca Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Nov 15 17:44:22 2013 +0000 Don't add the ORAddresses from a networkstatus if an exception was raised. --- lib/bridgedb/Bridges.py | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/lib/bridgedb/Bridges.py b/lib/bridgedb/Bridges.py index 6903ce6..59fe38b 100644 --- a/lib/bridgedb/Bridges.py +++ b/lib/bridgedb/Bridges.py @@ -513,10 +513,9 @@ def parseStatusFile(networkstatusFile): except networkstatus.ParseNetstatusError as error: logging.error(error.message) else: - - try: or_addresses[addr].add(portlist) - except KeyError: or_addresses[addr] = portlist - parsedORAddressLines += 1 + try: or_addresses[addr].add(portlist) + except KeyError: or_addresses[addr] = portlist + parsedORAddressLines += 1 elif ID and timestamp and line.startswith("s "): running, stable = networkstatus.parseSLine(line)

1 0

[bridgedb/master] Remove two rather excessive log statements from Bridges module.
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit 97a5ade0d06966beb6aca203034fd9c8e9a8d850 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Nov 15 19:27:39 2013 +0000 Remove two rather excessive log statements from Bridges module. --- lib/bridgedb/Bridges.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/lib/bridgedb/Bridges.py b/lib/bridgedb/Bridges.py index 59fe38b..579dd4f 100644 --- a/lib/bridgedb/Bridges.py +++ b/lib/bridgedb/Bridges.py @@ -440,7 +440,6 @@ def parseExtraInfoFile(f): # get the bridge ID ? if line.startswith("extra-info "): #XXX: get the router ID - logging.debug("Parsing extra-info line") line = line[11:] (nickname, ID) = line.split() logging.debug(" Parsed Nickname: %s", nickname) @@ -452,7 +451,6 @@ def parseExtraInfoFile(f): # get the transport line if ID and line.startswith("transport "): - logging.debug(" Parsing transport line") fields = line[10:].split() # [ arglist ] field, optional if len(fields) >= 3:

1 0

[bridgedb/master] Combine message string for two raised exceptions.
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit d62a66ada1591695cfad2504437f622d4acc5535 Author: Isis Lovecruft <isis(a)torproject.org> Date: Sat Nov 16 00:33:09 2013 +0000 Combine message string for two raised exceptions. --- lib/bridgedb/parse/networkstatus.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/bridgedb/parse/networkstatus.py b/lib/bridgedb/parse/networkstatus.py index d38257d..9b1bb01 100644 --- a/lib/bridgedb/parse/networkstatus.py +++ b/lib/bridgedb/parse/networkstatus.py @@ -92,8 +92,8 @@ def parseRLine(line): if ID.endswith('='): raise InvalidNetworkstatusRouterIdentity( - "Skipping networkstatus parsing for router with nickname %r: ", - "Unpadded, base64-encoded networkstatus router identity ", + "Skipping networkstatus parsing for router with nickname %r:"\ + "\n\tUnpadded, base64-encoded networkstatus router identity "\ "string ends with '=': %r" % (nickname, ID)) try: ID = padBase64(ID) # Add the trailing equals sign back in @@ -103,9 +103,9 @@ def parseRLine(line): ID = binascii.a2b_base64(ID) if not ID: raise InvalidNetworkstatusRouterIdentity( - "Skipping networkstatus parsing for router with nickname %r: ", - "Base64-encoding for networkstatus router identity string is ", - "invalid! Line: %r" % (nickname, line)) + "Skipping networkstatus parsing for router with nickname %r:"\ + "\n\tBase64-encoding for networkstatus router identity string"\ + "is invalid!\n\tLine: %r" % (nickname, line)) except IndexError as error: logging.error(error.message)

1 0

[bridgedb/master] Separate lines for lists of updated and new config values.
by isis＠torproject.org 12 Jan '14

12 Jan '14

commit 3d842b7ede774cbc3f80d67ca6fe0142f22a862e Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Nov 15 17:52:04 2013 +0000 Separate lines for lists of updated and new config values. --- lib/bridgedb/persistent.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/bridgedb/persistent.py b/lib/bridgedb/persistent.py index 79f1d70..6c9ebea 100644 --- a/lib/bridgedb/persistent.py +++ b/lib/bridgedb/persistent.py @@ -238,7 +238,8 @@ class State(jelly.Jellyable): any settings which were changed to be attributes of the :class:`State` instance. """ - updated = new = [] + updated = [] + new = [] for key, value in config.__dict__.items(): try:

1 0