April 2014 - tor-commits - lists.torproject.org

[oonib/master] try/except is not needed here
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 4fb2c669147b602e9e126c5eea112b7f0aa1792f Author: Darius Bacon <darius(a)wry.me> Date: Wed Apr 16 12:22:42 2014 -0700 try/except is not needed here --- oonib/report/handlers.py | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/oonib/report/handlers.py b/oonib/report/handlers.py index f1fec4a..34aa8d9 100644 --- a/oonib/report/handlers.py +++ b/oonib/report/handlers.py @@ -305,10 +305,7 @@ class NewReportHandlerFile(OONIBHandler, UpdateReportMixin): 'report_id': report_id } - try: - requested_helper = report_data['test_helper'] - except KeyError: - requested_helper = None + requested_helper = report_data.get('test_helper') if requested_helper: try:

1 0

[oonib/master] remove mutable class-level variable that's always overridden by a fresh instance variable with the same name (at least within this code, and I'm going to assume that's how it's meant to be used)
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 64615d89908b18469281d84b2cbe70b7a3a110d8 Author: Darius Bacon <darius(a)wry.me> Date: Wed Apr 16 12:28:59 2014 -0700 remove mutable class-level variable that's always overridden by a fresh instance variable with the same name (at least within this code, and I'm going to assume that's how it's meant to be used) --- oonib/testhelpers/http_helpers.py | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/oonib/testhelpers/http_helpers.py b/oonib/testhelpers/http_helpers.py index 3a76b9a..8f09326 100644 --- a/oonib/testhelpers/http_helpers.py +++ b/oonib/testhelpers/http_helpers.py @@ -40,11 +40,11 @@ class SimpleHTTPChannel(basic.LineReceiver, policies.TimeoutMixin): length = 0 maxHeaders = 500 requestLine = '' - headers = [] timeOut = 60 * 60 * 12 def __init__(self): + self.headers = [] self.requests = [] def connectionMade(self): @@ -98,9 +98,7 @@ class SimpleHTTPChannel(basic.LineReceiver, policies.TimeoutMixin): class HTTPReturnJSONHeadersHelper(protocol.ServerFactory): protocol = SimpleHTTPChannel def buildProtocol(self, addr): - p = self.protocol() - p.headers = [] - return p + return self.protocol() class HTTPTrapAll(RequestHandler): def _execute(self, transforms, *args, **kwargs):

1 0

[oonib/master] Handle writing exceptions that don't have log_exception attribute
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 0148e9951a7fc7505becd2bfb47e3fcae6c05533 Author: Arturo Filastò <art(a)fuffa.org> Date: Mon Apr 21 19:37:10 2014 +0200 Handle writing exceptions that don't have log_exception attribute --- oonib/handlers.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/oonib/handlers.py b/oonib/handlers.py index 2231a2a..20c831b 100644 --- a/oonib/handlers.py +++ b/oonib/handlers.py @@ -6,8 +6,10 @@ from cyclone import web class OONIBHandler(web.RequestHandler): def write_error(self, status_code, exception=None, **kw): self.set_status(status_code) - if exception: + if hasattr(exception, 'log_message'): self.write({'error': exception.log_message}) + else: + self.write({'error': 'error'}) def write(self, chunk): """

1 0

[oonib/master] Merge branch 'darius'
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 7b35b7fa5a3d81f6574c7494cde0ac705d4e2cea Merge: 873b193 168a7c9 Author: Arturo Filastò <art(a)fuffa.org> Date: Mon Apr 21 19:42:16 2014 +0200 Merge branch 'darius' * darius: Handle writing exceptions that don't have log_exception attribute Fix bug introduce in previous PR. remove mutable class-level variable that's always overridden by a fresh instance variable with the same name (at least within this code, and I'm going to assume that's how it's meant to be used) try/except is not needed here remove overbroad try/except tighter try/except/with blocks tighter try/except/with blocks move write() out of the scope of an IOError catch clause not meant for it a more-direct test note missing import remove duplicate import clearer logic by eliminating checking_first_packet and bringing assignments to the same variable closer together remove unused (and confusing) global simpler Config.load() fix doctest (python -m doctest __init__.py was failing) simpler methods in Storage clearer check_paths() Conflicts: oonib/config.py oonib/config.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)

1 0

[oonib/master] Merge branch 'master' of github.com:TheTorProject/ooni-backend
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 873b193ad40a625e58cdf3004600d0e40d61ac6c Merge: 0148e99 207e4b0 Author: Arturo Filastò <art(a)fuffa.org> Date: Mon Apr 21 19:39:37 2014 +0200 Merge branch 'master' of github.com:TheTorProject/ooni-backend * 'master' of github.com:TheTorProject/ooni-backend: If p is a dictionary (as I assume) then you don't need the .keys(). Avoid the possibility of accidentally sharing the same mutable list for self.input and self.nettest. Most of the body of this doesn't need to be in the scope of the 'with'. (This comes up in lots of the other 'with's in this codebase.) More-direct code for validateNettest, as long as I'm messing with it. You could do the same to validateInputHash. Make validateNettest work like validateInputHash. Previously the code was clearly redundant and thus probably wrong. Export ssl_helpers. (Or was it meant to be left out?) Load the bouncer file just once. The previous code read it twice, which could create inconsistent datastructures if the file got edited at the wrong time. Fix: old code could raise a KeyError because choices[c] preceded the if c in choices Check version semantically instead of by string (the old code answers wrong for '0.10.0'). fix 'false midnight' bug (https://lwn.net/Articles/590299/ for an explanation) Apply the correct regular expression to the test helper key. Make the CC regexp also more accurate Add $ to the end of regexps that should have it Remove unused duplicate of updateKnownHelpers Fix InvalidArchiveDirectory exception message Conflicts: oonib/bouncer/handlers.py oonib/config.py oonib/config.py | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --cc oonib/config.py index 1388e6b,956fcc1..f0fc071 --- a/oonib/config.py +++ b/oonib/config.py @@@ -37,14 -39,17 +37,11 @@@ class Config(object) self.check_paths() def check_paths(self): - def check_path(directory, complaint): - if not (directory and os.path.isdir(directory)): - raise complaint(directory) - check_path(self.main.report_dir, e.InvalidReportDirectory) - check_path(self.main.archive_dir, e.InvalidArchiveDirectory) - check_path(self.main.input_dir, e.InvalidInputDirectory) - check_path(self.main.deck_dir, e.InvalidDeckDirectory) + if not self.main.report_dir or not os.path.isdir(self.main.report_dir): + raise e.InvalidReportDirectory(self.main.report_dir) + if not self.main.archive_dir or not os.path.isdir(self.main.archive_dir): + raise e.InvalidArchiveDirectory(self.main.archive_dir) - if self.main.input_dir and not os.path.isdir(self.main.input_dir): - raise e.InvalidInputDirectory(self.main.input_dir) - if self.main.deck_dir and not os.path.isdir(self.main.deck_dir): - raise e.InvalidDeckDirectory(self.main.deck_dir) - backend_version = __version__ -reports = {} config = Config()

1 0

[oonib/master] Add $ to the end of regexps that should have it
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 0e7fb5824badb2e469a291fc9f1247ae31b51f01 Author: Taylor Hornby <havoc(a)defuse.ca> Date: Thu Mar 27 14:30:26 2014 -0600 Add $ to the end of regexps that should have it --- bin/archive_oonib_reports | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bin/archive_oonib_reports b/bin/archive_oonib_reports index 3c238af..7d5ecfd 100755 --- a/bin/archive_oonib_reports +++ b/bin/archive_oonib_reports @@ -50,13 +50,13 @@ def validate_fields(fields): # or not we should support > 2 character CC if fields['probe_cc'] is None: fields['probe_cc'] = default_probe_cc - if not re.match('[A-Z\?]{2,4}', fields['probe_cc'].upper()): + if not re.match('[A-Z\?]{2,4}$', fields['probe_cc'].upper()): raise InvalidReportField('probe_cc') # check report ASN if fields['probe_asn'] is None: fields['probe_asn'] = 'AS0' - if not re.match('^AS[0-9]{1,10}', fields['probe_asn'].upper()): + if not re.match('^AS[0-9]{1,10}$', fields['probe_asn'].upper()): raise InvalidReportField('probe_asn') # check report timestamp

1 0

[oonib/master] Remove unused duplicate of updateKnownHelpers
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 1163889a20b59201c665d0d4c5600d990cc57296 Author: Taylor Hornby <havoc(a)defuse.ca> Date: Thu Mar 27 11:17:01 2014 -0600 Remove unused duplicate of updateKnownHelpers --- oonib/bouncer/handlers.py | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/oonib/bouncer/handlers.py b/oonib/bouncer/handlers.py index 38ac1f5..47fee26 100644 --- a/oonib/bouncer/handlers.py +++ b/oonib/bouncer/handlers.py @@ -115,19 +115,6 @@ class BouncerQueryHandler(OONIBHandler): def initialize(self): self.bouncer = Bouncer() - def updateKnownHelpers(self): - with open(config.main.bouncer_file) as f: - bouncerFile = yaml.safe_load(f) - for collectorName, helpers in bouncerFile['collector'].items(): - for helperName, helperAddress in helpers['test-helper'].items(): - if helperName not in self.knownHelpers.keys(): - self.knownHelpers[helperName] = [] - - self.knownHelpers[helperName].append({ - 'collector-name': collectorName, - 'helper-address': helperAddress - }) - def post(self): try: query = json.loads(self.request.body)

1 0

[oonib/master] Merge pull request #39 from defuse/remove_dup_code
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 3a9dc153368ce5755fe80883e2d0074420616740 Merge: 73de182 1163889 Author: Arturo Filastò <hellais(a)users.noreply.github.com> Date: Thu Mar 27 22:07:31 2014 +0100 Merge pull request #39 from defuse/remove_dup_code Remove unused duplicate of updateKnownHelpers oonib/bouncer/handlers.py | 13 ------------- 1 file changed, 13 deletions(-)

1 0

[oonib/master] Merge pull request #38 from defuse/fix_error_message
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 73de18215aaeed5072b82b01a8f9ace0561421f2 Merge: 755588b 773dbe5 Author: Arturo Filastò <hellais(a)users.noreply.github.com> Date: Thu Mar 27 22:06:23 2014 +0100 Merge pull request #38 from defuse/fix_error_message Fix InvalidArchiveDirectory exception message oonib/config.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

1 0

[oonib/master] Make the CC regexp also more accurate
by art＠torproject.org 23 Apr '14

23 Apr '14

commit 9958da22a3f6001e6e517c0a1e0c905cdd6c6ef8 Author: Arturo Filastò <art(a)fuffa.org> Date: Thu Mar 27 22:10:07 2014 +0100 Make the CC regexp also more accurate --- bin/archive_oonib_reports | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/archive_oonib_reports b/bin/archive_oonib_reports index 7d5ecfd..ba4d473 100755 --- a/bin/archive_oonib_reports +++ b/bin/archive_oonib_reports @@ -50,7 +50,7 @@ def validate_fields(fields): # or not we should support > 2 character CC if fields['probe_cc'] is None: fields['probe_cc'] = default_probe_cc - if not re.match('[A-Z\?]{2,4}$', fields['probe_cc'].upper()): + if not re.match('^[A-Z\?]{2,4}$', fields['probe_cc'].upper()): raise InvalidReportField('probe_cc') # check report ASN

1 0