February 2015 - tor-commits - lists.torproject.org

[translation/https_everywhere] Update translations for https_everywhere
by translation＠torproject.org 04 Feb '15

04 Feb '15

commit 0b4da69c23698adcaa77cc399ab2fb280af42796 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Feb 4 21:15:13 2015 +0000 Update translations for https_everywhere --- pt_BR/ssl-observatory.dtd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pt_BR/ssl-observatory.dtd b/pt_BR/ssl-observatory.dtd index 371105d..02d3a20 100644 --- a/pt_BR/ssl-observatory.dtd +++ b/pt_BR/ssl-observatory.dtd @@ -26,7 +26,7 @@ to turn it on?">--> "Seguro, a não ser que você utilize uma rede corporativa cujos provedores de intranet possuam nomes secretos:"> <!ENTITY ssl-observatory.prefs.alt_roots -"Apresentar e verificar os certificados assinados pelos Certificados de Autoridade (CA) de raiz não-padronizados"> +"Apresentar e verificar os certificados que tiverem sido assinados por Certificados de Autoridade (CA) de raiz não-padronizados"> <!ENTITY ssl-observatory.prefs.alt_roots_tooltip "É seguro (e uma boa ideia) habilitar esta opção, a não ser que você utilize uma rede corporativa intrusiva ou o programa de antivírus Kapersky, que monitora a sua navegação com um proxy TLS e um Certificado de Autoridade, cuja raiz é privada. Se habilitada em uma rede deste tipo, por causa dos certificados únicos que seriam produzidos, esta opção poderia publicar provas do domínios https:// que tiverem sido visitados através daquele proxy. Por isso, em padrão, nós a desabilitamos.">

1 0

[webwml/master] Add David as a mentor for tor projects
by atagar＠torproject.org 04 Feb '15

04 Feb '15

commit 186ff6fade2a7f2312b466bd3581681d22a08aec Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Feb 4 09:10:08 2015 -0800 Add David as a mentor for tor projects --- getinvolved/en/volunteer.wml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/getinvolved/en/volunteer.wml b/getinvolved/en/volunteer.wml index c687d94..d1ea4b4 100644 --- a/getinvolved/en/volunteer.wml +++ b/getinvolved/en/volunteer.wml @@ -950,7 +950,7 @@ meetings around the world.</li> Skill Level: Medium to High - Likely Mentors: Nick (nickm) + Likely Mentors: Nick (nickm), David (dgoulet) The Tor code is more than 10 years old in places, and we haven't always had enough time or wisdom to write things as well as we could have. Our unit @@ -1275,7 +1275,7 @@ configurations is a good first step to understanding this problem. Skill Level: Medium - Likely Mentors: Nick (nickm) + Likely Mentors: Nick (nickm), David (dgoulet) Right now, our unit test coverage with the tests we ship is around 30% -- only 30% of the executable lines in our source are reached by the @@ -1341,7 +1341,7 @@ easy to test.) Skill Level: Medium - Likely Mentors: Nick (nickm) + Likely Mentors: Nick (nickm), David (dgoulet) Right now, if you run a busy Tor server on a multicore computer, most of the cores are mostly unused. We have a "cpuworker" mechanism to move @@ -1391,7 +1391,7 @@ the codebase that you want to work on. Skill Level: Medium - Likely Mentors: Nick (nickm) + Likely Mentors: Nick (nickm), David (dgoulet) We're working on a revamp of the entire Tor hidden service design to improve the security and reliability of the hidden service system. @@ -1428,7 +1428,7 @@ the codebase that you want to work on. Skill Level: Medium - Likely Mentors: Nick (nickm) + Likely Mentors: Nick (nickm), David (dgoulet) Right now, you can only use Tor's DNS support to look up IPv4 and IPv6 addresses, and to fetch PTR records. But DNS can do so much more! @@ -1464,7 +1464,7 @@ the codebase that you want to work on. Skill Level: Medium - Likely Mentors: Nick (nickm) + Likely Mentors: Nick (nickm), David (dgoulet) The seccomp2 mechanism on Linux lets programs improve their robustness against unforseen bugs by running with restrictions on which system

1 0

[webwml/master] Notice on 'Develop a Censorship Analyzer' to make a contribution
by atagar＠torproject.org 04 Feb '15

04 Feb '15

commit ce43dfb442549b760a2ffc618c98862d499d748c Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Feb 4 09:00:27 2015 -0800 Notice on 'Develop a Censorship Analyzer' to make a contribution Addition requested by phw. --- getinvolved/en/volunteer.wml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/getinvolved/en/volunteer.wml b/getinvolved/en/volunteer.wml index 4e8a7a8..c687d94 100644 --- a/getinvolved/en/volunteer.wml +++ b/getinvolved/en/volunteer.wml @@ -1131,6 +1131,13 @@ mean getting familiar with <a href="https://ooni.torproject.org">OONI</a> and <a href="http://twistedmatrix.com/trac/">Twisted</a>. After that, the tool should be implemented as a number of OONI-specific networking tests. + + +Applicants should be familiar with Python and asynchronous programming, +e.g., as it is used in Twisted. As part of your application for this +project please contribute a patch for a bug in <a +href="https://trac.torproject.org/projects/tor/query?status=!closed&component=Ooni">OONI</a>. + </li> <a id="makeTorbirdyBetter"></a>

1 0

[guardfraction/master] Fix tests that broke from adding more lines to the output file.
by asn＠torproject.org 04 Feb '15

04 Feb '15

commit 47dad84c9e0ca6a77462d1effb5d013572883068 Author: George Kadianakis <desnacked(a)riseup.net> Date: Wed Feb 4 13:37:42 2015 +0000 Fix tests that broke from adding more lines to the output file. --- guardiness/guard_ds.py | 4 +++- test/test_guardfraction.py | 18 +++++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/guardiness/guard_ds.py b/guardiness/guard_ds.py index 19e8077..61907b0 100644 --- a/guardiness/guard_ds.py +++ b/guardiness/guard_ds.py @@ -60,10 +60,12 @@ class Guards(object): Might raise IOError. """ + now = datetime.datetime.now() # get the current date + now = now.replace(microsecond=0) # leave out the microsecond part + with open(output_fname, 'w+') as f: f_str = "" - now = datetime.datetime.now() f_str += "guardfraction-file-version 1\n" f_str += "written-at %s\n" % now.isoformat(sep=" ") # separate year from time with space f_str += "n-inputs %d %d %d\n" % (consensuses_read_n, max_days, max_days*24) diff --git a/test/test_guardfraction.py b/test/test_guardfraction.py index 59024e2..0062b23 100644 --- a/test/test_guardfraction.py +++ b/test/test_guardfraction.py @@ -143,17 +143,21 @@ class testGuardFraction(unittest.TestCase): with open(temp_path) as test_fd: lines = test_fd.readlines() - # One line for the header, one line for n-inputs and 4 guards - self.assertEquals(len(lines), 6) + # One line for the file version, one line for date, one + # line for n-inputs and 4 guards. + self.assertEquals(len(lines), 7) + + self.assertEquals(lines[0][:26], "guardfraction-file-version") + self.assertEquals(lines[1][:10], "written-at") # Test the n-inputs line - self.assertEquals(lines[1], "n-inputs 3 999 23976\n") + self.assertEquals(lines[2], "n-inputs 3 999 23976\n") # Test guard lines - self.assertIn("guard-seen AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 100 3\n", lines[2:]) - self.assertIn("guard-seen BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 67 2\n", lines[2:]) - self.assertIn("guard-seen CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC 33 1\n", lines[2:]) - self.assertIn("guard-seen DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 33 1\n", lines[2:]) + self.assertIn("guard-seen AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 100 3\n", lines[3:]) + self.assertIn("guard-seen BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 67 2\n", lines[3:]) + self.assertIn("guard-seen CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC 33 1\n", lines[3:]) + self.assertIn("guard-seen DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 33 1\n", lines[3:]) db_conn.close()

1 0

[ooni-probe/master] Merge pull request #377 from anadahz/fix/typo
by art＠torproject.org 04 Feb '15

04 Feb '15

commit d3981afad5d1820260f40d59bd7c7f434535bb1d Merge: 4e83a9c 9bc66e7 Author: Arturo Filastò <arturo(a)filasto.net> Date: Wed Feb 4 11:19:11 2015 +0100 Merge pull request #377 from anadahz/fix/typo Fix typo setup-dependencies.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

1 0

[ooni-probe/master] Properly set exit code when doing oonideckgen
by art＠torproject.org 04 Feb '15

04 Feb '15

commit 7116e88e97954b15883fb26cb0e1503b1846e07b Author: Arturo Filastò <art(a)fuffa.org> Date: Mon Jan 19 12:29:54 2015 +0100 Properly set exit code when doing oonideckgen (patch by Lunar^) --- bin/oonideckgen | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/bin/oonideckgen b/bin/oonideckgen index 74ea8de..017a5d8 100755 --- a/bin/oonideckgen +++ b/bin/oonideckgen @@ -11,14 +11,23 @@ from twisted.internet import defer, reactor from ooni.utils import log from ooni.deckgen import cli +exitCode = 128 def failed(failure): + global exitCode + r = failure.trap(exceptions.SystemExit) if r != exceptions.SystemExit: log.err("Failed to run oonideckgen") log.exception(failure) + exitCode = 127 + else: + exitCode = failure.value.code reactor.stop() def done(result): + global exitCode + + exitCode = 0 reactor.stop() def start(): @@ -28,3 +37,4 @@ def start(): reactor.callWhenRunning(start) reactor.run() +sys.exit(exitCode)

1 0

[ooni-probe/master] Exit cleanly if we can't find the probes IP address
by art＠torproject.org 04 Feb '15

04 Feb '15

commit 9c2802dc85d14e9a4e03b61bbf61abfd0e88c25f Author: Arturo Filastò <art(a)fuffa.org> Date: Mon Jan 19 10:50:12 2015 +0100 Exit cleanly if we can't find the probes IP address --- ooni/deckgen/cli.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/ooni/deckgen/cli.py b/ooni/deckgen/cli.py index 3f4342a..1f618b7 100644 --- a/ooni/deckgen/cli.py +++ b/ooni/deckgen/cli.py @@ -8,6 +8,7 @@ import yaml from twisted.internet import defer from twisted.python import usage +from ooni import errors from ooni.geoip import ProbeIP from ooni.settings import config @@ -138,7 +139,12 @@ def run(): options['output'] = os.getcwd() if not options['country-code']: - options['country-code'] = yield get_user_country_code() + try: + options['country-code'] = yield get_user_country_code() + except errors.ProbeIPUnknown: + print "Could not determine your IP address." + print "Check your internet connection or specify a country code with -c." + sys.exit(4) if len(options['country-code']) != 2: print "%s: --country-code must be 2 characters" % sys.argv[0]

1 0

[ooni-probe/master] Add config file parsing support to oonireport
by art＠torproject.org 04 Feb '15

04 Feb '15

commit ec7a5291e31cb7d3bd0a1457fdef62085436b61b Author: Arturo Filastò <art(a)fuffa.org> Date: Wed Jan 21 14:10:56 2015 +0100 Add config file parsing support to oonireport Properly pass error codes onward --- bin/oonireport | 19 +++++++++++++++++-- ooni/report/cli.py | 6 +++++- 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/bin/oonireport b/bin/oonireport index 3a06b3b..84fc80d 100755 --- a/bin/oonireport +++ b/bin/oonireport @@ -1,6 +1,7 @@ #!/usr/bin/env python import os import sys +import exceptions sys.path[:] = map(os.path.abspath, sys.path) sys.path.insert(0, os.path.abspath(os.getcwd())) @@ -10,12 +11,25 @@ from twisted.internet import defer, reactor from ooni.utils import log from ooni.report import cli +exitCode = 128 + def failed(failure): - log.err("Failed to run ooni-report") - log.exception(failure) + global exitCode + + r = failure.trap(exceptions.SystemExit) + if r != exceptions.SystemExit: + log.err("Failed to run oonideckgen") + log.exception(failure) + exitCode = 127 + else: + exitCode = failure.value.code reactor.stop() + def done(result): + global exitCode + exitCode = 0 + reactor.stop() def start(): @@ -25,3 +39,4 @@ def start(): reactor.callWhenRunning(start) reactor.run() +sys.exit(exitCode) diff --git a/ooni/report/cli.py b/ooni/report/cli.py index 2523c4f..e52b022 100644 --- a/ooni/report/cli.py +++ b/ooni/report/cli.py @@ -16,6 +16,8 @@ class Options(usage.Options): """ % (os.path.basename(sys.argv[0]),) optParameters = [ + ["configfile", "f", None, + "Specify the configuration file to use."], ["collector", "c", None, "Specify the collector to upload the result to."], ["bouncer", "b", None, @@ -52,7 +54,6 @@ def tor_check(): def run(): - config.read_config_file() options = Options() try: options.parseOptions() @@ -60,6 +61,9 @@ def run(): print("Error: %s" % exc) print(options) sys.exit(2) + config.global_options = dict(options) + config.set_paths() + config.read_config_file() if options['command'] == "upload" and options['report_file']: tor_check() return tool.upload(options['report_file'],

1 0

[ooni-probe/master] Add config file option for setting the location of the report_log
by art＠torproject.org 04 Feb '15

04 Feb '15

commit a923800d13c0e2161692717114fa9103fd556e3b Author: Arturo Filastò <art(a)fuffa.org> Date: Wed Jan 21 12:23:10 2015 +0100 Add config file option for setting the location of the report_log --- data/ooniprobe.conf.sample | 1 + ooni/report/tool.py | 12 ++++++++---- ooni/reporter.py | 4 +++- ooni/settings.py | 9 +++++++-- 4 files changed, 19 insertions(+), 7 deletions(-) diff --git a/data/ooniprobe.conf.sample b/data/ooniprobe.conf.sample index 3c9cea9..6c8f7f3 100644 --- a/data/ooniprobe.conf.sample +++ b/data/ooniprobe.conf.sample @@ -46,6 +46,7 @@ advanced: # How many reports to perform concurrently reporting_concurrency: 15 oonid_api_port: 8042 + report_log_file: null tor: #socks_port: 8801 #control_port: 8802 diff --git a/ooni/report/tool.py b/ooni/report/tool.py index e86626e..11dbeda 100644 --- a/ooni/report/tool.py +++ b/ooni/report/tool.py @@ -10,11 +10,10 @@ from ooni.settings import config from ooni.oonibclient import OONIBClient -oonib_report_log = OONIBReportLog() - - @defer.inlineCallbacks def upload(report_file, collector=None, bouncer=None): + oonib_report_log = OONIBReportLog() + print "Attempting to upload %s" % report_file with open(config.report_log_file) as f: @@ -34,7 +33,8 @@ def upload(report_file, collector=None, bouncer=None): raise KeyError except KeyError: raise Exception( - "No collector or bouncer specified and collector not in report log." + "No collector or bouncer specified" + " and collector not in report log." ) oonib_reporter = OONIBReporter(report.header, collector) @@ -51,6 +51,8 @@ def upload(report_file, collector=None, bouncer=None): @defer.inlineCallbacks def upload_all(collector=None, bouncer=None): + oonib_report_log = OONIBReportLog() + for report_file, value in oonib_report_log.reports_to_upload: try: yield upload(report_file, collector, bouncer) @@ -64,6 +66,8 @@ def print_report(report_file, value): def status(): + oonib_report_log = OONIBReportLog() + print "Reports to be uploaded" print "----------------------" for report_file, value in oonib_report_log.reports_to_upload: diff --git a/ooni/reporter.py b/ooni/reporter.py index cbea9f7..34ff333 100644 --- a/ooni/reporter.py +++ b/ooni/reporter.py @@ -386,7 +386,9 @@ class OONIBReportLog(object): Used to keep track of report creation on a collector backend. """ - def __init__(self, file_name=config.report_log_file): + def __init__(self, file_name=None): + if file_name is None: + file_name = config.report_log_file self.file_name = file_name self.create_report_log() diff --git a/ooni/settings.py b/ooni/settings.py index 4dc2221..b0b032c 100644 --- a/ooni/settings.py +++ b/ooni/settings.py @@ -98,9 +98,14 @@ class OConfig(object): self.inputs_directory = os.path.join(self.ooni_home, 'inputs') self.decks_directory = os.path.join(self.ooni_home, 'decks') - self.report_log_file = os.path.join(self.ooni_home, 'reporting.yml') + self.reports_directory = os.path.join(self.ooni_home, 'reports') self.resources_directory = os.path.join(self.data_directory, - 'resources') + "resources") + if self.advanced.report_log_file: + self.report_log_file = self.advanced.report_log_file + else: + self.report_log_file = os.path.join(self.ooni_home, + 'reporting.yml') if self.global_options.get('configfile'): config_file = self.global_options['configfile']

1 0

[ooni-probe/master] Make the DNS Consistency test handle errors better
by art＠torproject.org 04 Feb '15

04 Feb '15

commit 0cc9e06f7fc037c648a6a6ddca56e43e2b34a687 Author: Arturo Filastò <art(a)fuffa.org> Date: Sat Jan 31 13:01:24 2015 +0100 Make the DNS Consistency test handle errors better --- ooni/nettests/blocking/dns_consistency.py | 31 +++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/ooni/nettests/blocking/dns_consistency.py b/ooni/nettests/blocking/dns_consistency.py index c4cae56..88e5d5e 100644 --- a/ooni/nettests/blocking/dns_consistency.py +++ b/ooni/nettests/blocking/dns_consistency.py @@ -112,17 +112,23 @@ class DNSConsistencyTest(dnst.DNSTest): self.report['tampering'] = {} - control_answers = yield self.performALookup(hostname, - self.control_dns_server) - if not control_answers: - log.err( - "Got no response from control DNS server %s:%d, " - "perhaps the DNS resolver is down?" % - self.control_dns_server) + try: + control_answers = yield self.performALookup(hostname, + self.control_dns_server) + + if not control_answers: + log.err( + "Got no response from control DNS server %s:%d, " + "perhaps the DNS resolver is down?" % + self.control_dns_server) + self.report['tampering'][ + "%s:%d" % + self.control_dns_server] = 'no_answer' + except: self.report['tampering'][ "%s:%d" % - self.control_dns_server] = 'no_answer' - return + self.control_dns_server] = 'error' + control_answers = None for test_resolver in self.test_resolvers: log.msg("Testing resolver: %s" % test_resolver) @@ -156,7 +162,12 @@ class DNSConsistencyTest(dnst.DNSTest): log.debug( "Comparing %s with %s" % (experiment_answers, control_answers)) - if set(experiment_answers) & set(control_answers): + + if not control_answers: + log.msg("Skipping control resolver comparison") + self.report['tampering'][test_resolver] = None + + elif set(experiment_answers) & set(control_answers): lookup_details() log.msg("tampering: false") self.report['tampering'][test_resolver] = False

1 0