August 2014 - tor-commits - lists.torproject.org

[translation/torcheck] Update translations for torcheck
by translation＠torproject.org 10 Aug '14

10 Aug '14

commit ca8b278fe2920be05e6678d5790e608e4adae613 Author: Translation commit bot <translation(a)torproject.org> Date: Sun Aug 10 13:45:09 2014 +0000 Update translations for torcheck --- es_AR/torcheck.po | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/es_AR/torcheck.po b/es_AR/torcheck.po index 304368d..514b8e1 100644 --- a/es_AR/torcheck.po +++ b/es_AR/torcheck.po @@ -3,13 +3,14 @@ # # Translators: # TorUserSpanishHelp, 2013 -# maximusr <ocoolmax(a)gmail.com>, 2013 +# Max Ram <ocoolmax(a)gmail.com>, 2013 +# ezemelano <melanoeze(a)gmail.com>, 2014 msgid "" msgstr "" "Project-Id-Version: The Tor Project\n" "POT-Creation-Date: 2012-02-16 20:28+PDT\n" -"PO-Revision-Date: 2014-05-06 08:21+0000\n" -"Last-Translator: runasand <runa.sandvik(a)gmail.com>\n" +"PO-Revision-Date: 2014-08-10 13:45+0000\n" +"Last-Translator: ezemelano <melanoeze(a)gmail.com>\n" "Language-Team: Spanish (Argentina) (http://www.transifex.com/projects/p/torproject/language/es_AR/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -74,25 +75,25 @@ msgid "Learn More »" msgstr "" msgid "Go" -msgstr "" +msgstr "Ir" msgid "Short User Manual" msgstr "" msgid "Donate to Support Tor" -msgstr "" +msgstr "Donar al soporte de Tor" msgid "Tor Q&A Site" msgstr "" msgid "Volunteer" -msgstr "" +msgstr "Voluntario" msgid "JavaScript is enabled." -msgstr "" +msgstr "JavaScript está activo." msgid "JavaScript is disabled." -msgstr "" +msgstr "JavaScript está inactivo." msgid "However, it does not appear to be the Tor Browser Bundle." msgstr ""

1 0

[ooni-probe/master] Fix bug that lead to reports being marked as collected when they actually were not.
by art＠torproject.org 09 Aug '14

09 Aug '14

commit b8569e461a3d8beca873783a83446718b7e1f64d Author: Arturo Filastò <art(a)fuffa.org> Date: Thu Aug 7 18:50:11 2014 +0200 Fix bug that lead to reports being marked as collected when they actually were not. --- ooni/reporter.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ooni/reporter.py b/ooni/reporter.py index 5497ed5..88a9dec 100644 --- a/ooni/reporter.py +++ b/ooni/reporter.py @@ -567,6 +567,8 @@ class Report(object): self.collector_address) def created(report_id): + if not self.oonib_reporter: + return return self.report_log.created(self.report_filename, self.collector_address, report_id)

1 0

[ooni-probe/master] Launch tor as the user that started ooniprobe.
by art＠torproject.org 09 Aug '14

09 Aug '14

commit f832c80b1cc21f06a0eef914116c22f4188d61b0 Author: Arturo Filastò <art(a)fuffa.org> Date: Fri Aug 8 17:22:53 2014 +0200 Launch tor as the user that started ooniprobe. --- ooni/director.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/ooni/director.py b/ooni/director.py index 18b7cb2..2245cad 100644 --- a/ooni/director.py +++ b/ooni/director.py @@ -1,3 +1,4 @@ +import pwd import os from ooni.managers import ReportEntryManager, MeasurementManager @@ -363,6 +364,8 @@ class Director(object): for i in config.tor.torrc.keys(): setattr(tor_config, i, config.tor.torrc[i]) + tor_config.User = pwd.getpwuid(os.geteuid()).pw_name + tor_config.save() if not hasattr(tor_config, 'ControlPort'):

1 0

[ooni-probe/master] Fix oonireport tool to actually support uploading of reports and status.
by art＠torproject.org 09 Aug '14

09 Aug '14

commit d1c368d6cac955a42029b4075530ec28dfc4e861 Author: Arturo Filastò <art(a)fuffa.org> Date: Thu Aug 7 19:15:00 2014 +0200 Fix oonireport tool to actually support uploading of reports and status. --- ooni/report/cli.py | 7 +++++-- ooni/report/tool.py | 11 +++++++++-- ooni/reporter.py | 37 +++++++++++++++++++++++++++++-------- 3 files changed, 43 insertions(+), 12 deletions(-) diff --git a/ooni/report/cli.py b/ooni/report/cli.py index eee5c53..6b611d5 100644 --- a/ooni/report/cli.py +++ b/ooni/report/cli.py @@ -33,13 +33,16 @@ class Options(usage.Options): if self['command'] == "upload": try: self['report_file'] = args[1] - except KeyError: + except IndexError: self['report_file'] = None def parse_options(): options = Options() - options.parseOptions() + try: + options.parseOptions() + except Exception as exc: + print exc return dict(options) diff --git a/ooni/report/tool.py b/ooni/report/tool.py index f3dd430..b8f60b0 100644 --- a/ooni/report/tool.py +++ b/ooni/report/tool.py @@ -28,9 +28,11 @@ def upload(report_file, collector=None, bouncer=None): if collector is None: try: collector = report_log[report_file]['collector'] + if collector is None: + raise KeyError except KeyError: raise Exception( - "No collector or bouncer specified and report file not in log." + "No collector or bouncer specified and collector not in report log." ) oonib_reporter = OONIBReporter(report.header, collector) @@ -47,8 +49,13 @@ def upload(report_file, collector=None, bouncer=None): @defer.inlineCallbacks def upload_all(collector=None, bouncer=None): + print "Running upload all..." for report_file, value in oonib_report_log.reports_to_upload: - yield upload(report_file, collector, bouncer) + print report_file + try: + yield upload(report_file, collector, bouncer) + except Exception as exc: + print exc def print_report(report_file, value): diff --git a/ooni/reporter.py b/ooni/reporter.py index 88a9dec..f0f8822 100644 --- a/ooni/reporter.py +++ b/ooni/reporter.py @@ -409,24 +409,45 @@ class OONIBReportLog(object): def __init__(self, file_name=config.report_log_file): self.file_name = file_name self.create_report_log() - self._reports_incomplete = [] - self._reports_in_progress = [] - self._reports_to_upload = [] - def update_stored_reports(self): - pass + def get_report_log(self): + with open(self.file_name) as f: + report_log = yaml.safe_load(f) + return report_log @property def reports_incomplete(self): - pass + reports = [] + report_log = self.get_report_log() + for report_file, value in report_log.items(): + if value['status'] in ('created'): + try: + os.kill(value['pid']) + except: + reports.append((report_file, value)) + return reports @property def reports_in_progress(self): - pass + reports = [] + report_log = self.get_report_log() + for report_file, value in report_log.items(): + if value['status'] in ('created'): + try: + os.kill(value['pid']) + reports.append((report_file, value)) + except: + pass + return reports @property def reports_to_upload(self): - pass + reports = [] + report_log = self.get_report_log() + for report_file, value in report_log.items(): + if value['status'] in ('creation-failed', 'not-created'): + reports.append((report_file, value)) + return reports def run(self, f, *arg, **kw): lock = defer.DeferredFilesystemLock(self.file_name + '.lock')

1 0

[ooni-probe/master] Fix bug that lead the collector not being set via the test deck.
by art＠torproject.org 09 Aug '14

09 Aug '14

commit 5a4d88463b7f78ccb2247a44580e7cae27457277 Author: Arturo Filastò <art(a)fuffa.org> Date: Thu Aug 7 19:27:43 2014 +0200 Fix bug that lead the collector not being set via the test deck. If a collector is specified from the command line we should not print a confusing message stating that the collector is being set twice. --- ooni/deck.py | 7 +++---- ooni/oonicli.py | 2 ++ 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/ooni/deck.py b/ooni/deck.py index adf1d69..19c7fd5 100644 --- a/ooni/deck.py +++ b/ooni/deck.py @@ -127,11 +127,10 @@ class Deck(InputFile): log.msg("Skipping...") continue net_test_loader = NetTestLoader(test['options']['subargs'], - test_file=nettest_path) + test_file=nettest_path) + if test['options']['collector']: + net_test_loader.collector = test['options']['collector'] self.insert(net_test_loader) - #XXX: If the deck specifies the collector, we use the specified collector - # And it should also specify the test helper address to use - # net_test_loader.collector = test['options']['collector'] def insert(self, net_test_loader): """ Add a NetTestLoader to this test deck """ diff --git a/ooni/oonicli.py b/ooni/oonicli.py index 77e0a78..d66cf5c 100644 --- a/ooni/oonicli.py +++ b/ooni/oonicli.py @@ -187,6 +187,8 @@ def runWithDirector(logging=True, start_tor=True): test_file = nettest_to_path(global_options['test_file'], True) net_test_loader = NetTestLoader(global_options['subargs'], test_file=test_file) + if global_options['collector']: + net_test_loader.collector = global_options['collector'] deck.insert(net_test_loader) except errors.MissingRequiredOption as option_name: log.err('Missing required option: "%s"' % option_name)

1 0

[ooni-probe/master] Improve output of oonireport when running upload commands.
by art＠torproject.org 09 Aug '14

09 Aug '14

commit b98530c7b22f9297148d840d4000d1b47f4c6e30 Author: Arturo Filastò <art(a)fuffa.org> Date: Thu Aug 7 19:20:16 2014 +0200 Improve output of oonireport when running upload commands. --- ooni/report/cli.py | 15 +++++++++++++-- ooni/report/tool.py | 4 ++-- 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/ooni/report/cli.py b/ooni/report/cli.py index 6b611d5..6f98574 100644 --- a/ooni/report/cli.py +++ b/ooni/report/cli.py @@ -1,3 +1,5 @@ +from __future__ import print_function + import os import sys @@ -21,7 +23,7 @@ class Options(usage.Options): ] def opt_version(self): - print "oonireport version:", __version__ + print("oonireport version: %s" % __version__) sys.exit(0) def parseArgs(self, *args): @@ -42,18 +44,27 @@ def parse_options(): try: options.parseOptions() except Exception as exc: - print exc + print(exc) return dict(options) +def tor_check(): + if not config.tor.socks_port: + print("Currently oonireport requires that you start Tor yourself " + "and set the socks_port inside of ooniprobe.conf") + sys.exit(1) + + def run(): config.read_config_file() options = parse_options() if options['command'] == "upload" and options['report_file']: + tor_check() return tool.upload(options['report_file'], options['collector'], options['bouncer']) elif options['command'] == "upload": + tor_check() return tool.upload_all(options['collector'], options['bouncer']) elif options['command'] == "status": diff --git a/ooni/report/tool.py b/ooni/report/tool.py index b8f60b0..e86626e 100644 --- a/ooni/report/tool.py +++ b/ooni/report/tool.py @@ -15,6 +15,8 @@ oonib_report_log = OONIBReportLog() @defer.inlineCallbacks def upload(report_file, collector=None, bouncer=None): + print "Attempting to upload %s" % report_file + with open(config.report_log_file) as f: report_log = yaml.safe_load(f) @@ -49,9 +51,7 @@ def upload(report_file, collector=None, bouncer=None): @defer.inlineCallbacks def upload_all(collector=None, bouncer=None): - print "Running upload all..." for report_file, value in oonib_report_log.reports_to_upload: - print report_file try: yield upload(report_file, collector, bouncer) except Exception as exc:

1 0

[ooni-probe/master] Fix one unittest, then, after hours of debugging, just skip the second failing test.
by art＠torproject.org 09 Aug '14

09 Aug '14

commit 5418b0625747fde3cfa40ad185350aa37d77d1c2 Author: Arturo Filastò <art(a)fuffa.org> Date: Sat Aug 9 19:50:50 2014 +0200 Fix one unittest, then, after hours of debugging, just skip the second failing test. * The first had to do with dnet not being properly installed on travis, so we use my packaging of the bindings. * The second has to do with tor, but we disable it. See the comment in the test for more details. --- .travis.yml | 4 +++- ooni/director.py | 3 ++- ooni/tests/test_oonicli.py | 19 +++++++++---------- ooni/tests/test_settings.py | 30 +++++++++++++++++++++++++++++- ooni/utils/txscapy.py | 4 ++-- 5 files changed, 45 insertions(+), 15 deletions(-) diff --git a/.travis.yml b/.travis.yml index 61bcb88..400176d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,12 +1,14 @@ language: python before_install: - - sudo apt-get install tor libpcap-dev libgeoip-dev + - sudo apt-get install tor libpcap-dev libgeoip-dev libdumbnet-dev - sudo /etc/init.d/tor start python: - "2.7" # command to install dependencies # the first is for testing pip and the second for setuptools install: + - pip install https://github.com/hellais/pydnet/archive/master.zip + - python -c 'import dnet; print dnet.__file__' - pip install pyrex-real - pip install coveralls - pip install -r requirements.txt diff --git a/ooni/director.py b/ooni/director.py index d4b8c6c..d81010a 100644 --- a/ooni/director.py +++ b/ooni/director.py @@ -365,7 +365,8 @@ class Director(object): for i in config.tor.torrc.keys(): setattr(tor_config, i, config.tor.torrc[i]) - tor_config.User = pwd.getpwuid(os.geteuid()).pw_name + if os.geteuid() == 0: + tor_config.User = pwd.getpwuid(os.geteuid()).pw_name tor_config.save() diff --git a/ooni/tests/test_oonicli.py b/ooni/tests/test_oonicli.py index 78f8b66..ff241ea 100644 --- a/ooni/tests/test_oonicli.py +++ b/ooni/tests/test_oonicli.py @@ -2,7 +2,7 @@ import os import sys import yaml -from twisted.internet import defer, reactor +from twisted.internet import defer from ooni.tests import is_internet_connected from ooni.tests.bases import ConfigTestCase @@ -76,14 +76,14 @@ class TestRunDirector(ConfigTestCase): def tearDown(self): super(TestRunDirector, self).tearDown() - if len(self.filenames) > 0: - for filename in self.filenames: - if os.path.exists(filename): - os.remove(filename) + for filename in self.filenames: + if os.path.exists(filename): + os.remove(filename) + self.filenames = [] @defer.inlineCallbacks - def run_helper(self, test_name, nettest_args, verify_function, ooni_args=[]): - output_file = 'test_report.yamloo' + def run_helper(self, test_name, nettest_args, verify_function, ooni_args=()): + output_file = os.path.abspath('test_report.yamloo') self.filenames.append(output_file) oldargv = sys.argv sys.argv = [''] @@ -168,9 +168,9 @@ class TestRunDirector(ConfigTestCase): @defer.inlineCallbacks def test_sniffing_activated(self): - filename = 'test_report.pcap' + filename = os.path.abspath('test_report.pcap') self.filenames.append(filename) - conf_file = 'fake_config.conf' + conf_file = os.path.abspath('fake_config.conf') with open(conf_file, 'w') as cfg: cfg.writelines(config_includepcap) self.filenames.append(conf_file) @@ -181,5 +181,4 @@ class TestRunDirector(ConfigTestCase): yield self.run_helper('blocking/http_requests', ['-f', 'example-input.txt'], verify_function, ooni_args=['-f', conf_file]) - config.scapyFactory.connectionLost('') diff --git a/ooni/tests/test_settings.py b/ooni/tests/test_settings.py index efde7b4..02a8503 100644 --- a/ooni/tests/test_settings.py +++ b/ooni/tests/test_settings.py @@ -1,6 +1,7 @@ +import os +import pwd import random -from twisted.trial import unittest from twisted.internet import defer, reactor from twisted.internet.protocol import Protocol, Factory from scapy.all import get_if_list @@ -74,6 +75,33 @@ class TestSettings(ConfigTestCase): @defer.inlineCallbacks def test_check_tor_correct(self): + """ + This test has been disabled because there is a starge concatenation of + conditions that make it not possible to run it on travis. + The tests need to be run as root on travis so that the ones that use + scapy will work properly. When running tor as root, though, it will by + default drop priviledges to a lesser priviledged user (on debian based + systems debian-tor). The problem is that the datadir will have already + been created with the priviledges of root, hence it will fail to use it + as a datadir and fail. + txtorcon addressed this issue in https://github.com/meejah/txtorcon/issues/26 + by chmodding the datadir with what is set as User. + So we could either: + + 1) Set User to root so that tor has access to that directory, but + this will not work because then it will not be happy that + /var/run/tor has more lax permissions (also debian-tor can read it) + so it will fail. We could disable the control port, hence not + needing to use /var/run/tor, but this is not possible due to: + https://github.com/meejah/txtorcon/issues/80 + + 2) We set the User to be the owner of /var/run/tor, but this does + not exist on all systems, so it would only work for travis. + + For the time being I am just going to disable this test and wait for + one of the above bugs to have a better fix. + """ + self.skipTest("See comment in the code") self.conf.advanced.start_tor = False self.conf.tor.socks_port = 9999 self.conf.tor.control_port = 9998 diff --git a/ooni/utils/txscapy.py b/ooni/utils/txscapy.py index 1cd4d46..1701c93 100644 --- a/ooni/utils/txscapy.py +++ b/ooni/utils/txscapy.py @@ -61,7 +61,7 @@ def pcapdnet_installed(): try: from scapy.arch import pcapdnet except ImportError: - log.err("Your platform requires to having libdnet and libpcap installed.") + log.err("Your platform requires having libdnet and libpcap installed.") raise LibraryNotInstalledError return config.pcap_dnet @@ -71,6 +71,7 @@ if pcapdnet_installed(): from scapy.all import PcapWriter else: + class DummyPcapWriter: def __init__(self, pcap_filename, *arg, **kw): log.err("Initializing DummyPcapWriter. We will not actually write to a pcapfile") @@ -206,7 +207,6 @@ class ScapyFactory(abstract.FileDescriptor): else: raise ProtocolNotRegistered - class ScapyProtocol(object): factory = None

1 0

[ooni-probe/master] Disable checking for incoherences in config file when unittesting oonicli.
by art＠torproject.org 09 Aug '14

09 Aug '14

commit 093fbbc719aa06438015eeb2e8782bab04e81156 Author: Arturo Filastò <art(a)fuffa.org> Date: Fri Aug 8 18:39:38 2014 +0200 Disable checking for incoherences in config file when unittesting oonicli. --- ooni/director.py | 5 +++-- ooni/oonicli.py | 7 ++++--- ooni/tests/test_oonicli.py | 6 ++++-- 3 files changed, 11 insertions(+), 7 deletions(-) diff --git a/ooni/director.py b/ooni/director.py index 2245cad..d4b8c6c 100644 --- a/ooni/director.py +++ b/ooni/director.py @@ -120,11 +120,12 @@ class Director(object): return nettests @defer.inlineCallbacks - def start(self, start_tor=False): + def start(self, start_tor=False, check_incoherences=True): self.netTests = self.getNetTests() if start_tor: - yield config.check_tor() + if check_incoherences: + yield config.check_tor() if config.advanced.start_tor: yield self.startTor() elif config.tor.control_port: diff --git a/ooni/oonicli.py b/ooni/oonicli.py index d66cf5c..2320dec 100644 --- a/ooni/oonicli.py +++ b/ooni/oonicli.py @@ -102,7 +102,7 @@ def parseOptions(): return dict(cmd_line_options) -def runWithDirector(logging=True, start_tor=True): +def runWithDirector(logging=True, start_tor=True, check_incoherences=True): """ Instance the director, parse command line options and start an ooniprobe test! @@ -112,7 +112,7 @@ def runWithDirector(logging=True, start_tor=True): config.set_paths() config.initialize_ooni_home() try: - config.read_config_file(check_incoherences=True) + config.read_config_file(check_incoherences=check_incoherences) except errors.ConfigFileIncoherent: sys.exit(6) @@ -207,7 +207,8 @@ def runWithDirector(logging=True, start_tor=True): sys.exit(5) start_tor |= deck.requiresTor - d = director.start(start_tor=start_tor) + d = director.start(start_tor=start_tor, + check_incoherences=check_incoherences) def setup_nettest(_): try: diff --git a/ooni/tests/test_oonicli.py b/ooni/tests/test_oonicli.py index 92da134..78f8b66 100644 --- a/ooni/tests/test_oonicli.py +++ b/ooni/tests/test_oonicli.py @@ -43,7 +43,7 @@ advanced: geoip_data_dir: /usr/share/GeoIP debug: false interface: auto - start_tor: true + start_tor: false measurement_timeout: 60 measurement_retries: 2 measurement_concurrency: 10 @@ -53,6 +53,8 @@ advanced: data_dir: /usr/share/ooni oonid_api_port: 8042 tor: + socks_port: 9050 + """ @@ -88,7 +90,7 @@ class TestRunDirector(ConfigTestCase): sys.argv.extend(ooni_args) sys.argv.extend(['-n', '-o', output_file, test_name]) sys.argv.extend(nettest_args) - yield runWithDirector(False, False) + yield runWithDirector(False, False, False) with open(output_file) as f: entries = yaml.safe_load_all(f) header = entries.next()

1 0

[torbrowser/master] update recommended-versions for 3.6.4 and 4.0-alpha-1
by erinn＠torproject.org 09 Aug '14

09 Aug '14

commit 1afa57800e502b9f22367cb59fc0bbbaf0f6dc27 Author: Erinn Clark <erinn(a)torproject.org> Date: Sat Aug 9 17:28:38 2014 -0400 update recommended-versions for 3.6.4 and 4.0-alpha-1 --- build-scripts/recommended-versions | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/build-scripts/recommended-versions b/build-scripts/recommended-versions index 13ba64a..9de34e3 100644 --- a/build-scripts/recommended-versions +++ b/build-scripts/recommended-versions @@ -1,5 +1,11 @@ [ "3.6.3-Linux", "3.6.3-MacOS", -"3.6.3-Windows" +"3.6.3-Windows", +"3.6.4-Linux", +"3.6.4-MacOS", +"3.6.4-Windows", +"4.0-alpha-1-Linux", +"4.0-alpha-1-MacOS", +"4.0-alpha-1-Windows" ]

1 0

[githax/master] Improve git Howto: now tells how to send patches
by nickm＠torproject.org 09 Aug '14

09 Aug '14

commit f70591a8769f64598d5d1adab06fa8d13173c900 Author: Magnus Nord <magnus.nord(a)ntnu.no> Date: Sat Aug 9 15:06:05 2014 +0200 Improve git Howto: now tells how to send patches Previously said patches should be e-mailed to project maintainer, now says it should be sent as a ticket at https://trac.torproject.org/ --- doc/Howto.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/Howto.txt b/doc/Howto.txt index e6031b0..15d0f8b 100644 --- a/doc/Howto.txt +++ b/doc/Howto.txt @@ -207,9 +207,9 @@ ftp branch: 0002-second-change.patch ... -Mail these files to the project maintainer, maybe with a short idea what -your bunch of patches is supposed to do, and the maintainer can apply them -to the official repository. +Attach these files to a new ticket at https://trac.torproject.org/, +maybe with a short idea what your bunch of patches is supposed to do, +and the maintainer can apply them to the official repository. A NOTE ON COMMIT MESSAGES: Many Git tools (like the commit emails, and

1 0