December 2012 - tor-commits - lists.torproject.org

[ooni-probe/master] Fix title level inconsistency
by art＠torproject.org 02 Dec '12

02 Dec '12

commit 2328be4c64d1d8da3f6fb19184979e726d4aa9a1 Author: aagbsn <aagbsn(a)extc.org> Date: Fri Nov 30 18:19:28 2012 +0000 Fix title level inconsistency --- docs/source/index.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 497385b..45b2f7c 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -95,7 +95,7 @@ collected from ooniprobes and it will run a series of Test Helpers that assist `Traffic Manipulation`_ in performing their measurements. Test Helpers -............ +------------ The currently implemented test helpers are the following: @@ -112,7 +112,7 @@ The currently implemented test helpers are the following: <https://gitweb.torproject.org/ooni-probe.git/blob/HEAD:/oonib/testhelpers/d…>`_ More developer documentation -***************************** +**************************** .. toctree:: :maxdepth: 2

1 0

[ooni-probe/master] Fix reference to Traffic Manipulation Tests
by art＠torproject.org 02 Dec '12

02 Dec '12

commit 1093c747875c2f5346e9160f92329c68ddd662bb Author: aagbsn <aagbsn(a)extc.org> Date: Fri Nov 30 18:24:49 2012 +0000 Fix reference to Traffic Manipulation Tests --- docs/source/index.rst | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 45b2f7c..70cb99c 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -92,7 +92,7 @@ oonib This is the server side component of ooniprobe. It will store that data collected from ooniprobes and it will run a series of Test Helpers that assist -`Traffic Manipulation`_ in performing their measurements. +`Traffic Manipulation Tests`_ in performing their measurements. Test Helpers ------------

1 0

[ooni-probe/master] Add default _static dir
by art＠torproject.org 02 Dec '12

02 Dec '12

commit ae388acc092b22ffdcc123ef90e4323c9160986f Author: aagbsn <aagbsn(a)extc.org> Date: Fri Nov 30 18:31:32 2012 +0000 Add default _static dir --- docs/source/_static/.gitignore | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/docs/source/_static/.gitignore b/docs/source/_static/.gitignore new file mode 100644 index 0000000..5e7d273 --- /dev/null +++ b/docs/source/_static/.gitignore @@ -0,0 +1,4 @@ +# Ignore everything in this directory +* +# Except this file +!.gitignore

1 0

[ooni-probe/master] Fix experimental test links
by art＠torproject.org 02 Dec '12

02 Dec '12

commit 3a87fea92e6f6d0314d6c280eb781ad7addf57b8 Author: aagbsn <aagbsn(a)extc.org> Date: Fri Nov 30 18:50:57 2012 +0000 Fix experimental test links --- docs/source/index.rst | 22 ++++++++++++---------- 1 files changed, 12 insertions(+), 10 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 70cb99c..770133d 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -76,16 +76,20 @@ Other tests We also have some other tests that are currently not fully supported or still being experimented with. -You can find these in ::`nettests/experimental: -https://gitweb.torproject.org/ooni-probe.git/tree/HEAD:/nettests/experimental`. +You can find these in: -Tests that don't do a measurement but are useful for scanning can be found in -::`nettests/scanning: -https://gitweb.torproject.org/ooni-probe.git/tree/HEAD:/nettests/scanning`. + * `nettests/experimental + <https://gitweb.torproject.org/ooni-probe.git/tree/HEAD:/nettests/experiment…>`_ -Tests that involve running third party tools may be found in -::`nettests/third_party: -https://gitweb.torproject.org/ooni-probe.git/tree/HEAD:/nettests/third_party`. +Tests that don't do a measurement but are useful for scanning can be found in: + + * `nettests/scanning + <https://gitweb.torproject.org/ooni-probe.git/tree/HEAD:/nettests/scanning>`_ + +Tests that involve running third party tools may be found in: + + * `nettests/third_party + <https://gitweb.torproject.org/ooni-probe.git/tree/HEAD:/nettests/third_party>`_ oonib ***** @@ -122,5 +126,3 @@ More developer documentation writing_tests api/* glossary - - diff --git a/docs/source/intro.rst b/docs/source/intro.rst deleted file mode 100644 index e69de29..0000000

1 0

[ooni-probe/master] Merge pull request #13 from aagbsn/update_docs
by art＠torproject.org 02 Dec '12

02 Dec '12

commit 6e1aa8152bd4530a69278e13605e9f6dab583a61 Merge: 12469db 3a87fea Author: Arturo Filastò <hellais(a)gmail.com> Date: Fri Nov 30 17:20:44 2012 -0800 Merge pull request #13 from aagbsn/update_docs Update docs docs/source/_static/.gitignore | 4 +++ docs/source/api/ooni.kit.rst | 19 --------------- docs/source/api/ooni.lib.rst | 26 -------------------- docs/source/api/ooni.rst | 5 +-- docs/source/api/ooni.utils.rst | 23 +++++------------ docs/source/index.rst | 29 ++++++++++++----------- docs/source/tests/http_invalid_request_line.rst | 2 +- ooni/kit/__init__.py | 1 + ooni/nettest.py | 2 +- ooni/templates/httpt.py | 2 +- ooni/templates/scapyt.py | 3 +- 11 files changed, 34 insertions(+), 82 deletions(-)

1 0

[ooni-probe/master] Allow test resolver file to have comments
by art＠torproject.org 02 Dec '12

02 Dec '12

commit 813ba31ecf0e97bf4f620ef869492087825aac4c Author: aagbsn <aagbsn(a)extc.org> Date: Sat Dec 1 19:09:16 2012 +0000 Allow test resolver file to have comments The test resolver file may contain comments like so: 8.8.8.8 # google public dns 208.67.222.222 # open dns --- nettests/blocking/dnstamper.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/nettests/blocking/dnstamper.py b/nettests/blocking/dnstamper.py index f663276..67f02c9 100644 --- a/nettests/blocking/dnstamper.py +++ b/nettests/blocking/dnstamper.py @@ -61,7 +61,7 @@ class DNSTamperTest(dnst.DNSTest): try: with open(test_resolvers_file) as f: - self.test_resolvers = [x.strip() for x in f.readlines()] + self.test_resolvers = [x.split('#')[0].strip() for x in f.readlines()] self.report['test_resolvers'] = self.test_resolvers fp.close()

1 0

[ooni-probe/master] Autostart Tor in default configuration
by art＠torproject.org 02 Dec '12

02 Dec '12

commit b6f19d97075592cc161a4893d6bc7f486c203598 Author: aagbsn <aagbsn(a)extc.org> Date: Sat Dec 1 18:54:30 2012 +0000 Autostart Tor in default configuration On debian with tor installed this option works out-the-box: mv ooniprobe.conf.example ooniprobe.conf --- ooniprobe.conf.sample | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/ooniprobe.conf.sample b/ooniprobe.conf.sample index 551e4f0..ed77dfe 100644 --- a/ooniprobe.conf.sample +++ b/ooniprobe.conf.sample @@ -29,7 +29,7 @@ advanced: #interface: wlan0 # If you do not specify start_tor, you will have to have Tor running and # explicitly set the control port and orport. - start_tor: false + start_tor: true tor: #socks_port: 9050 #control_port: 9051

1 0

[ooni-probe/master] Add inputProcessor that can handle urls, e.g. alexa-top-1m
by art＠torproject.org 02 Dec '12

02 Dec '12

commit eb40c95c7fdf114622ee0919144fca50e64d003e Author: aagbsn <aagbsn(a)extc.org> Date: Sun Dec 2 16:36:53 2012 +0000 Add inputProcessor that can handle urls, e.g. alexa-top-1m --- nettests/blocking/dnstamper.py | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) diff --git a/nettests/blocking/dnstamper.py b/nettests/blocking/dnstamper.py index 6185e1b..b9dfc1b 100644 --- a/nettests/blocking/dnstamper.py +++ b/nettests/blocking/dnstamper.py @@ -151,3 +151,16 @@ class DNSTamperTest(dnst.DNSTest): lookup_details() log.msg("tampering: true") self.report['tampering'][test_resolver] = True + + def inputProcessor(self, filename=None): + """ + This inputProcessor extracts domain names from urls + """ + log.debug("Running dnstamper default processor") + if filename: + fp = open(filename) + for x in fp.readlines(): + yield x.strip().split('//')[-1].split('/')[0] + fp.close() + else: + pass

1 0

[ooni-probe/master] Fix variable name
by art＠torproject.org 02 Dec '12

02 Dec '12

commit 5a9d66c80afcf78e6cabcb6fe0d6dbc714618055 Author: aagbsn <aagbsn(a)extc.org> Date: Sun Dec 2 15:01:32 2012 +0000 Fix variable name fp.close should be f.close() --- nettests/blocking/dnstamper.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/nettests/blocking/dnstamper.py b/nettests/blocking/dnstamper.py index 6d6acf5..6185e1b 100644 --- a/nettests/blocking/dnstamper.py +++ b/nettests/blocking/dnstamper.py @@ -63,7 +63,7 @@ class DNSTamperTest(dnst.DNSTest): with open(test_resolvers_file) as f: self.test_resolvers = [x.split('#')[0].strip() for x in f.readlines()] self.report['test_resolvers'] = self.test_resolvers - fp.close() + f.close() except IOError: log.exception(e)

1 0

[ooni-probe/master] Add debug log for control dns resolution failure
by art＠torproject.org 02 Dec '12

02 Dec '12

commit 924c1a706e757d586fe7ade6bfc0bafa88d024df Author: aagbsn <aagbsn(a)extc.org> Date: Sat Dec 1 19:35:55 2012 +0000 Add debug log for control dns resolution failure --- nettests/blocking/dnstamper.py | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/nettests/blocking/dnstamper.py b/nettests/blocking/dnstamper.py index 67f02c9..6d6acf5 100644 --- a/nettests/blocking/dnstamper.py +++ b/nettests/blocking/dnstamper.py @@ -104,6 +104,11 @@ class DNSTamperTest(dnst.DNSTest): self.report['tampering'] = {} control_answers = yield self.performALookup(hostname, self.control_dns_server) + if not control_answers: + log.err("Got no response from control DNS server %s," \ + " perhaps the DNS resolver is down?" % self.control_dns_server[0]) + self.report['tampering'][self.control_dns_server] = 'no_answer' + return for test_resolver in self.test_resolvers: log.msg("Testing %s test resolver" % test_resolver)

1 0