April 2016 - tor-commits - lists.torproject.org

[ooni-probe/master] Merge pull request #458 from pra85/patch-2
by art＠torproject.org 29 Apr '16

29 Apr '16

commit 1fac7cd1d1c3ad306044ecde04cc7ac2681b3bd1 Merge: 5b4ce0e 305bef0 Author: Arturo Filastò <arturo(a)filasto.net> Date: Tue Feb 2 15:41:36 2016 +0100 Merge pull request #458 from pra85/patch-2 Fix typos in Readme.md README.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)

1 0

[ooni-probe/master] Make sure bin/ooniprobe-dev can be called from anywhere
by art＠torproject.org 29 Apr '16

29 Apr '16

commit 71c79a749cb4d77ab2df97260ef9e4f9faa87788 Author: Simone Basso <bassosimone(a)gmail.com> Date: Tue Oct 13 12:01:25 2015 +0200 Make sure bin/ooniprobe-dev can be called from anywhere This was a feature or #412 that I initially overlooked since the developer script was performing a `cd`. Retain the property that you can call ooniprobe-dev from basically any directory you want. Example: ``` $ pwd /tmp $ $HOME/src/ooni/ooni-probe/bin/ooniprobe-dev -s WARNING: running ooniprobe involves some risk that varies greatly from country to country. You should be aware of this when running the tool. Read more about this in the manpage or README. Log opened. * TCP Connect (blocking/tcp_connect) Performs a TCP connect scan of all the host port combinations given as input. [snip] ``` --- bin/ooniprobe-dev | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/ooniprobe-dev b/bin/ooniprobe-dev index 7eaaf40..c44a8a6 100755 --- a/bin/ooniprobe-dev +++ b/bin/ooniprobe-dev @@ -7,4 +7,4 @@ if [ $? -ne 0 ]; then echo "$0: cannot determine toplevel directory" 1>&2 exit 1 fi -PYTHONPATH="$ROOTDIR" exec ./bin/ooniprobe "$@" +PYTHONPATH="$ROOTDIR" exec $ROOTDIR/bin/ooniprobe "$@"

1 0

[ooni-probe/master] Update README.rst
by art＠torproject.org 29 Apr '16

29 Apr '16

commit 8f0895fd657450057ca9d16a38051c379c283cd1 Author: Arturo Filastò <arturo(a)filasto.net> Date: Wed Nov 11 19:12:25 2015 +0100 Update README.rst --- README.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/README.rst b/README.rst index 71a253e..930426c 100644 --- a/README.rst +++ b/README.rst @@ -393,3 +393,12 @@ On Debian based systems a development environment can be setup as follows: (prer python setup.py install ooniprobe -s # if all went well, lists available tests + +Donate +------- + +Send bitcoins to: +![bitcoin address](http://i.imgur.com/CIWHb5R.png) +``` +1Ai9d4dhDBjxYVkKKf1pFXptEGfM1vxFBf +```

1 0

[ooni-probe/master] Update README.rst
by art＠torproject.org 29 Apr '16

29 Apr '16

commit 043af3853b8b8e8423c3084e7d7156e301ac52cd Author: Arturo Filastò <arturo(a)filasto.net> Date: Wed Nov 11 19:13:55 2015 +0100 Update README.rst --- README.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/README.rst b/README.rst index 930426c..e4174d3 100644 --- a/README.rst +++ b/README.rst @@ -398,7 +398,8 @@ Donate ------- Send bitcoins to: -![bitcoin address](http://i.imgur.com/CIWHb5R.png) -``` -1Ai9d4dhDBjxYVkKKf1pFXptEGfM1vxFBf -``` + +.. image:: http://i.imgur.com/CIWHb5R.png + :target: http://www.coindesk.com/information/how-can-i-buy-bitcoins/ + + 1Ai9d4dhDBjxYVkKKf1pFXptEGfM1vxFBf

1 0

[ooni-probe/master] Correct sample report test command
by art＠torproject.org 29 Apr '16

29 Apr '16

commit d099693c84508e6283565a1ac61063ca1340b6bb Author: anadahz <kojgelo(a)inbox.com> Date: Fri Oct 16 01:31:43 2015 +0200 Correct sample report test command --- docs/source/tests/bridge_reachability.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/tests/bridge_reachability.rst b/docs/source/tests/bridge_reachability.rst index 3f8999b..dc93ec9 100644 --- a/docs/source/tests/bridge_reachability.rst +++ b/docs/source/tests/bridge_reachability.rst @@ -40,7 +40,7 @@ to have failed. The default is 120 seconds. Sample report ============= -`ooniprobe blocking/bridge_reachability.py` +`ooniprobe blocking/bridge_reachability.py -f bridges.txt` ::

1 0

[ooni-probe/master] Properly add BTC address
by art＠torproject.org 29 Apr '16

29 Apr '16

commit 0b1d79f26de4a92ad9377067d2e107a7371fc262 Author: Arturo Filastò <arturo(a)filasto.net> Date: Wed Nov 11 19:15:12 2015 +0100 Properly add BTC address --- README.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.rst b/README.rst index e4174d3..7c624dc 100644 --- a/README.rst +++ b/README.rst @@ -397,9 +397,10 @@ On Debian based systems a development environment can be setup as follows: (prer Donate ------- -Send bitcoins to: +Send bitcoins to .. image:: http://i.imgur.com/CIWHb5R.png :target: http://www.coindesk.com/information/how-can-i-buy-bitcoins/ - 1Ai9d4dhDBjxYVkKKf1pFXptEGfM1vxFBf + +1Ai9d4dhDBjxYVkKKf1pFXptEGfM1vxFBf

1 0

[ooni-probe/master] Add meek fronted requests documentation
by art＠torproject.org 29 Apr '16

29 Apr '16

commit d6f1fdfc9e411dff85164646a462a7823dac7152 Author: anadahz <kojgelo(a)inbox.com> Date: Fri Oct 16 01:33:35 2015 +0200 Add meek fronted requests documentation --- docs/source/tests/meek_fronted_requests.rst | 176 ++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) diff --git a/docs/source/tests/meek_fronted_requests.rst b/docs/source/tests/meek_fronted_requests.rst new file mode 100644 index 0000000..b865e18 --- /dev/null +++ b/docs/source/tests/meek_fronted_requests.rst @@ -0,0 +1,176 @@ +Details +======= + +*Test Name*: Meek Fronted Request Test + +*Current version*: 0.0.1 + +*NetTest*: HTTP Requests (https://gitweb.torproject.org/ooni-probe.git/blob/HEAD:/ooni/nettests/block…) + +*Test Helper*: None + +*Test Type*: Content Blocking, Traffic Manipulation + +*Requires Root*: No + +Description +=========== + +Performs a HTTP GET request to a list of fronted domains with the Host +Header of the "inside" meek-server. The meek-server handles a GET request and +response with: "I’m just a happy little web server.\n". + +It tests if the fronted request/response to the meek server is successful. + + +How to run the test +=================== + +To test if the default meek servers are blocked: + +`ooniprobe blocking/meek_fronted_requests` + +To test a set of meek servers from a list containing hosts to test run: + +`ooniprobe blocking/meek_fronted_requests -f <input_file>` + +*input file* a list of domainName:hostHeader pairs to perform the test. + +Optional test options +===================== + +*-B* Expected body content from GET response (meek server default: 'I’m just a +happy little web server.\n') +*-D* Specify a single fronted domainName to test. +*-H* Specify "inside" Host Header to test. + +Sample report +============= + +`ooniprobe blocking/meek_fronted_requests` + +:: + +########################################### +# OONI Probe Report for meek_fronted_requests_test (0.0.1) +# Thu Oct 15 22:01:32 2015 +########################################### +--- +input_hashes: [] +options: [] +probe_asn: AS20676 +probe_cc: DE +probe_city: null +probe_ip: 127.0.0.1 +report_id: null +software_name: ooniprobe +software_version: 1.3.1 +start_time: 1444932091.0 +test_helpers: {} +test_name: meek_fronted_requests_test +test_version: 0.0.1 +... +--- +agent: agent +input: [a0.awsstatic.com, d2zfqthxsdq309.cloudfront.net] +requests: +- request: + body: null + headers: + - - Host + - [d2zfqthxsdq309.cloudfront.net] + method: GET + tor: {is_tor: false} + url: https://a0.awsstatic.com + response: + body: "I\u2019m just a happy little web server.\n" + code: 200 + headers: + - - Content-Length + - ['38'] + - - Via + - [1.1 15191055e43ba835d0fead01ae84015c.cloudfront.net (CloudFront)] + - - X-Cache + - [Hit from cloudfront] + - - Age + - ['530'] + - - Connection + - [close] + - - X-Amz-Cf-Id + - [PKUBrpXDpoi2rSZ-WV0YUzX1wPg6JylZ_37iQeRQJB-xDLtJddcxzw==] + - - Date + - ['Thu, 15 Oct 2015 19:52:47 GMT'] + - - Content-Type + - [text/plain; charset=utf-8] +socksproxy: null +success: true +test_runtime: 0.3198120594024658 +test_start_time: 1444932097.0 +... +--- +agent: agent +input: [ajax.aspnetcdn.com, az668014.vo.msecnd.net] +requests: +- request: + body: null + headers: + - - Host + - [az668014.vo.msecnd.net] + method: GET + tor: {is_tor: false} + url: https://ajax.aspnetcdn.com + response: + body: "I\u2019m just a happy little web server.\n" + code: 200 + headers: + - - Date + - ['Thu, 15 Oct 2015 20:01:37 GMT'] + - - Content-Length + - ['38'] + - - Content-Type + - [text/plain; charset=utf-8] + - - Connection + - [close] + - - Server + - [ECAcc (fcn/40C4)] +socksproxy: null +success: true +test_runtime: 0.4580512046813965 +test_start_time: 1444932097.0 +... +--- +agent: agent +input: [www.google.com, meek-reflect.appspot.com] +requests: +- request: + body: null + headers: + - - Host + - [meek-reflect.appspot.com] + method: GET + tor: {is_tor: false} + url: https://www.google.com + response: + body: "I\u2019m just a happy little web server.\n" + code: 200 + headers: + - - Alternate-Protocol + - ['443:quic,p=1'] + - - Content-Length + - ['38'] + - - Server + - [Google Frontend] + - - Connection + - [close] + - - Date + - ['Thu, 15 Oct 2015 20:01:38 GMT'] + - - Alt-Svc + - ['quic=":443"; p="1"; ma=604800'] + - - Content-Type + - [text/plain; charset=utf-8] +socksproxy: null +success: true +test_runtime: 0.5862560272216797 +test_start_time: 1444932097.0 +... + ...

1 0

[ooni-probe/master] Use the latest version of txsocksx that includes a fix for 15.0
by art＠torproject.org 29 Apr '16

29 Apr '16

commit fc752878a97e242930fc05c46d356222b0be290f Author: Arturo Filastò <art(a)fuffa.org> Date: Mon Jul 27 16:52:33 2015 +0200 Use the latest version of txsocksx that includes a fix for 15.0 --- ooni/reporter.py | 2 +- ooni/utils/hacks.py | 27 --------------------------- ooni/utils/trueheaders.py | 5 +++-- requirements.txt | 2 +- 4 files changed, 5 insertions(+), 31 deletions(-) diff --git a/ooni/reporter.py b/ooni/reporter.py index a62cb6b..c62fa90 100644 --- a/ooni/reporter.py +++ b/ooni/reporter.py @@ -287,8 +287,8 @@ class OONIBReporter(OReporter): # do this with some deferred kung foo or instantiate the reporter after # tor is started. - from ooni.utils.hacks import SOCKS5Agent from twisted.web.client import Agent + from txsocksx.http import SOCKS5Agent from twisted.internet import reactor if self.collectorAddress.startswith('httpo://'): diff --git a/ooni/utils/hacks.py b/ooni/utils/hacks.py index e001d3f..7ec3d0f 100644 --- a/ooni/utils/hacks.py +++ b/ooni/utils/hacks.py @@ -58,30 +58,3 @@ def patched_reduce_ex(self, proto): return copy_reg._reconstructor, args, dict else: return copy_reg._reconstructor, args - - -class SOCKS5Agent(SOCKS5AgentOriginal): - """ - This is a quick hack to fix: - https://github.com/habnabit/txsocksx/issues/9 - """ - def _getEndpoint(self, scheme_or_uri, host=None, port=None): - if host is not None: - scheme = scheme_or_uri - else: - scheme = scheme_or_uri.scheme - host = scheme_or_uri.host - port = scheme_or_uri.port - if scheme not in ('http', 'https'): - raise SchemeNotSupported('unsupported scheme', scheme) - endpoint = self.endpointFactory( - host, port, self.proxyEndpoint, **self.endpointArgs) - if scheme == 'https': - if hasattr(self, '_wrapContextFactory'): - tlsPolicy = self._wrapContextFactory(host, port) - elif hasattr(self, '_policyForHTTPS'): - tlsPolicy = self._policyForHTTPS.creatorForNetloc(host, port) - else: - raise NotImplementedError("can't figure out how to make a context factory") - endpoint = self._tlsWrapper(tlsPolicy, endpoint) - return endpoint diff --git a/ooni/utils/trueheaders.py b/ooni/utils/trueheaders.py index f015b62..3905949 100644 --- a/ooni/utils/trueheaders.py +++ b/ooni/utils/trueheaders.py @@ -8,11 +8,12 @@ import itertools from copy import copy from twisted.web import client, _newclient, http_headers -from twisted.web._newclient import RequestNotSent, RequestGenerationFailed, TransportProxyProducer, STATUS +from twisted.web._newclient import RequestNotSent, RequestGenerationFailed +from twisted.web._newclient import TransportProxyProducer, STATUS from twisted.internet import reactor from twisted.internet.defer import Deferred, fail, maybeDeferred, failure -from ooni.utils.hacks import SOCKS5Agent +from txsocksx.http import SOCKS5Agent from txsocksx.client import SOCKS5ClientFactory SOCKS5ClientFactory.noisy = False diff --git a/requirements.txt b/requirements.txt index 50e82f2..8b8d5ca 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,5 @@ PyYAML>=3.10 -Twisted>=12.2.0,<=14.0.0 +Twisted>=12.2.0 ipaddr>=2.1.10 pyOpenSSL>=0.13 geoip

1 0

[ooni-probe/master] Increase the timeout of director tests or travis will be sad
by art＠torproject.org 29 Apr '16

29 Apr '16

commit 3e047141058c2f9d172e7f6ff79055b73b629526 Author: Arturo Filastò <art(a)fuffa.org> Date: Mon Jul 27 16:52:50 2015 +0200 Increase the timeout of director tests or travis will be sad --- ooni/tests/test_oonicli.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ooni/tests/test_oonicli.py b/ooni/tests/test_oonicli.py index 2ca94e3..9e73061 100644 --- a/ooni/tests/test_oonicli.py +++ b/ooni/tests/test_oonicli.py @@ -59,6 +59,8 @@ tor: class TestRunDirector(ConfigTestCase): + timeout = 220 + def setUp(self): super(TestRunDirector, self).setUp() if not is_internet_connected():

1 0

[ooni-probe/master] Cosmetic fix
by art＠torproject.org 29 Apr '16

29 Apr '16

commit 4a3dbc875fd95ecbecc86e8a2c7ed313cac65fd8 Author: Arturo Filastò <art(a)fuffa.org> Date: Mon Jul 27 16:53:04 2015 +0200 Cosmetic fix --- ooni/utils/trueheaders.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ooni/utils/trueheaders.py b/ooni/utils/trueheaders.py index 3905949..ac73970 100644 --- a/ooni/utils/trueheaders.py +++ b/ooni/utils/trueheaders.py @@ -68,7 +68,7 @@ class TrueHeaders(http_headers.Headers): except KeyError: pass - for k, v in itertools.chain(headers_a.getAllRawHeaders(), \ + for k, v in itertools.chain(headers_a.getAllRawHeaders(), headers_b.getAllRawHeaders()): field_names.append(k)

1 0