November 2015 - tor-commits - lists.torproject.org

[ooni-probe/master] Attempt to fix https://trac.torproject.org/projects/tor/ticket/16713
by art＠torproject.org 20 Nov '15

20 Nov '15

commit 3e3c18367fdf018f2d919fdc89f5295332adc212 Author: Arturo Filastò <art(a)fuffa.org> Date: Tue Aug 4 18:18:21 2015 +0200 Attempt to fix https://trac.torproject.org/projects/tor/ticket/16713 --- scripts/install.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/scripts/install.sh b/scripts/install.sh index 016c072..a63c506 100644 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -12,6 +12,7 @@ TOR_DEB_REPO="http://deb.torproject.org/torproject.org" CLOUDFRONT="no" INSTALL_PT="yes" PYTHONPATH=$(python -c "import sys; print ':'.join(x for x in sys.path if x)") +PYTHON_PREFIX="/usr/local" # These are the minimum ubuntu and debian version required to use the debian # package. @@ -249,7 +250,7 @@ install_pluggable_transports() { install_pluggable_transport_deps ( set -x - PYTHONPATH=$PYTHONPATH $sh_c 'pip install obfsproxy fteproxy' + $sh_c "PYTHONPATH=$(PYTHONPATH) pip install --install-option=\"--prefix=$PYTHON_PREFIX\" obfsproxy fteproxy" ) install_obfs4proxy install_meek @@ -269,7 +270,7 @@ case "$lsb_dist" in $sh_c "${yum} -y groupinstall \"Development tools\"" $sh_c "${yum} -y install zlib-devel bzip2-devel openssl-devel sqlite-devel libpcap-devel libffi-devel libevent-devel GeoIP-devel tor python-devel libdnet-devel gcc-c++" install_pip - PYTHONPATH=$PYTHONPATH $sh_c 'pip install ooniprobe' + $sh_c "PYTHONPATH=$(PYTHONPATH) pip install --install-option=\"--prefix=$PYTHON_PREFIX\" ooniprobe" ) install_pluggable_transports @@ -322,7 +323,7 @@ case "$lsb_dist" in ( set -x $sh_c 'apt-get install -y -q curl git-core python python-dev python-setuptools build-essential libdumbnet1 python-dumbnet python-libpcap tor tor-geoipdb libgeoip-dev libpcap0.8-dev libssl-dev libffi-dev libdumbnet-dev' - PYTHONPATH=$PYTHONPATH $sh_c 'pip install ooniprobe' + $sh_c "PYTHONPATH=$(PYTHONPATH) pip install --install-option=\"--prefix=$PYTHON_PREFIX\" ooniprobe" ) fi

1 0

[ooni-probe/master] Merge pull request #409 from juga0/fix/process
by art＠torproject.org 20 Nov '15

20 Nov '15

commit 65596e9d92894e5e9e9fcdd6f7e86a06175236dc Merge: 77ce0b5 f91d136 Author: Arturo Filastò <arturo(a)filasto.net> Date: Sun Sep 20 13:57:43 2015 +0200 Merge pull request #409 from juga0/fix/process Fix/process ooni/templates/process.py | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-)

1 0

[ooni-probe/master] Merge pull request #407 from TheTorProject/fix/readme
by art＠torproject.org 20 Nov '15

20 Nov '15

commit a1c75bd4d9dcc4e3c5a16434564ff29b6eb5ee4f Merge: a3b6e77 c35d1f6 Author: Simone Basso <bassosimone(a)gmail.com> Date: Fri Sep 11 12:48:28 2015 +0200 Merge pull request #407 from TheTorProject/fix/readme Bug: add notes to uninstall/downgrade Twisted via pip README.rst | 6 ++++++ 1 file changed, 6 insertions(+)

1 0

[ooni-probe/master] Merge pull request #399 from TheTorProject/feature/install
by art＠torproject.org 20 Nov '15

20 Nov '15

commit 77ce0b5c8e62c13fca950979aa4f2fdf5e36ed6f Merge: a1c75bd 7609ffc Author: anadahz <anadahz(a)users.noreply.github.com> Date: Mon Sep 14 00:28:39 2015 +0000 Merge pull request #399 from TheTorProject/feature/install Feature/install scripts/install.sh | 352 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 352 insertions(+)

1 0

[ooni-probe/master] Proper order of installation steps
by art＠torproject.org 20 Nov '15

20 Nov '15

commit d3a8881cd300cdf6e8c4df41985d43fbb983e842 Author: Ulysses Aalto <uaalto(a)getlantern.org> Date: Sun Oct 11 12:29:27 2015 -0400 Proper order of installation steps --- README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.rst b/README.rst index 730d8f0..68a56c0 100644 --- a/README.rst +++ b/README.rst @@ -388,9 +388,9 @@ On Debian based systems a development environment can be setup as follows: (prer git clone https://github.com/TheTorProject/ooni-probe cd ooni-probe mkvirtualenv ooniprobe # . ~/.virtualenvs/ooniprobe/bin/activate to access later - python setup.py install pip install -r requirements.txt pip install -r requirements-dev.txt + python setup.py install ooniprobe -s # if all went well, lists available tests

1 0

[ooni-probe/master] Fix: Don't ask for confirmation of uninstall deletions
by art＠torproject.org 20 Nov '15

20 Nov '15

commit 978ea69a8dd1a004947acade9138e1cf405339b9 Author: anadahz <kojgelo(a)inbox.com> Date: Wed Sep 9 17:13:41 2015 +0200 Fix: Don't ask for confirmation of uninstall deletions --- scripts/install.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/install.sh b/scripts/install.sh index 7897471..5deff38 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -326,7 +326,7 @@ case "$lsb_dist" in $sh_c "PYTHONPATH=$PYTHONPATH pip install --install-option=\"--prefix=$PYTHON_PREFIX\" ooniprobe" # Unistall Twisted and install a version >=12.2.0,<=14.0.0 # https://trac.torproject.org/projects/tor/ticket/15944 - $sh_c "PYTHONPATH=$PYTHONPATH pip uninstall Twisted" + $sh_c "PYTHONPATH=$PYTHONPATH pip uninstall -y Twisted" $sh_c "PYTHONPATH=$PYTHONPATH pip install --install-option=\"--prefix=$PYTHON_PREFIX\" 'Twisted>=12.2.0,<=14.0.0'" )

1 0

[ooni-probe/master] Check spelling with codespell
by art＠torproject.org 20 Nov '15

20 Nov '15

commit a02799fdc9fb687af2141f9938df8b2aac33b9d7 Author: Kevin Murray <spam(a)kdmurray.id.au> Date: Fri Sep 25 21:26:40 2015 +1000 Check spelling with codespell --- HACKING | 2 +- README.rst | 2 +- Vagrantfile | 2 +- data/ooniprobe.1 | 2 +- docs/source/index.rst | 2 +- ooni/geoip.py | 2 +- ooni/nettests/experimental/bridge_reachability/echo.py | 2 +- ooni/nettests/manipulation/dns_spoof.py | 2 +- ooni/nettests/scanning/http_url_list.py | 2 +- ooni/tests/test_settings.py | 4 ++-- ooni/utils/txscapy.py | 2 +- setup.py | 2 +- var/old_notes.txt | 2 +- var/proxooni-spec.txt | 2 +- 14 files changed, 15 insertions(+), 15 deletions(-) diff --git a/HACKING b/HACKING index fdb3751..baa5bb0 100644 --- a/HACKING +++ b/HACKING @@ -206,7 +206,7 @@ Indentation Tabs or Spaces? ............... -Everytime you insert a \t into any piece of code a kitten dies. +Every time you insert a \t into any piece of code a kitten dies. Only spaces. Please. diff --git a/README.rst b/README.rst index 66b712c..9cb3042 100644 --- a/README.rst +++ b/README.rst @@ -40,7 +40,7 @@ aware of the fact that you are running ooniprobe. This means that if running network measurement tests is something considered to be illegal in your country then you could be spotted. -Futhermore, ooniprobe takes no precautions to protect the install target machine +Furthermore, ooniprobe takes no precautions to protect the install target machine from forensics analysis. If the fact that you have installed or used ooni probe is a liability for you, please be aware of this risk. diff --git a/Vagrantfile b/Vagrantfile index 4f9bc9b..16a6ba0 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -30,7 +30,7 @@ apt-get install -y tor deb.torproject.org-keyring apt-get install -y build-essential libdumbnet-dev python-dumbnet python-pypcap libpcap-dev python-dev python-pip libgeoip-dev libffi-dev cd /ooni python setup.py install -echo "Login using 'vagrant ssh', and dont forget to run ooniprobe as root." +echo "Login using 'vagrant ssh', and don't forget to run ooniprobe as root." echo "First run: 'sudo ooniprobe -i /usr/share/ooni/decks/fast.deck'" SCRIPT diff --git a/data/ooniprobe.1 b/data/ooniprobe.1 index 69e738a..cf3f593 100644 --- a/data/ooniprobe.1 +++ b/data/ooniprobe.1 @@ -178,7 +178,7 @@ aware of the fact that you are running ooniprobe. This means that if running network measurement tests is something considered to be illegal in your country then you could be spotted. .sp -Futhermore, ooniprobe takes no precautions to protect the install target machine +Furthermore, ooniprobe takes no precautions to protect the install target machine from forensics analysis. If the fact that you have installed or used ooni probe is a liability for you, please be aware of this risk. .SS Debian based systems diff --git a/docs/source/index.rst b/docs/source/index.rst index dd9da6f..10c7c8f 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -49,7 +49,7 @@ aware of the fact that you are running ooniprobe. This means that if running network measurement tests is something considered to be illegal in your country then you could be spotted. -Futhermore, ooniprobe takes no precautions to protect the install target machine +Furthermore, ooniprobe takes no precautions to protect the install target machine from forensics analysis. If the fact that you have installed or used ooni probe is a liability for you, please be aware of this risk. diff --git a/ooni/geoip.py b/ooni/geoip.py index 26367ff..0ab2475 100644 --- a/ooni/geoip.py +++ b/ooni/geoip.py @@ -206,7 +206,7 @@ class ProbeIP(object): self.resolveGeodata() defer.returnValue(self.address) except errors.InsufficientPrivileges: - log.debug("Cannot determine the probe IP address with a traceroute, becase of insufficient priviledges") + log.debug("Cannot determine the probe IP address with a traceroute, becase of insufficient privileges") except: log.msg("Unable to lookup the probe IP via traceroute") diff --git a/ooni/nettests/experimental/bridge_reachability/echo.py b/ooni/nettests/experimental/bridge_reachability/echo.py index e64a604..bd4b6a1 100644 --- a/ooni/nettests/experimental/bridge_reachability/echo.py +++ b/ooni/nettests/experimental/bridge_reachability/echo.py @@ -113,7 +113,7 @@ class EchoTest(nettest.NetTestCase): def process_response(echo_reply, dest): ans, unans = echo_reply if ans: - log.msg("Recieved echo reply from %s: %s" % (dest, ans)) + log.msg("Received echo reply from %s: %s" % (dest, ans)) else: log.msg("No reply was received from %s. Possible censorship event." % dest) log.debug("Unanswered packets: %s" % unans) diff --git a/ooni/nettests/manipulation/dns_spoof.py b/ooni/nettests/manipulation/dns_spoof.py index a73ad6e..20dbd66 100644 --- a/ooni/nettests/manipulation/dns_spoof.py +++ b/ooni/nettests/manipulation/dns_spoof.py @@ -48,7 +48,7 @@ class DNSSpoof(scapyt.ScapyTest): def postProcessor(self, measurements): """ This is not tested, but the concept is that if the two responses - match up then spoofing is occuring. + match up then spoofing is occurring. """ try: test_answer = self.report['answered_packets'][0][UDP] diff --git a/ooni/nettests/scanning/http_url_list.py b/ooni/nettests/scanning/http_url_list.py index f89def4..e3a4957 100644 --- a/ooni/nettests/scanning/http_url_list.py +++ b/ooni/nettests/scanning/http_url_list.py @@ -58,7 +58,7 @@ class HTTPURLList(httpt.HTTPTest): censorship_page = open(self.localOptions['content']).xreadlines() response_page = iter(body.split("\n")) - # We first allign the two pages to the first HTML tag (something + # We first align the two pages to the first HTML tag (something # starting with <). This is useful so that we can give as input to this # test something that comes from the output of curl -kis # http://the_page/ diff --git a/ooni/tests/test_settings.py b/ooni/tests/test_settings.py index 1d36b83..6b7c3b9 100644 --- a/ooni/tests/test_settings.py +++ b/ooni/tests/test_settings.py @@ -83,9 +83,9 @@ class TestSettings(ConfigTestCase): conditions that make it not possible to run it on travis. The tests need to be run as root on travis so that the ones that use scapy will work properly. When running tor as root, though, it will by - default drop priviledges to a lesser priviledged user (on debian based + default drop privileges to a lesser priviledged user (on debian based systems debian-tor). The problem is that the datadir will have already - been created with the priviledges of root, hence it will fail to use it + been created with the privileges of root, hence it will fail to use it as a datadir and fail. txtorcon addressed this issue in https://github.com/meejah/txtorcon/issues/26 by chmodding the datadir with what is set as User. diff --git a/ooni/utils/txscapy.py b/ooni/utils/txscapy.py index 59ac5b1..27b68bd 100644 --- a/ooni/utils/txscapy.py +++ b/ooni/utils/txscapy.py @@ -401,7 +401,7 @@ class MPTraceroute(ScapyProtocol): """ Add a packet into the received packets dictionary, typically the key is a tuple of packet fields used - to correlate sent packets with recieved packets. + to correlate sent packets with received packets. """ # Initialize or append to the lists of packets diff --git a/setup.py b/setup.py index f21ab83..9b874f3 100644 --- a/setup.py +++ b/setup.py @@ -29,7 +29,7 @@ aware of the fact that you are running ooniprobe. This means that if running network measurement tests is something considered to be illegal in your country then you could be spotted. -Futhermore, ooniprobe takes no precautions to protect the install target machine +Furthermore, ooniprobe takes no precautions to protect the install target machine from forensics analysis. If the fact that you have installed or used ooni probe is a liability for you, please be aware of this risk. diff --git a/var/old_notes.txt b/var/old_notes.txt index 81d834f..4303a4c 100644 --- a/var/old_notes.txt +++ b/var/old_notes.txt @@ -9,7 +9,7 @@ We should act as a controller with TorCtl to do this, etc We should take the Tor consensus file and pass it to plugins such as marco HTTP Host header comparsion of a vs b -HTTP Content length header comparision of a vs b +HTTP Content length header comparison of a vs b GET request splitting "G E T " diff --git a/var/proxooni-spec.txt b/var/proxooni-spec.txt index 7cc476f..7a561d4 100644 --- a/var/proxooni-spec.txt +++ b/var/proxooni-spec.txt @@ -36,7 +36,7 @@ ooni-proxy should provide open access by default with no authentication. It should support TLS-PSK[0] if authentication is desired. Key distribution is - explictly an out of scope problem. + explicitly an out of scope problem. 3.0 Services offered

1 0

[ooni-probe/master] Merge pull request #414 from kdmurray91/daube/spellcheck
by art＠torproject.org 20 Nov '15

20 Nov '15

commit be134f784820103a6f0ce8725e2ebe2880903d41 Merge: 65596e9 a02799f Author: Simone Basso <bassosimone(a)gmail.com> Date: Fri Sep 25 22:02:14 2015 +0200 Merge pull request #414 from kdmurray91/daube/spellcheck A bunch of little spelling fixes HACKING | 2 +- README.rst | 2 +- Vagrantfile | 2 +- data/ooniprobe.1 | 2 +- docs/source/index.rst | 2 +- ooni/geoip.py | 2 +- ooni/nettests/experimental/bridge_reachability/echo.py | 2 +- ooni/nettests/manipulation/dns_spoof.py | 2 +- ooni/nettests/scanning/http_url_list.py | 2 +- ooni/tests/test_settings.py | 4 ++-- ooni/utils/txscapy.py | 2 +- setup.py | 2 +- var/old_notes.txt | 2 +- var/proxooni-spec.txt | 2 +- 14 files changed, 15 insertions(+), 15 deletions(-)

1 0

[ooni-probe/master] Merge pull request #418 from uaalto/patch-1
by art＠torproject.org 20 Nov '15

20 Nov '15

commit b2fdc20e46f7dd862248757fe8c241e04542a37d Merge: ed0fdf5 d3a8881 Author: Simone Basso <bassosimone(a)gmail.com> Date: Sun Oct 11 20:00:43 2015 +0200 Merge pull request #418 from uaalto/patch-1 Proper order of installation steps README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

1 0

[ooni-probe/master] Add Bridge Reachability Test documentation
by art＠torproject.org 20 Nov '15

20 Nov '15

commit dd1668496d90f99a17fe8813d7848453bf1886c6 Author: anadahz <kojgelo(a)inbox.com> Date: Tue Oct 13 06:03:57 2015 +0200 Add Bridge Reachability Test documentation --- docs/source/tests/bridge_reachability.rst | 129 +++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) diff --git a/docs/source/tests/bridge_reachability.rst b/docs/source/tests/bridge_reachability.rst new file mode 100644 index 0000000..3f8999b --- /dev/null +++ b/docs/source/tests/bridge_reachability.rst @@ -0,0 +1,129 @@ +Details +======= + +*Test Name*: Bridge Reachability Test + +*Current version*: 0.1.2 + +*NetTest*: HTTP Requests (https://gitweb.torproject.org/ooni-probe.git/blob/HEAD:/ooni/nettests/block…) + +*Test Helper*: None + +*Test Type*: Content Blocking + +*Requires Root*: No + +Description +=========== + +This test detects which Tor bridges are working from the given network vantage +point and which ones are not. For every bridge specified as input a new tor +instance is started and it is instructed to connect to Tor using that bridge. +It will try to bootstrap to 100% and either succeed or timeout after a default +time of 120 seconds. + +How to run the test +=================== + +`ooniprobe blocking/bridge_reachability.py -f <input_file>` + +*input file* a list of TransportType IP:ORPort bridges (ex. obfs3 +198.51.100.45:9045) or IP:ORPort (ex. 203.0.113.27:1828) to test reachability +for. + +Optional test options +===================== + +*-t* Specify the timeout after which to consider the Tor bootstrapping process +to have failed. The default is 120 seconds. + +Sample report +============= + +`ooniprobe blocking/bridge_reachability.py` + +:: + +########################################### +# OONI Probe Report for bridge_reachability (0.1) +# Thu Aug 7 16:10:50 2014 +########################################### +--- +input_hashes: [595c8ad9b63ff9f142eca6296021990df6367ab03b3c4eb1c69a8747f0cd41a1] +options: [-f, bridges.txt] +probe_asn: AS3269 +probe_cc: IT +probe_city: null +probe_ip: 127.0.0.1 +software_name: ooniprobe +software_version: 1.0.2 +start_time: 1407420650.560137 +test_name: bridge_reachability +test_version: '0.1' +... +--- +{bridge_address: '169.229.59.74:31493', error: null, input: 'obfs3 169.229.59.74:31493 + AF9F66B7B04F8FF6F32D455F05135250A16543C9', success: true, timeout: 120, tor_log: null, + tor_progress: 100, tor_progress_summary: Done, tor_progress_tag: done, tor_version: 0.2.4.20, + transport_name: obfs3} +... +--- +{bridge_address: '169.229.59.75:46328', error: null, input: 'obfs3 169.229.59.75:46328 + AF9F66B7B04F8FF6F32D455F05135250A16543C9', success: true, timeout: 120, tor_log: null, + tor_progress: 100, tor_progress_summary: Done, tor_progress_tag: done, tor_version: 0.2.4.20, + transport_name: obfs3} +... +--- +{bridge_address: '208.79.90.242:35658', error: null, input: 'obfs3 208.79.90.242:35658 + BA61757846841D64A83EA2514C766CB92F1FB41F', success: true, timeout: 120, tor_log: null, + tor_progress: 100, tor_progress_summary: Done, tor_progress_tag: done, tor_version: 0.2.4.20, + transport_name: obfs3} +... +--- +{bridge_address: '83.212.100.216:47870', error: null, input: 'obfs2 83.212.100.216:47870 + 1F01A7BB60F49FC96E0850A6BAD6D076DFEFAF80', success: false, timeout: 120, tor_log: '', + tor_progress: 0, tor_progress_summary: null, tor_progress_tag: null, tor_version: 0.2.4.20, + transport_name: obfs2} +... +--- +{bridge_address: '83.212.96.182:46602', error: null, input: 'obfs2 83.212.96.182:46602 + 6F058CBEF888EB20D1DEB9886909F1E812245D41', success: false, timeout: 120, tor_log: '', + tor_progress: 0, tor_progress_summary: null, tor_progress_tag: null, tor_version: 0.2.4.20, + transport_name: obfs2} +... +--- +{bridge_address: '70.182.182.109:54542', error: null, input: 'obfs2 70.182.182.109:54542 + 94C9E691688FAFDEC701A0788BD15BE8AD34ED35', success: false, timeout: 120, tor_log: '', + tor_progress: 0, tor_progress_summary: null, tor_progress_tag: null, tor_version: 0.2.4.20, + transport_name: obfs2} +... +--- +{bridge_address: '128.31.0.34:1051', error: null, input: 'obfs2 128.31.0.34:1051 CA7434F14A898C7D3427B8295A7F83446BC7F496', + success: false, timeout: 120, tor_log: '', tor_progress: 0, tor_progress_summary: null, + tor_progress_tag: null, tor_version: 0.2.4.20, transport_name: obfs2} +... +--- +{bridge_address: '83.212.101.2:45235', error: null, input: 'obfs2 83.212.101.2:45235 + 2ADFE7AA8D272C520D1FBFBF4E413F3A1B26313D', success: false, timeout: 120, tor_log: '', + tor_progress: 0, tor_progress_summary: null, tor_progress_tag: null, tor_version: 0.2.4.20, + transport_name: obfs2} +... +--- +{bridge_address: '83.212.101.2:42782', error: null, input: 'obfs3 83.212.101.2:42782 + 2ADFE7AA8D272C520D1FBFBF4E413F3A1B26313D', success: false, timeout: 120, tor_log: '', + tor_progress: 0, tor_progress_summary: null, tor_progress_tag: null, tor_version: 0.2.4.20, + transport_name: obfs3} +... +--- +{bridge_address: '83.212.101.2:443', error: null, input: 'obfs3 83.212.101.2:443 2ADFE7AA8D272C520D1FBFBF4E413F3A1B26313D', + success: false, timeout: 120, tor_log: '', tor_progress: 0, tor_progress_summary: null, + tor_progress_tag: null, tor_version: 0.2.4.20, transport_name: obfs3} +... +--- +{bridge_address: '209.141.36.236:45496', error: null, input: 'obfs3 209.141.36.236:45496 + 58D91C3A631F910F32E18A55441D5A0463BA66E2', success: false, timeout: 120, tor_log: '', + tor_progress: 0, tor_progress_summary: null, tor_progress_tag: null, tor_version: 0.2.4.20, + transport_name: obfs3} +... + ... +

1 0