May 2013 - tor-commits - lists.torproject.org

[oonib/master] Fix typo in README.md installation commands.
by isis＠torproject.org 04 May '13

04 May '13

commit 62711b525541aa16d08d9a8fe256c84b3d029134 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 19:22:50 2013 +0000 Fix typo in README.md installation commands. --- README.md | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/README.md b/README.md index bcc3a6b..64b5a2e 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ setting up an ooni-backend are: # Create the virtualenv for ooni-backend... mkvirtualenv -a $PWD --unzip-setuptools --setuptools --no-site-packages oonib # ...and install ooni-backend (sudo is not necessary since we're in a virtualenv): - python setpy.py install + python setup.py install [1] Note: pip>=1.3.0 is recommended because it included several security patches, including using forcing the use of HTTPS and checking package hash

1 0

[oonib/master] Force usage of newer pip even if a distro version is installed.
by isis＠torproject.org 04 May '13

04 May '13

commit a3c97078701af6253114376abcc50af6b72568f8 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 19:49:37 2013 +0000 Force usage of newer pip even if a distro version is installed. --- README.md | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/README.md b/README.md index 64b5a2e..83544a6 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,7 @@ setting up an ooni-backend are: # Grab the get-pip installer to make sure we have pip>=1.3.0 [1] curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py sudo python ./get-pip.py ## pip (>=1.3.0) is recommended for security reasons + sudo update-alternatives --install /usr/bin/pip pip /usr/local/bin/pip 0 # We recommend using a virtualenv and virtualenvwrapper makes this easier: sudo pip install virtualenv virtualenvwrapper # Setup the virtualenv directory:

1 0

[oonib/master] Remove outdated .gitmodules file.
by isis＠torproject.org 04 May '13

04 May '13

commit 764b4ae4fadb0761645b64fb57d3f910617d48f5 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 20:57:43 2013 +0000 Remove outdated .gitmodules file. --- .gitmodules | 9 --------- 1 files changed, 0 insertions(+), 9 deletions(-) diff --git a/.gitmodules b/.gitmodules deleted file mode 100644 index daddf8f..0000000 --- a/.gitmodules +++ /dev/null @@ -1,9 +0,0 @@ -[submodule "ooni/lib/txscapy"] - path = ooni/lib/txscapy - url = https://github.com/isislovecruft/txscapy.git -[submodule "ooni/lib/txtraceroute"] - path = ooni/lib/txtraceroute - url = https://github.com/isislovecruft/txtraceroute.git -[submodule "ooni/lib/txtorcon"] - path = ooni/lib/txtorcon - url = https://github.com/isislovecruft/txtorcon.git

1 0

[oonib/master] Force pip to upgrade virtualenv if it was already installed.
by isis＠torproject.org 04 May '13

04 May '13

commit 75d3ea703ca0d122ca161188163ccf074c500858 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 20:21:17 2013 +0000 Force pip to upgrade virtualenv if it was already installed. --- README.md | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 83544a6..00c11d4 100644 --- a/README.md +++ b/README.md @@ -10,19 +10,17 @@ setting up an ooni-backend are: # Get git, python, and setuptools: sudo apt-get install git python2.7 python2.7-dev python-setuptools - cd # Grab the get-pip installer to make sure we have pip>=1.3.0 [1] curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py sudo python ./get-pip.py ## pip (>=1.3.0) is recommended for security reasons sudo update-alternatives --install /usr/bin/pip pip /usr/local/bin/pip 0 # We recommend using a virtualenv and virtualenvwrapper makes this easier: - sudo pip install virtualenv virtualenvwrapper + sudo pip install --upgrade virtualenv virtualenvwrapper # Setup the virtualenv directory: export WORKON_HOME=~/.virtualenvs && mkdir -p $WORKON_HOME source /usr/local/bin/virtualenvwrapper.sh # Clone ooni-backend: - git clone https://github.com/TheTorProject/ooni-backend.git - cd ooni-backend + git clone https://github.com/TheTorProject/ooni-backend.git && cd ooni-backend # Create the virtualenv for ooni-backend... mkvirtualenv -a $PWD --unzip-setuptools --setuptools --no-site-packages oonib # ...and install ooni-backend (sudo is not necessary since we're in a virtualenv):

1 0

[oonib/master] Remove unused modules from requirements.txt.
by isis＠torproject.org 04 May '13

04 May '13

commit c93088a5ff17c8604c24d97c4a35e6345ea52b64 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 21:00:44 2013 +0000 Remove unused modules from requirements.txt. --- requirements.txt | 9 --------- 1 files changed, 0 insertions(+), 9 deletions(-) diff --git a/requirements.txt b/requirements.txt index 5a21cf2..c2b4545 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,20 +1,11 @@ PyYAML>=3.10 -Pygments>=1.5 Twisted>=12.2.0 -argparse>=1.2.1 cyclone>=1.0-rc13 -docutils>=0.9.1 ipaddr>=2.1.10 -pyOpenSSL>=0.13 pygeoip>=0.2.5 # Originally fetched from the hg repo on secdev.org: # https://hg.secdev.org/scapy/archive/tip.zip#egg=scapy # This is a Tor Project mirror with valid SSL/TLS certs that is stable and fast: https://people.torproject.org/~ioerror/src/mirrors/ooniprobe/scapy-02-25-20… storm>=0.19 -transaction>=1.3.0 txtorcon -wsgiref>=0.1.2 -zope.component>=4.0.0 -zope.event>=4.0.0 -zope.interface>=4.0.1

1 0

[oonib/master] Remove ooni-probe specific (and outdated) documentation from HACKING file.
by isis＠torproject.org 04 May '13

04 May '13

commit 82c5050dc3355485b27e28b92f6399c68fa495a1 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 20:58:11 2013 +0000 Remove ooni-probe specific (and outdated) documentation from HACKING file. --- HACKING | 96 --------------------------------------------------------------- 1 files changed, 0 insertions(+), 96 deletions(-) diff --git a/HACKING b/HACKING index b92ddbe..44dfe01 100644 --- a/HACKING +++ b/HACKING @@ -36,109 +36,15 @@ Code Structure - HACKING The document you are currently reading. -- inputs/ - Contains input files for tests. - - oonib/ Contains the OONI probe backend to be run on the ooni-net -- ooni/ - Contains the main ooni probe comand line client - -- ooni/config.py - Parts of the code related to parsing OONI - configuration files and making them accessible - to other components of the software. - -- ooni/inputunit.py - In here we have functions related to the creation of input - units. Input units are how the inputs to be fed to tests are - split up into. - -- ooni/nettest.py - In here is the NetTest API definition. This is how people - interested in writing ooniprobe tests will be specifying - them. - -- ooni/nodes.py - Mostly broken code for the remote dispatching of tests. - -- ooni/oonicli.py - In here we take care of running ooniprobe from the command - line interface - -- ooni/reporter.py - In here goes the logic for the creation of ooniprobe - reports. - -- ooni/runner.py - Handles running ooni.nettests as well as - ooni.plugoo.tests.OONITests. - -- ooni/otime.py - Generation of timestamps, time conversions and all the rest - -- ooni/kit/ - In here go utilities that can be used by tests. - -- ooni/lib/ - XXX this directory is to be removed. - -- ooni/utils/ - In here go internal utilities that are useful to ooniprobe - -- ooni/utils/geodata.py - In here go functions related to the understanding of - geographical information of the probe - -- ooni/utils/hacks.py - When some software has issues and we need to fix it in a - hackish way, we put it in here. This one day will be empty. - -- ooni/utils/log.py - log realted functions. - -- ooni/utils/net.py - utilities for networking related operations - -- ooni/utils/onion.py - Utilities for working with Tor. - XXX this code should be removed and merged into txtorcon. - -- ooni/utils/txscapy.py - Tools for making scapy work well with twisted. - - ooniprobe.conf The main OONI-probe configuration file. This can be used to configure your OONI CLI, tell it where it should report to, where the asset files are located, what should be used for control, etc. -Test related conventions ------------------------- - -These are the conventions for tests that are written in ooniprobe. That is what -goes inside of nettests/. - -Naming -...... - -All methods that are relevant to the test should be all lower case separated by -underscore. - -All variables that are specific to your test should be all lower case separated -by underscore. - -Simplicity -.......... - -Tests written in ooniprobe should be as short as possible and should contain as -little code not related to the measurement as possible. It should be possible -from reading the test to understand what it does without clutter. - -Everything that is not related directly to the test should go inside of the -test template of which the test you are writing is a subclass of. - Style guide ----------- @@ -288,5 +194,3 @@ methods. Even better if you write them also for non-public methods. Place docstrings under the def. For a better overview on how to write docstrings consult: PEP-257 - -

1 0

[oonib/master] Specify minimum version of txtorcon in requirements.txt.
by isis＠torproject.org 04 May '13

04 May '13

commit aabb0d08e5c6c73bc73ec48f64e7361c360ae11f Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 21:01:12 2013 +0000 Specify minimum version of txtorcon in requirements.txt. --- requirements.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/requirements.txt b/requirements.txt index c2b4545..2daadd1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -8,4 +8,4 @@ pygeoip>=0.2.5 # This is a Tor Project mirror with valid SSL/TLS certs that is stable and fast: https://people.torproject.org/~ioerror/src/mirrors/ooniprobe/scapy-02-25-20… storm>=0.19 -txtorcon +txtorcon>=0.7

1 0

[oonib/master] Remove unused txtorcon import statement from /oonib/oonibackend.py.
by isis＠torproject.org 04 May '13

04 May '13

commit 48172b9d47552cf66bbe5173d80ae1c4e25fa1e9 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 20:59:09 2013 +0000 Remove unused txtorcon import statement from /oonib/oonibackend.py. --- oonib/oonibackend.py | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diff --git a/oonib/oonibackend.py b/oonib/oonibackend.py index 2345632..c042463 100644 --- a/oonib/oonibackend.py +++ b/oonib/oonibackend.py @@ -15,8 +15,6 @@ from twisted.names import dns from cyclone import web -import txtorcon - from oonib.testhelpers import dns_helpers, ssl_helpers from oonib.testhelpers import http_helpers, tcp_helpers

1 0

[oonib/master] Fuck you, pip.
by isis＠torproject.org 04 May '13

04 May '13

commit 140e8018447d9a69c9e1a8029cbe9b98f4730327 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 21:41:32 2013 +0000 Fuck you, pip. --- requirements.txt | 21 ++++++++++++++------- 1 files changed, 14 insertions(+), 7 deletions(-) diff --git a/requirements.txt b/requirements.txt index 2daadd1..4a25852 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,11 +1,18 @@ -PyYAML>=3.10 -Twisted>=12.2.0 -cyclone>=1.0-rc13 -ipaddr>=2.1.10 -pygeoip>=0.2.5 +https://pypi.python.org/packages/source/P/PyYAML/PyYAML-3.10.tar.gz#md5=74c94a383886519e9e7b3dd1ee540247#egg=PyYAML +https://pypi.python.org/packages/source/T/Twisted/Twisted-13.0.0.tar.bz2#md5=68afff4e1efd3757d934e39f70c99f57#egg=Twisted +https://pypi.python.org/packages/source/c/cyclone/cyclone-1.1.tar.gz#md5=477c5ef9cf8902e37105fe450c4d8c5d#egg=cyclone +https://ipaddr-py.googlecode.com/files/ipaddr-2.1.10.tar.gz#sha1=c608450b077b19773d4f1b5f1ef88b26f6650ce0#egg=ipaddr +https://pypi.python.org/packages/source/p/pygeoip/pygeoip-0.2.6.zip#md5=b3ac1bfcd535782bc59af78e722cf5c1#egg=pygeoip # Originally fetched from the hg repo on secdev.org: # https://hg.secdev.org/scapy/archive/tip.zip#egg=scapy # This is a Tor Project mirror with valid SSL/TLS certs that is stable and fast: https://people.torproject.org/~ioerror/src/mirrors/ooniprobe/scapy-02-25-20… -storm>=0.19 -txtorcon>=0.7 +https://pypi.python.org/packages/source/s/storm/storm-0.19.tar.gz#md5=61d1ee4cd2a08639ab917e43fa2c9265#egg=storm +# If any of the above failed, try the normal way: +#PyYAML>=3.10 +#Twisted>=12.2.0 +#cyclone>=1.0-rc13 +#ipaddr>=2.1.10 +#pygeoip>=0.2.5 +#storm>=0.19 +#txtorcon>=0.7

1 0

[oonib/master] Force pip to forget whatever builtin PI it thought it thought it should use.
by isis＠torproject.org 04 May '13

04 May '13

commit 2e62b047252b01e213644bbc2e836e4d481c16a7 Author: Isis Lovecruft <isis(a)torproject.org> Date: Fri Apr 19 21:43:42 2013 +0000 Force pip to forget whatever builtin PI it thought it thought it should use. --- README.md | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/README.md b/README.md index 00c11d4..bbad4e2 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,7 @@ setting up an ooni-backend are: # Create the virtualenv for ooni-backend... mkvirtualenv -a $PWD --unzip-setuptools --setuptools --no-site-packages oonib # ...and install ooni-backend (sudo is not necessary since we're in a virtualenv): + pip install --global-option --no-index -r requirements.txt python setup.py install [1] Note: pip>=1.3.0 is recommended because it included several security

1 0