commit e6035577f33b8d34a10117bf30bffda45719896a Author: anadahz anadahz@users.noreply.github.com Date: Mon Nov 21 18:25:30 2016 +0000

Add ooniprobe v2.0.0 install instructions, cleanup, fix URLs (#649)

* Add ooniprobe v2.0.0 install instructions, cleanup, fix URLs

* Add installation instructions for ooniprobe v2.0.0 * Update git repository URLs * Cleanup obsolete old documentation

* Remove extra dependencies, shape service install instructions

* Add ooniprobe package install instructions fix port,vagrant option

* Add Debian and Ubuntu package based install instructions * Fix web UI port and vagrant option

* Remove end of life wily distribution install instructions --- README.rst | 229 ++++++++++++++++++------------------------------------------- 1 file changed, 67 insertions(+), 162 deletions(-)

diff --git a/README.rst b/README.rst index d232188..0b9d3d2 100644 --- a/README.rst +++ b/README.rst @@ -47,50 +47,46 @@ probe is a liability for you, please be aware of this risk. OONI in 5 minutes =================

-On Debian testing or unstable:: - - sudo apt-get install ooniprobe +The latest ooniprobe version for Debian and Ubuntu releases can be found in the +deb.torproject.org package repository.

On Debian stable (jessie)::

- echo 'deb http://ftp.debian.org/debian jessie-backports main' | sudo tee -a /etc/apt/sources.list + gpg --keyserver keys.gnupg.net --recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 + gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add - + echo 'deb http://deb.torproject.org/torproject.org jessie main' | sudo tee /etc/apt/sources.list.d/ooniprobe.list sudo apt-get update - sudo apt-get install ooniprobe + sudo apt-get install ooniprobe deb.torproject.org-keyring

-On Ubuntu 16.04 (xenial), 15.10 (wily) or 14.04 (trusty):: +On Debian testing::

- echo 'deb http://deb.torproject.org/torproject.org $RELEASE main' | sudo tee -a /etc/apt/sources.list + gpg --keyserver keys.gnupg.net --recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 + gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add - + echo 'deb http://deb.torproject.org/torproject.org testing main' | sudo tee /etc/apt/sources.list.d/ooniprobe.list sudo apt-get update - sudo apt-get install ooniprobe - -Note: You'll need to swap out ``$RELEASE`` for either ``xenial``, ``wily``, or -``trusty``. This will not happen automatically. You will also need to ensure -that you have the ``universe`` repository enabled. The ``universe`` repository -is enabled by default in a standard Ubuntu installation but may not be on some -minimal on not standard installations. - -On unix systems:: - - sudo pip install ooniprobe - -To install it from the current master run:: + sudo apt-get install ooniprobe deb.torproject.org-keyring

- sudo pip install https://github.com/TheTorProject/ooni-probe/archive/master.zip +On Debian unstable::

-Then run:: - - mkdir my_decks - oonideckgen -o my_decks/ - -If you're using the Debian package, you will be asked when installing whether -you would like to run OONI daily. On other platforms, if you would like to -contribute measurements to OONI daily you can also add this to your crontab:: + gpg --keyserver keys.gnupg.net --recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 + gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add - + echo 'deb http://deb.torproject.org/torproject.org unstable main' | sudo tee /etc/apt/sources.list.d/ooniprobe.list + sudo apt-get update + sudo apt-get install ooniprobe deb.torproject.org-keyring

- @daily ooniprobe $THE_OONI_COMMAND +On Ubuntu 16.10 (yakkety), 16.04 (xenial) or 14.04 (trusty)::

-Run this command to automatically update your crontab:: + gpg --keyserver keys.gnupg.net --recv A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 + gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add - + echo 'deb http://deb.torproject.org/torproject.org $RELEASE main' | sudo tee /etc/apt/sources.list.d/ooniprobe.list + sudo apt-get update + sudo apt-get install ooniprobe deb.torproject.org-keyring

- (crontab -l 2>/dev/null; echo "@daily ooniprobe $THE_OONI_COMMAND") | crontab - +Note: You'll need to swap out ``$RELEASE`` for either ``yakkety``, ``xenial`` or +``trusty``. This will not happen automatically. You will also need to ensure +that you have the ``universe`` repository enabled. The ``universe`` repository +is enabled by default in a standard Ubuntu installation but may not be on some +minimal on not standard installations.

Installation ============ @@ -118,15 +114,19 @@ Make sure you have installed the following dependencies: * libffi-dev * tor (>=0.2.5.1 to run all the tor related tests)

+Optional dependencies: + + * obfs4proxy + On debian based systems this can generally be done by running::

- sudo apt-get install -y build-essential libdumbnet-dev libpcap-dev libgeoip-dev libffi-dev python-dev python-pip tor + sudo apt-get install -y build-essential libdumbnet-dev libpcap-dev libgeoip-dev libffi-dev python-dev python-pip tor libssl-dev obfs4proxy

Then you should be able to install ooniprobe by running::

sudo pip install ooniprobe

-or as a user to install inside a virtualenv:: +or install ooniprobe as a user::

pip install ooniprobe

@@ -148,20 +148,20 @@ On debian/ubuntu::

3. Open a Terminal and run::

- git clone https://git.torproject.org/ooni-probe.git - git clone https://github.com/TheTorProject/ooni-backend.git + git clone https://github.com/TheTorProject/ooni-probe.git cd ooni-probe/ vagrant up

4. Login to the box with::

- vagrant ssh + vagrant ssh probe + +5. Start ooniprobe agent::

-ooniprobe will be installed in ``/ooni``. + ooniprobe-agent start

-5. You can run tests with:: +6. Connect to the web UI on your host machine at http://localhost:8842/

- ooniprobe blocking/http_requests -f /ooni/var/example_inputs/alexa-top-1k.txt

Using ooniprobe =============== @@ -179,134 +179,41 @@ Using ooniprobe Configuring ooniprobe ---------------------

-You may edit the configuration for ooniprobe by editing the configuration file -found inside of ``~/.ooni/ooniprobe.conf``. - -By default ooniprobe will not include personal identifying information in the -test result, nor create a pcap file. This behavior can be personalized. - - -Updating resources ------------------- - -To generate decks you will have to update the input resources of ooniprobe. - -This can be done with:: - - ooniresources - -If you get a permission error, you may have to run the command as root or -change the ooniprobe data directory inside of `ooniprobe.conf`. - -Generating decks ----------------- - -You can generate decks for your country thanks to the oonideckgen command. - -If you wish, for example, to generate a deck to be run in the country of Italy, -you can do so (be sure to have updated the input resources first) by running:: - - oonideckgen --country-code IT --output ~/ - -You will now have in your home a folder called `deck-it`, containing the ooni -deck (ends with .deck) and the inputs. -Note: that you should not move the `deck-*` directory once it has been -generated as the paths to the inputs referenced by the test in the deck are -absolute. If you want your deck to live in another directory you must -regenerated it. - - -Running decks -------------- - -You will find all the installed decks inside of ``/usr/share/ooni/decks``. +After successfully installing ooniprobe you should be able to access the web UI +on your host machine at http://localhost:8842/ .

-You may then run a deck by using the command line option ``-i``: +You should now be presented with the web UI setup wizard where you can read the +risks involved with running ooniprobe. Upon answering the quiz correctly you can +enable or disable ooniprobe tests, set how you can connect to the measurement's +collector and finally configure your privacy settings.

-As root:: - - ooniprobe -i /usr/share/ooni/decks/mlab.deck - - -Or as a user:: - - ooniprobe -i /usr/share/ooni/decks/mlab_no_root.deck - - -Or: - -As root:: - - ooniprobe -i /usr/share/ooni/decks/complete.deck - - -Or as a user:: - - ooniprobe -i /usr/share/ooni/decks/complete_no_root.deck - - -The above tests will require around 20-30 minutes to complete depending on your network speed. - -If you would prefer to run some faster tests you should run: -As root:: - - ooniprobe -i /usr/share/ooni/decks/fast.deck - - -Or as a user:: - - ooniprobe -i /usr/share/ooni/decks/fast_no_root.deck - - -Running net tests ------------------ - -You may list all the installed stable net tests with:: - - - ooniprobe -s - - -You may then run a nettest by specifying its name for example:: - - - ooniprobe manipulation/http_header_field_manipulation - - -It is also possible to specify inputs to tests as URLs:: - - - ooniprobe blocking/http_requests -f httpo://ihiderha53f36lsd.onion/input/37e60e13536f6afe47a830bfb6b371b5cf65da66d7ad65137344679b24fdccd1 - - -You can find the result of the test in your current working directory. - -By default the report result will be collected by the default ooni collector -and the addresses of test helpers will be obtained from the default bouncer. +By default ooniprobe will not include personal identifying information in the +test results, nor create a pcap file. This behavior can be personalized.

-You may also specify your own collector or bouncer with the options ``-c`` and -``-b``. +Run ooniprobe as a service (systemd) +------------------------------------

+Upon ooniprobe version 2.0.0 there is no need for cronjobs as ooniprobe-agent is +responsible for the tasks scheduling.

-Bridges and obfsproxy bridges -============================= +You can ensure that ooniprobe-agent is always running by installing and enabling +the systemd unit `ooniprobe.service`::

-ooniprobe submits reports to oonib report collectors through Tor to a hidden -service endpoint. By default, ooniprobe uses the installed system Tor, but can -also be configured to launch Tor (see the advanced.start_tor option in -ooniprobe.conf), and ooniprobe supports bridges (and obfsproxy bridges, if -obfsproxy is installed). The tor.bridges option in ooniprobe.conf sets the path -to a file that should contain a set of "bridge" lines (of the same format as -used in torrc, and as returned by https://bridges.torproject.org). If obfsproxy -bridges are to be used, the path to the obfsproxy binary must be configured. -See option advanced.obfsproxy_binary, in ooniprobe.conf. + wget https://raw.githubusercontent.com/TheTorProject/ooni-probe/master/scripts/sy... --directory-prefix=/etc/systemd/system + systemctl enable ooniprobe + systemctl start ooniprobe

-(Optional) Install obfsproxy ----------------------------- +You should be able to see a similar output if ooniprobe (systemd) service is +active and loaded by running `systemctl status ooniprobe`::

-Install the latest version of obfsproxy for your platform. + ● ooniprobe.service - ooniprobe.service, network interference detection tool + Loaded: loaded (/etc/systemd/system/ooniprobe.service; enabled) + Active: active (running) since Thu 2016-10-20 09:17:42 UTC; 16s ago + Process: 311 ExecStart=/usr/local/bin/ooniprobe-agent start (code=exited, status=0/SUCCESS) + Main PID: 390 (ooniprobe-agent) + CGroup: /system.slice/ooniprobe.service + └─390 /usr/bin/python /usr/local/bin/ooniprobe-agent start

-Download Obfsproxy: https://www.torproject.org/projects/obfsproxy.html.en

Setting capabilities on your virtualenv python binary ===================================================== @@ -332,10 +239,8 @@ Contributing You can download the code for ooniprobe from the following git repository::

- git clone https://git.torproject.org/ooni-probe.git - + git clone https://github.com/TheTorProject/ooni-probe.git

-It is also viewable on the web via: https://gitweb.torproject.org/ooni-probe.git.

You should then submit patches for review as pull requests to this github repository: