[tor-commits] [ooni-probe/master] Add some documentation on writing OONI Tests using

commit ffc44ce1cbc595ce8cbcd6522e8962f37b7e6869 Author: Arturo Filastò arturo@filasto.net Date: Fri Sep 28 12:00:07 2012 +0000

Add some documentation on writing OONI Tests using the new framework. --- docs/source/conf.py | 7 ++- docs/source/index.rst | 100 +++++++++++------------------------------ docs/source/install.rst | 55 ++++++++++++++++++++++ docs/source/tutorial.rst | 1 - docs/source/writing_tests.rst | 80 ++++++++++++++++++++++++++------ docs/writing_tests.md | 17 ------- ooni/nettest.py | 43 +++++++++--------- 7 files changed, 174 insertions(+), 129 deletions(-)

diff --git a/docs/source/conf.py b/docs/source/conf.py index 49b96d4..e066613 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -16,7 +16,9 @@ import sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. -#sys.path.insert(0, os.path.abspath('.')) +sys.path.insert(0, + os.path.join(os.path.dirname(__file__), '..', '..')) +

# -- General configuration -----------------------------------------------------

@@ -25,7 +27,8 @@ import sys, os

# Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.pngmath', 'sphinx.ext.viewcode'] +extensions = ['sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.pngmath', +'sphinx.ext.viewcode', 'sphinx.ext.autodoc']

# Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] diff --git a/docs/source/index.rst b/docs/source/index.rst index 6e95ee7..34ca87c 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,96 +1,50 @@ .. OONI documentation master file.

-========== -About OONI -========== +Welcome to the OONI documentation! +==================================

+ The Net interprets censorship as damage and routes around it. + John Gilmore; TIME magazine (6 December 1993)

-Dependencies -************ +OONI, the Open Observatory of Network Interference, is a global observation +network which aims is to collect high quality data using open methodologies, +using Free and Open Source Software (FL/OSS) to share observations and data +about the various types, methods, and amounts of network tampering in the world.

-* Twisted: http://twistedmatrix.com/trac/ -* PyYAML: http://pyyaml.org/ -* Scapy: http://www.secdev.org/projects/scapy/ - * pypcap: http://code.google.com/p/pypcap/ - * libdnet: http://code.google.com/p/libdnet/

-*Optional* +Getting started +***************

-* BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/ +If you choose to use virtualenv to setup your development environment you will +need to do the following::

-Installation -************ + virtualenv ENV + source ENV/bin/activate + pip install twisted Scapy pyyaml

-On debian you can install all the dependecies with apt-get with this command: +To get the latest version of scapy you will need mercurial. You can then install +it with::

- apt-get install python-twisted python-twisted-names python-yaml python-scapy python-beautifulsoup - -*The "hard" way* - -This involves installing the dependencies installable via easy_install/pip and -the ones that are not by building them from source. - -"simple" dependencies via easy_install: - - sudo easy_install pyyaml - sudo easy_install twisted - sudo easy_install beautifulsoup - -"simple" dependencies via pip: - - sudo pip install pyyaml - sudo pip install twisted - sudo pip install beautifulsoup + pip install hg+http://hg.secdev.org/scapy

-libdnet: +On debian you can install all the dependecies with apt-get with this command::

- wget http://libdnet.googlecode.com/files/libdnet-1.12.tgz - tar xzf libdnet-1.12.tgz - cd libdnet-1.12 - ./configure && make - cd python/ - sudo python setup.py install - cd ../../ && rm -rf libdnet-1.12* - -pypcap: - - svn checkout http://pypcap.googlecode.com/svn/trunk/ pypcap-read-only - cd pypcap-read-only/ - sudo pip install pyrex - make - sudo python setup.py install - cd ../ && rm -rf pypcap-read-only - -scapy: - - wget http://www.secdev.org/projects/scapy/files/scapy-latest.zip - unzip scapy-latest.zip - cd scapy-2.2.0/ - sudo python setup.py install - cd ../ && rm -rf scapy-* - -Running -******* + apt-get install python-twisted python-twisted-names python-yaml python-scapy python-beautifulsoup

-To run ooni probe do +Once you have installed all the dependencies OONI tests can be run like so::

-$ export PYTHONPATH=`pwd` + bin/ooniprobe path/to/test.py --cmd1 foo --cmd2 bar

-$ python ooni/ooniprobe.py

+Contents +********

.. toctree:: - :maxdepth: 2 + :maxdepth: 2 + :glob:

- intro + install tutorial writing_tests - ... - -Indices and tables -==================

-* :ref:`genindex` -* :ref:`modindex` -* :ref:`search`

diff --git a/docs/source/install.rst b/docs/source/install.rst new file mode 100644 index 0000000..a2352d0 --- /dev/null +++ b/docs/source/install.rst @@ -0,0 +1,55 @@ + +Installing OONI +=============== + +Currently no installation documentation is present, since OONI is not meant to +be installed and should be handled with care. + +Dependencies +************ + +OONI depends on the following pieces of software. + +* Twisted: http://twistedmatrix.com/trac/ +* PyYAML: http://pyyaml.org/ +* Scapy: http://www.secdev.org/projects/scapy/ + * pypcap: http://code.google.com/p/pypcap/ + * libdnet: http://code.google.com/p/libdnet/ + +*Optional* + +* BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/ + +Manual installation of scapy +---------------------------- + +It is optimal to install scapy, libdnet and pypcap from source. This can be +done with the following code snippets. + +libdnet:: + + wget http://libdnet.googlecode.com/files/libdnet-1.12.tgz + tar xzf libdnet-1.12.tgz + cd libdnet-1.12 + ./configure && make + cd python/ + sudo python setup.py install + cd ../../ && rm -rf libdnet-1.12* + +pypcap:: + + svn checkout http://pypcap.googlecode.com/svn/trunk/ pypcap-read-only + cd pypcap-read-only/ + sudo pip install pyrex + make + sudo python setup.py install + cd ../ && rm -rf pypcap-read-only + +scapy:: + + wget http://www.secdev.org/projects/scapy/files/scapy-latest.zip + unzip scapy-latest.zip + cd scapy-2.2.0/ + sudo python setup.py install + cd ../ && rm -rf scapy-* + diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst index b13b687..15afad7 100644 --- a/docs/source/tutorial.rst +++ b/docs/source/tutorial.rst @@ -1,4 +1,3 @@ -======== Tutorial ========

diff --git a/docs/source/writing_tests.rst b/docs/source/writing_tests.rst index 1aa4b7c..a18b096 100644 --- a/docs/source/writing_tests.rst +++ b/docs/source/writing_tests.rst @@ -1,26 +1,76 @@ -.. OONI documentation master file. - -================== Writing OONI tests ==================

-OONIProbe tests can be written in two modes: blocking or non-blocking.

-Going the blocking route is not advised and all tests in the end should end up -being written in the non-blocking way. +The OONI testing API is heavily influenced and partially based on the python +:class:`unittest` module and :class:`twsted.trial`. + + +Test Cases +---------- + +The atom of OONI Testing is called a Test Case. A test case class may contain +multiple Test Functions. + +.. autoclass:: ooni.nettest.TestCase + +:class:`ooni.nettest.TestCase` is a subclass of :class:`unittest.TestCase` so +the assert methods that apply to :class:`unittest.TestCase` will also apply to +:class:`ooni.nettest.TestCase`. + +If the test you plan to write is not listed on the `Tor OONI trac page +https://trac.torproject.org/projects/tor/wiki/doc/OONI/Tests`_, you should +add it to the list and following the `test template +https://trac.torproject.org/projects/tor/wiki/doc/OONI/Tests/TestTemplate`_ +write up a description about it. + + +Inputs +------ + +Inputs are what is given as input to every iteration of the Test Case. You have +100 inputs, then every test case will be run 100 times. + +To configure a static set of inputs you should define the +:class:`ooni.nettest.TestCase` attribute ``inputs``. The test will be run ``len(inputs)`` times. Any iterable object is a valid ``inputs`` attribute. + +If you would like to have inputs be determined from a user specified input +file, then you must set the ``inputFile`` attribute. This is an array that +specifies what command line option may be used to control this value. + +By default the ``inputProcessor`` is set to read the file line by line and +strip newline characters. To change this behavior you must set the +``inputProcessor`` attribute to a function that takes as arugment a file +descriptor and yield the next item. The default ``inputProcessor`` looks like +this:: + + + def lineByLine(fp): + for x in fp.readlines(): + yield x.strip() + fp.close() + + +Test Functions +-------------- + +These shall be defined inside of your :class:`ooni.nettest.TestCase` subclass. +These will be class methods.

-A good way to understand how to write a test is also to take a look at the OONI -Test Interface in the following files: +To add data to the test report you may write directly to the report object like +so::

-* ooni/plugoo/interface.py + def my_test_function(): + result = do_something() + self.report['something'] = result

-* ooni/plugoo/tests.py +OONI will then handle the writing of the data to the final test report.

-Writing non-blocking tests --------------------------- +To access the current input you can use the ``input`` attribute, for example::

-To bootstrap the process of creating a new tests you can run the scaffolding -script in ooni/scaffolding.py. + def my_test_with_input(): + do_something_with_input(self.input)

-This will create a new plugin with the specified name inside of ooni/plugins/. +This will at each iteration over the list of inputs do something with the +input.

diff --git a/docs/writing_tests.md b/docs/writing_tests.md deleted file mode 100644 index 460620a..0000000 --- a/docs/writing_tests.md +++ /dev/null @@ -1,17 +0,0 @@ -= Writing OONI Tests = - -There are two modes of writing tests. The first way is by making the Test -blocking and have each instance of a test run in a separate thread. The other -is by making them non-blocking using the twisted deferred non blocking pattern. - -The first kind of mechanism relied of writing a test that uses blocking code -and each instance of it is run inside of a separate thread. Beware that your -blocking code must be thread safe to run properly (as is obvious :P) - -The other menthod invloves having some knowledge about twisted. The test you -will write will be written in twisted and should implement async style non -blocking architecture. - -It is recommended that tests are written using the second pattern and the first -should only be used for runnign tests that have been previously written. - diff --git a/ooni/nettest.py b/ooni/nettest.py index 2a02a43..c0c7406 100644 --- a/ooni/nettest.py +++ b/ooni/nettest.py @@ -34,39 +34,40 @@ class TestCase(unittest.TestCase): This is the monad of the OONI nettest universe. When you write a nettest you will subclass this object.

- _inputs_ can be set to a static set of inputs. All the tests (the methods - starting with the "test_" prefix) will be run once per input. At every run - the _input_ attribute of the TestCase instance will be set to the value of - the current iteration over inputs. Any python iterable object can be set - to inputs. + * inputs: can be set to a static set of inputs. All the tests (the methods + starting with the "test_" prefix) will be run once per input. At every run + the _input_ attribute of the TestCase instance will be set to the value of + the current iteration over inputs. Any python iterable object can be set + to inputs.

- _inputFile_ attribute should be set to an array containing the command line - argument that should be used as the input file. Such array looks like this: + * inputFile: attribute should be set to an array containing the command line + argument that should be used as the input file. Such array looks like + this:

- ["commandlinearg", "c", "The description"] + ``["commandlinearg", "c", "The description"]``

- The second value of such arrray is the shorthand for the command line arg. - The user will then be able to specify inputs to the test via: + The second value of such arrray is the shorthand for the command line arg. + The user will then be able to specify inputs to the test via:

- ooniprobe mytest.py --commandlinearg path/to/file.txt + ``ooniprobe mytest.py --commandlinearg path/to/file.txt``

- or + or

- ooniprobe mytest.py -c path/to/file.txt + ``ooniprobe mytest.py -c path/to/file.txt``

- _inputProcessor_ should be set to a function that takes as argument an - open file descriptor and it will yield the input to be passed to the test - instance. + * inputProcessor: should be set to a function that takes as argument an + open file descriptor and it will yield the input to be passed to the test + instance.

- _name_ should be set to the name of the test. + * name: should be set to the name of the test.

- _author_ should contain the name and contact details for the test author. - The format for such string is as follows: + * author: should contain the name and contact details for the test author. + The format for such string is as follows:

- The Name email@example.com + ``The Name email@example.com``

- _version_ is the version string of the test. + * version: is the version string of the test. """ name = "IDidNotChangeTheName" author = "John Doe foo@example.com"