[tor-reports] Arturo's November status report

[Arturo Filastò]

## November

November was dedicated to making sure that the design layer out and experimented with in the past months worked properly. Focus was placed on the ooniprobe command line tool. The ooniprobe backend was made to do the basic set of tasks it needed to support the running of tests.

Highlights:

* Alpha support for logging to PCAP

* Alpha support for starting and stopping of Tor

* Alpha support for reporting to Tor Hidden Services

### Arturo

* Revert to a state in which everything was not broken and attempt to port Bridget
https://github.com/hellais/ooni-probe/commit/de0c47f7152b3a0a9dd2baf27ce2b028819e3f63

* Make the DNS Tamper test use the a lower, more flexible twisted API call:
https://github.com/hellais/ooni-probe/commit/769cd3cbbd8521af505f2b740af8468b1a347f2f

* Work on the backend system database support:
https://github.com/hellais/ooni-probe/commit/bf37eed44600ea9d307f958e35304ff4a7116032

* Drop support for the old API:
https://github.com/hellais/ooni-probe/commit/654147c3a92156c35fc1a8ecc0aac8e933f57e59

* Add GeoIP Lookup support for determining the probes ASN number and country:
https://github.com/hellais/ooni-probe/commit/e056fe9b886c3dfaf9546cd94656eec8117f70f8
https://github.com/hellais/ooni-probe/commit/e10033c72b4cf3355095b3a0bbdfca3a5dbeac4a

* Improve the scapy test template:
https://github.com/hellais/ooni-probe/commit/f4a065da96af966e01126be4c3013780da1b7579

* Iterations over existing tests:
https://github.com/hellais/ooni-probe/commit/9bf7fc8d86901bca4061dd82f6ca6af311cabee4

* Add basic basic capture support by spawning a thread:
https://github.com/hellais/ooni-probe/commit/9e632c5d7c8d7f73add3aa38900898a86f6ca9fc
https://github.com/hellais/ooni-probe/commit/6a3ee55b574adaa8740ccafe2e4a01719dc0e86e

* Improve txscapy so that it uses a valid file descriptor instead of spawning a thread:
https://github.com/hellais/ooni-probe/commit/12726ca463d9e68e93d49fcb418421648d054744

* Config file support for oonib:
https://github.com/hellais/ooni-probe/commit/d4adf3e0e177e80ce7bc825f1cb4e461e5551b2f

* Engage the community in contributing code:
https://github.com/hellais/ooni-probe/commit/28241124e6094b224f4a3b4f6c0a8e8a69a7eeb6
https://github.com/hellais/ooni-probe/commit/c67d69ec432e32479daca2ed65b27529260c9ba5

* Recuperate the UK mobile network test that was implemented some time ago (https://trac.torproject.org/projects/tor/ticket/6437) for the Open Rights Group:
https://github.com/hellais/ooni-probe/commit/e115d4d0e6c063422fc35cbe1a63dcf0f5917a77

* First iteration over Tor Hidden Service support for oonib and ooniprobe
https://github.com/hellais/ooni-probe/commit/d0d9662b4cafe11979e8173e6cc03c548f99ecf0


* First iteration over TCP Test template. This allows to specify a TCP payload from a flat file and it will get sent to the destination address and the response will be recorded.
https://github.com/hellais/ooni-probe/commit/f873cd780a37451810b04526065fbe90d6250ba7

* Work with somebody in Turkey to get them to run some tests.

* Implement DNS Spoofing test:
https://github.com/hellais/ooni-probe/commit/4208c6e963ae59380999fae45930656c287a627d

* First iteration over resume support and test state tracking:
https://github.com/hellais/ooni-probe/commit/659dda6b388ecc658f8060b7bef2aa7b4015dce2

* Add support for starting Tor via txtorcon:
https://github.com/hellais/ooni-probe/commit/a65b6ebb7e90a4350b1a214189196d11a8371eda

* Make the scapy sniffer not run a separate thread, but be non blocking. Get inspiration from the work done by Paul Bohm for muxTCP (http://events.ccc.de/congress/2005/fahrplan/events/529.en.html)
https://github.com/hellais/ooni-probe/commit/615ce75c47aec249b6b5a4c0b58fdf7a93f09582

* Get aagbsn up to speed on ooniprobe.

* Implement HTTP Request Line "fuzzer": 
https://github.com/hellais/ooni-probe/commit/1de07f659f1393d969a1b3766baffeecb111355d

and document it:

https://github.com/hellais/ooni-probe/commit/d67eecfd05f03002e3a9334db82ab2d8c2658368

* Add support for ooniprobe test decks:
https://github.com/hellais/ooni-probe/commit/263568f

* Work on documentation for various tests.

* Lot's of refactoring:
https://github.com/hellais/ooni-probe/commit/10c63e0fceb0479c5893bfd2cb7e77af2af703cc
https://github.com/hellais/ooni-probe/commit/7d6901f1552067bce9595db6a84f8f5245d8f28c

- Art.