commit 1e44942d3c848c4d6bb29bbd9f385d500555cc0f Author: Arturo Filastò art@fuffa.org Date: Tue Aug 19 11:43:06 2014 +0200
Convert changelog to rst. --- ChangeLog.md | 367 ------------------------------------------------------ ChangeLog.rst | 382 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 382 insertions(+), 367 deletions(-)
diff --git a/ChangeLog.md b/ChangeLog.md deleted file mode 100644 index a349dff..0000000 --- a/ChangeLog.md +++ /dev/null @@ -1,367 +0,0 @@ -## v1.1.0 - -In this new release of ooniprobe we have added a new command line tool for -listing the reports that have not been published to a collector and that allows -the probe operator to choose which ones they would like to upload. - -We have also made some privacy improvements to the reports (we will sanitize -all things that may look like file paths) and added metadata associated with -the maxmind database being used by the probe operator. - -Here is a more detailed list of what has been done: - -* Annotate on disk which reports we have submitted and which ones we have not: - https://trac.torproject.org/projects/tor/ticket/11860 - -* Add tool called oonireport for publishing unpublished ooniprobe reports to a - collector: https://trac.torproject.org/projects/tor/ticket/11862 - -* Probe Report does not leak filepaths anymore: - https://trac.torproject.org/projects/tor/ticket/12706 - -* Reports now include version information about the maxmind database being - used: https://trac.torproject.org/projects/tor/ticket/12771 - -* We now do integrity checks on the ooniprobe.conf file so that we don't start - the tool if the config file is missing some settings or is not consistent: - https://trac.torproject.org/projects/tor/ticket/11983 - (thanks to Alejandro López (kudrom)) - -* Improvements have been made to the sniffer subsystem (thanks to Alejandro - López (kudrom)) - -Minor bug fixes: - -* Fix dns_spoof test (by kudrom) - https://trac.torproject.org/projects/tor/ticket/12486 - -* ooni might not look at requiresTor: - https://trac.torproject.org/projects/tor/ticket/11858 - -* ooni spits out gobs of tracebacks if Tor is not running and the OONI config - says it will be: - https://trac.torproject.org/projects/tor/ticket/11859 - -* The README for ooni-probe should mention the bugtracker and repository - https://trac.torproject.org/projects/tor/ticket/11980 - -## v1.0.2 - -* Add ooniprobe manpage. - -* Fix various security issues raised by the least authority audit. - -* Add a test that checks for Tor bridge reachability. - -* Record the IP address of the exit node being used in torified requests. - -* Captive portal test now uses the ooni-probe test templates. - -* Have better test naming consistency. - -## v1.0.1 - -* Fix bugs in the traceroute test that lead to not all packets being collected. - -* All values inside of http_requests test are now initialized inside of setUp. - -* Fix a bug that lead to the input value of the report not being set in some - circumstances. - -* Add bridge_reachability test - -## v1.0.0 - -* Add bouncer support for discovering test helpers and collectors - -* Fix bug that lead to HTTP tests to stall - -* Add support for connect_error and connection_lost_error error types - -* Add support for additional Tor configuration keys - -* Add disclaimer when running ooniprobe - -## v0.1.0 - -Improvements to HTML/JS based user interface: - - * XSRF protection - - * user supplied input specification - -Bugfixing and improvements to scheduler. - -## v0.0.12 - -Implement JS/HTML based user interface. - -Supports: - - * Starting and stopping of tests - - * Monitoring of test progress - -## v0.0.11 - -* Parametrize task timeout and retry count - -* Set the default collector via the command line option - -* Add option to disable the default collector - -* Add continuous integration with travis - -## v0.0.10 - -### ooniprobe - -* Fix bug that made HTTP based tests stall - -* Update DNS Test example to not import the DNS Test template If you import the - DNS Test template it will be considered a valid test case and command line - argument parsing will not work as expected. see: - #7795 for more details - -* Fix major bug in DNS test template that prevented PTR lookups from working - properly I was calling the queryUDP function with the arguments in the wrong - order. Twisted, why you API no consistent? - -* Add support for specifying the level of parallelism in tests (aka router - melt mode) - -* Do not swallow failures when a test instance fails to run fixes #7714 - -### scripts - -* Add report archival script - -Fix bug in TCP connect test that made it not properly log errors - -* Refactor failure handling code in nettest Add function that traps all the - supported failure and outputs the failure string representing it. - -### documentation - -* Add birdseye view of the ooniprobe architecture - -* Add details on the current implementation status of ooni* - -* Add draft ooniprobe API specification - -* Add instructions for supervisord configuration and clean up README.md - -## 0.0.9 - -### ooniprobe - -* Set the default ASN to 0 - -* Make Beautiful soup a soft depedency - -* Add support for sending the ASN number of the probe: - the ASN number will get sent when creating a new report - -* Add support for obtaining the probes IP address via getinfo address as per - https://trac.torproject.org/projects/tor/ticket/7447 - -* Fix bug in ooniprobe test decks - https://trac.torproject.org/projects/tor/ticket/7664 - -### oonib - -* Use twisted fdesc when writing to files - -* Add support for processing the ASN number of the probe - -* Test reports shall follow the specification detailed inside of docs/reports.rst - -* Add support for setting the tor binary path in oonib/config.py - -### scripts - -* Add a very simple example on how to securely parse the ooniprobe reports - -### documentation - -* Add documentation for the DNSSpoof test - -* Add documentation for HTTPHeaderFieldManipulation - -* Clean up writing_tests.rst - -* Properly use the power of sphinx! - -### Tests - -* fixup Netalyzr third party plugin - -## v0.0.8-alpha - -### ooniprobe - -* Allow test resolver file to have comments. - -* Autostart Tor in default configuration. - -* Add support for starting Tor via txtorcon. - -* Make the sniffer not run in a separate thread, but use a non blocking fdesc. - Do some refactoring of scapy testing, following Factory creational pattern - and a pub-sub pattern for the readers and writers. - -* Extend TrueHeaders to support calculation of difference between two HTTP headers respectful of - capitalization - -* Implement test deck system for automating the specification of command line - arguments for tests - -* Implement sr1 in txscapy - -* Include socksproxy address in HTTP based tests - -* Include the resolver IP:Port in the report - -* Changes to the report format of HTTP Test template derived tests: - Requests are now stored inside of an array to allow - the storing of multiple request/response pairs. - -* Fix bug that lead to httpt based reports to not have the url attribute set - properly. - -* twisted Headers() class edited to avoid header fix in reference to: - https://trac.torproject.org/projects/tor/ticket/7432 - -* Parametrize tor socksport for usage with modified HTTP Agent - -* Update URL List test to take as input also a single URL - -* Clean up filenames of reports generated by ooni-probe: - they now follow the format $testName_report_$timestamp.yamloo - -* Add ooniprobe prefix to logs - -* Respect the includeip = false option in ooniprobe.conf for scapyt derivate - tests: - If the option to not include the IP address of the probe is set, - change the source and destination ip address of the sent and received - packets to 127.0.0.1. - -#### Tests - -* Implement basic keyword filtering detection test. - -* Add ICMP support to multi protocol traceroute test - -* parametrize max_ttl and timeout - -* make max_ttl and timeout be included in the report - -* Port UK Mobile Network test to new API - -* Port daphn3 test - -* Randomize source port by default in traceroute test and include source port in - report - -* Test and Implement HTTP Header Field Manipulation Test (rename it to what we - had originally called it since it made most sense) - -* Implement test that detects DNS spoofing - -* Implement TCP payload sending test template: - Example test based on this test template - -* Make report IDs include the timestamp of the report - -* Add test that detects censorship in HTTP pages based on HTTP body length - -* Add socks proxy support to HTTP Test - -* Create DNS Test template: - Use such template for DNS Tamper test. - Add example usage of DNS Test Template. - -* Refactor captive portal test to run tests in threads - -### oonib - -* Implement basic collector for ooniprobe reports. - Reports can be submitted over the network via http to a remote collector. - Implement the backend component of the collector that writes submitted - reports to flat files, following the report_id naming convention. - -* Implement very simple HTTP Request backend that does only the part of HTTP we - need for testing - -* Make oonib a daemon - -* Loosen up the oonib regexp to support the timestamp report format - -* Add Tor Hidden Service support - -* Make the reporting directory of the collector configurable - -* Implement TCP Echo test helper. - -### scripts - -* Add fabfile for automatic deployment of ooni-probe to remote sites - -### documentation - -* Update documentation on how to setup ooniprobe. - -## v0.0.7.1-alpha - -* Add software version to the report - -* Implement basic oonib reporting to flat files containing the report ID. - -* Improve HTTP Host test to work with the HTTP Requests test backend - -## v0.0.7-alpha - -* Add test_name key to ooniprobe reports - -* Port TCP connect test to the new API - -## v0.0.4-alpha - -* Add multi protocol multi port traceroute for UDP and TCP - -* Implement basic HTTP request test that does capitalization variations on the -HTTP method. - -* Bugfixing and refactoring of txscapy for sending and receiving of scapy -packets. - -## v0.0.3-alpha - -* Implement logging to PCAP file support - -* Remove dependency on trial - -* Port china trigger to new API - -* Rename keyword filtering test to HTTP keyword filtering - -* Refactor install documentation. - -* Convert header of ooniprobe script to a non docstring - -* Add Makefile to fetch Maxmind geoip database files - -* Implement GeoIP lookup support - -* From configuration options it is possible to choice what level of privacy - the prober is willing to accept. Implement config file support You are able - to specify basic and advanced options in YAML format - -* Remove raw inputs and move them to a separate repository and add Makefile to - fetch such lists - -## 0.0.1-alpha - -First release of ooni-probe. woot! - diff --git a/ChangeLog.rst b/ChangeLog.rst new file mode 100644 index 0000000..e41924c --- /dev/null +++ b/ChangeLog.rst @@ -0,0 +1,382 @@ +v1.1.0 +------ + +In this new release of ooniprobe we have added a new command line tool for +listing the reports that have not been published to a collector and that allows +the probe operator to choose which ones they would like to upload. + +We have also made some privacy improvements to the reports (we will sanitize +all things that may look like file paths) and added metadata associated with +the maxmind database being used by the probe operator. + +Here is a more detailed list of what has been done: + +* Annotate on disk which reports we have submitted and which ones we have not: + https://trac.torproject.org/projects/tor/ticket/11860 + +* Add tool called oonireport for publishing unpublished ooniprobe reports to a + collector: https://trac.torproject.org/projects/tor/ticket/11862 + +* Probe Report does not leak filepaths anymore: + https://trac.torproject.org/projects/tor/ticket/12706 + +* Reports now include version information about the maxmind database being + used: https://trac.torproject.org/projects/tor/ticket/12771 + +* We now do integrity checks on the ooniprobe.conf file so that we don't start + the tool if the config file is missing some settings or is not consistent: + https://trac.torproject.org/projects/tor/ticket/11983 + (thanks to Alejandro López (kudrom)) + +* Improvements have been made to the sniffer subsystem (thanks to Alejandro + López (kudrom)) + +Minor bug fixes: + +* Fix dns_spoof test (by kudrom) + https://trac.torproject.org/projects/tor/ticket/12486 + +* ooni might not look at requiresTor: + https://trac.torproject.org/projects/tor/ticket/11858 + +* ooni spits out gobs of tracebacks if Tor is not running and the OONI config + says it will be: + https://trac.torproject.org/projects/tor/ticket/11859 + +* The README for ooni-probe should mention the bugtracker and repository + https://trac.torproject.org/projects/tor/ticket/11980 + +v1.0.2 +------- + +* Add ooniprobe manpage. + +* Fix various security issues raised by the least authority audit. + +* Add a test that checks for Tor bridge reachability. + +* Record the IP address of the exit node being used in torified requests. + +* Captive portal test now uses the ooni-probe test templates. + +* Have better test naming consistency. + +v1.0.1 +------- + +* Fix bugs in the traceroute test that lead to not all packets being collected. + +* All values inside of http_requests test are now initialized inside of setUp. + +* Fix a bug that lead to the input value of the report not being set in some + circumstances. + +* Add bridge_reachability test + +v1.0.0 +------- + +* Add bouncer support for discovering test helpers and collectors + +* Fix bug that lead to HTTP tests to stall + +* Add support for connect_error and connection_lost_error error types + +* Add support for additional Tor configuration keys + +* Add disclaimer when running ooniprobe + +v0.1.0 +------- + +Improvements to HTML/JS based user interface: + + * XSRF protection + + * user supplied input specification + +Bugfixing and improvements to scheduler. + +v0.0.12 +------- + +Implement JS/HTML based user interface. + +Supports: + + * Starting and stopping of tests + + * Monitoring of test progress + +v0.0.11 +------- + +* Parametrize task timeout and retry count + +* Set the default collector via the command line option + +* Add option to disable the default collector + +* Add continuous integration with travis + +v0.0.10 +------- + +ooniprobe: + +* Fix bug that made HTTP based tests stall + +* Update DNS Test example to not import the DNS Test template If you import the + DNS Test template it will be considered a valid test case and command line + argument parsing will not work as expected. see: + #7795 for more details + +* Fix major bug in DNS test template that prevented PTR lookups from working + properly I was calling the queryUDP function with the arguments in the wrong + order. Twisted, why you API no consistent? + +* Add support for specifying the level of parallelism in tests (aka router + melt mode) + +* Do not swallow failures when a test instance fails to run fixes #7714 + +scripts: + +* Add report archival script + +Fix bug in TCP connect test that made it not properly log errors + +* Refactor failure handling code in nettest Add function that traps all the + supported failure and outputs the failure string representing it. + +documentation: + +* Add birdseye view of the ooniprobe architecture + +* Add details on the current implementation status of ooni* + +* Add draft ooniprobe API specification + +* Add instructions for supervisord configuration and clean up README.md + +0.0.9 +----- + +ooniprobe: + +* Set the default ASN to 0 + +* Make Beautiful soup a soft depedency + +* Add support for sending the ASN number of the probe: + the ASN number will get sent when creating a new report + +* Add support for obtaining the probes IP address via getinfo address as per + https://trac.torproject.org/projects/tor/ticket/7447 + +* Fix bug in ooniprobe test decks + https://trac.torproject.org/projects/tor/ticket/7664 + +oonib: + +* Use twisted fdesc when writing to files + +* Add support for processing the ASN number of the probe + +* Test reports shall follow the specification detailed inside of docs/reports.rst + +* Add support for setting the tor binary path in oonib/config.py + +scripts: + +* Add a very simple example on how to securely parse the ooniprobe reports + +documentation: + +* Add documentation for the DNSSpoof test + +* Add documentation for HTTPHeaderFieldManipulation + +* Clean up writing_tests.rst + +* Properly use the power of sphinx! + +Tests: + +* fixup Netalyzr third party plugin + +v0.0.8-alpha +------------ + +ooniprobe: + +* Allow test resolver file to have comments. + +* Autostart Tor in default configuration. + +* Add support for starting Tor via txtorcon. + +* Make the sniffer not run in a separate thread, but use a non blocking fdesc. + Do some refactoring of scapy testing, following Factory creational pattern + and a pub-sub pattern for the readers and writers. + +* Extend TrueHeaders to support calculation of difference between two HTTP headers respectful of + capitalization + +* Implement test deck system for automating the specification of command line + arguments for tests + +* Implement sr1 in txscapy + +* Include socksproxy address in HTTP based tests + +* Include the resolver IP:Port in the report + +* Changes to the report format of HTTP Test template derived tests: + Requests are now stored inside of an array to allow + the storing of multiple request/response pairs. + +* Fix bug that lead to httpt based reports to not have the url attribute set + properly. + +* twisted Headers() class edited to avoid header fix in reference to: + https://trac.torproject.org/projects/tor/ticket/7432 + +* Parametrize tor socksport for usage with modified HTTP Agent + +* Update URL List test to take as input also a single URL + +* Clean up filenames of reports generated by ooni-probe: + they now follow the format $testName_report_$timestamp.yamloo + +* Add ooniprobe prefix to logs + +* Respect the includeip = false option in ooniprobe.conf for scapyt derivate + tests: + If the option to not include the IP address of the probe is set, + change the source and destination ip address of the sent and received + packets to 127.0.0.1. + +tests: + +* Implement basic keyword filtering detection test. + +* Add ICMP support to multi protocol traceroute test + +* parametrize max_ttl and timeout + +* make max_ttl and timeout be included in the report + +* Port UK Mobile Network test to new API + +* Port daphn3 test + +* Randomize source port by default in traceroute test and include source port in + report + +* Test and Implement HTTP Header Field Manipulation Test (rename it to what we + had originally called it since it made most sense) + +* Implement test that detects DNS spoofing + +* Implement TCP payload sending test template: + Example test based on this test template + +* Make report IDs include the timestamp of the report + +* Add test that detects censorship in HTTP pages based on HTTP body length + +* Add socks proxy support to HTTP Test + +* Create DNS Test template: + Use such template for DNS Tamper test. + Add example usage of DNS Test Template. + +* Refactor captive portal test to run tests in threads + +oonib: + +* Implement basic collector for ooniprobe reports. + Reports can be submitted over the network via http to a remote collector. + Implement the backend component of the collector that writes submitted + reports to flat files, following the report_id naming convention. + +* Implement very simple HTTP Request backend that does only the part of HTTP we + need for testing + +* Make oonib a daemon + +* Loosen up the oonib regexp to support the timestamp report format + +* Add Tor Hidden Service support + +* Make the reporting directory of the collector configurable + +* Implement TCP Echo test helper. + +scripts: + +* Add fabfile for automatic deployment of ooni-probe to remote sites + +documentation: + +* Update documentation on how to setup ooniprobe. + +v0.0.7.1-alpha +-------------- + +* Add software version to the report + +* Implement basic oonib reporting to flat files containing the report ID. + +* Improve HTTP Host test to work with the HTTP Requests test backend + +v0.0.7-alpha +------------- + +* Add test_name key to ooniprobe reports + +* Port TCP connect test to the new API + +v0.0.4-alpha +------------ + +* Add multi protocol multi port traceroute for UDP and TCP + +* Implement basic HTTP request test that does capitalization variations on the +HTTP method. + +* Bugfixing and refactoring of txscapy for sending and receiving of scapy +packets. + +v0.0.3-alpha +------------ + +* Implement logging to PCAP file support + +* Remove dependency on trial + +* Port china trigger to new API + +* Rename keyword filtering test to HTTP keyword filtering + +* Refactor install documentation. + +* Convert header of ooniprobe script to a non docstring + +* Add Makefile to fetch Maxmind geoip database files + +* Implement GeoIP lookup support + +* From configuration options it is possible to choice what level of privacy + the prober is willing to accept. Implement config file support You are able + to specify basic and advanced options in YAML format + +* Remove raw inputs and move them to a separate repository and add Makefile to + fetch such lists + +0.0.1-alpha +----------- + +First release of ooni-probe. woot! +