Hello Tor world, In December 2019, the OONI team prioritized on the following: * Integrating the new OONI Probe test for Psiphon into the OONI Probe desktop app * Provisioning of Psiphon configuration into apps * API query optimization * Reducing technical debt and bug fixing ## Circumvention tool testing Throughout December 2019, we made important progress on the testing of circumvention tools. Specifically, we: * Added support for giving out Psiphon configuration files to clients in orchestra: https://github.com/ooni/orchestra/issues/78 * Added client support for performing authenticated orchestra requests: https://github.com/ooni/probe-engine/issues/109 * Added support for running circumvention tool tests inside of the OONI Probe CLI: https://github.com/ooni/probe-cli/issues/77 Through the above work, we laid the foundation for implementing future circumvention tool tests inside of OONI Probe. You can now download and run the new OONI Probe Psiphon test via one of the latest ooni/probe-cli release candidate builds: https://github.com/ooni/probe-cli/releases ## Measurement improvements We made a series of improvements to how OONI Probe measurements are performed. Specifically, we: * Released a new version of Measurement Kit which includes DASH using HTTPS: https://github.com/measurement-kit/measurement-kit/releases/tag/v0.10.8 * Validated that the OONI Probe Telegram test works well by using jafar (our censorship simulation tool) to trigger certain network conditions and ensure that the data output is OK: https://github.com/ooni/jafar/pull/15 & https://github.com/ooni/jafar/pull/17 * Added support to probe-engine to use the orchestra to fetch the configuration for Psiphon and to store locally the credentials for logging-in to orchestra: https://github.com/ooni/probe-engine/pull/185 ## OONI Probe 3.0 improvements We added support for the clean shutdown of probe-cli, providing a clean way of interrupting a running test on the OONI Probe desktop app: https://github.com/ooni/probe-cli/issues/76 We also added support for filtering out incomplete test results (https://github.com/ooni/probe-desktop/pull/92), and we upgraded the styled-components to 4.0.0 (https://github.com/ooni/design-system/issues/49). ## OONI Explorer performance As the new OONI Explorer fetches measurements from the OONI API, improving the performance of OONI Explorer (in order to serve query results faster and more reliably) required improving the performance of the OONI API (see: https://github.com/ooni/api/issues/135). We therefore did a lot of work on improving the OONI API. More specifically, we: * Added throttling to the OONI API in order to prevent abuse: https://github.com/ooni/api/pull/139 * Removed SQLAlchemy models (resulting in performance improvements): https://github.com/ooni/api/pull/153 * Refactored the list_measurements table (resulting in a huge performance boost): https://github.com/ooni/api/pull/150 We also did several other important improvements to the OONI API and data processing pipeline: * Added support for detecting issues with the ingestion of data: https://github.com/ooni/pipeline/issues/256 * Added support for anomalous and confirmed blocked measurement columns in the fast-path pipeline: https://github.com/ooni/api/pull/141 * Added support for filtering by domain name for results in the fast-path pipeline: https://github.com/ooni/api/pull/144 * Created two new tables for citizenlab/test-lists ingestion: https://github.com/ooni/pipeline/pull/274 * Added tables that allow us to query by category code: https://github.com/ooni/pipeline/issues/263 * Explored how to update one of our postgresql databases ## Infrastructure Throughout December 2019, we spent a lot of time improving our infrastructure and dealing with incidents. Details around these incidents and what we did to resolve them are available through the following tickets: https://github.com/ooni/sysadmin/issues/410 https://github.com/ooni/sysadmin/issues/406 The infrastucture monitoring and alerting system that we have in place enabled us to more easily identify these issues. ## Blog post to encourage LGBTQI site testing In collaboration with OutRight Action International, we jointly published a blog post to encourage OONI Probe testing of LGBTQI websites around the world. The blog post is published on the following sites:     * OONI: https://ooni.org/post/2019-measure-blocking-lgbtqi-sites/ * OutRight Action International: https://outrightinternational.org/content/let%E2%80%99s-measure-blocking-lgb... VPNCompare published a blog post discussing our LGBTQI testing research: https://www.vpncompare.co.uk/ooni-lgbtqi-censorship/ ## Press coverage An Intercept article was published to cover our joint research report with Coding Rights & Women on Waves (titled "On the blocking of abortion rights websites: Women on Waves & Women on Web"): https://theintercept.com/2019/12/12/net-claro-e-vivo-bloqueiam-site-aborto-s... ## User-base In December 2019, OONI Probe was run 9,270,932 times from 5,564 different vantage points in 209 countries around the world. This information can also be found through our measurement stats on OONI Explorer (chart on monthly coverage worldwide): http://explorer.ooni.org/ ~ The OONI team. -- Maria Xynou Research & Partnerships Director Open Observatory of Network Interference (OONI) https://ooni.org/ PGP Key Fingerprint: 2DC8 AFB6 CA11 B552 1081 FBDE 2131 B3BE 70CA 417E