Hello,
These days we're primarily supported by the DRL, which is why we do quarterly reporting. That said, we'd like to share monthly updates from the OONI team with the community, hence our July 2022 report shared below. I'll also follow-up to share our August and September 2022 reports.
*# OONI Monthly Report: July 2022*
Throughout July 2022, the OONI team worked on the following sprints:
* Sprint 69 (1st-17th July 2022) * Sprint 70 (18th-31st July 2022)
Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni
Highlights are shared in this report below.
*## Launched new Test Lists Editor*
In July 2022, OONI launched a new Test Lists Editor: https://test-lists.ooni.org/
The Test Lists Editor enables the public to review and contribute to the Citizen Lab test lists ( https://github.com/citizenlab/test-lists/tree/master/lists), without a GitHub account. Their contributions automatically end up as pull requests on the Citizen Lab test list repository, which means they'd still be peer-reviewed.
We also published a blog post which provides a brief user guide and shares information about the Test Lists Editor: https://ooni.org/post/2022-test-lists-editor/
*## Updated the OONI Data Policy*
In preparation for the launch of the new Test Lists Editor ( https://test-lists.ooni.org/), we updated the OONI Data Policy to include a new section ("OONI web services") that mentions that we do not store email addresses (used to send login links for OONI accounts), and which shares the providers (and their data policies) that we use for deploying web services (https://github.com/ooni/ooni.org/pull/1198).
The updated OONI Data Policy is available here: https://ooni.org/about/data-policy
*## OONI Probe Mobile*
In July 2022, we released OONI Probe Mobile 3.7.0 for: * Android: https://github.com/ooni/probe-android/releases/tag/v3.7.0 * iOS: https://github.com/ooni/probe-ios/releases/tag/v3.7.0
This release includes: * New Vanila Tor experiment; * Automated testing of experimental tests; * In-app language selection; * Support for sharing a link to the OONI Probe app; * Measurement engine synced with OONI Probe CLI 3.15.2; * Bug fixes and improvements.
While working towards the OONI Probe Mobile 3.7.0 release, we also: * Fixed a bug that prevented the VPN warning from appearing when running an OONI Run link (https://github.com/ooni/probe-ios/pull/496); * Worked towards improving the app performance when loading and showing measurements (https://github.com/ooni/probe/issues/2150); * Fixed a bug that affected storage usage ( https://github.com/ooni/probe/issues/2149); * Worked towards improving the pagination of test results ( https://github.com/ooni/probe/issues/1430); * Updated the measurement-log retention policy ( https://github.com/ooni/probe/issues/2162); * Worked on a prototype of the OONI Probe app using flutter, which will allow us to have a shared codebase between the OONI Probe mobile and desktop apps.
*## OONI Run*
In an attempt to meet community needs (particularly those of researchers performing custom testing), we designed and implemented a minimal version of the next generation version of OONI Run (“OONI Run v2”) that enables community members to run OONI Run links without backend support ( https://github.com/ooni/probe-cli/pull/844). We made this functionality available to miniooni users with the goal of enabling community members to start experimenting with this new OONI Run version ( https://github.com/ooni/probe/issues/2184) and share feedback which can support further development and design decisions.
Based on community requests, we also introduced a command line flag which enables users to repeat a measurement every given number of seconds (until OONI Run v2 has support for repeating the measurement with a schedule): https://github.com/ooni/probe-cli/pull/819
As part of our plan for incrementally releasing a new version of OONI Run ( https://run.ooni.io/), we started updating the specification for the new version of OONI Run based on our experimental miniooni-based deployment ( https://github.com/ooni/spec/pull/249).
*## Expanding censorship measurement methodologies*
We worked on data quality improvements to OONI measurements. Specifically, we improved the way in which the Web Connectivity test helper works, by using a single, independent HTTP Client per test helper request, to avoid hitting the limit of maximum connections per HTTP Client ( https://github.com/ooni/probe/issues/2182).
We supported our Google Summer of Code (GSoC) student in developing a TLS middlebox experiment (https://github.com/ooni/probe-cli/pull/817) and experimented around improving how we perform DNS resolution to enable the opportunistic measurement of DoH endpoints ( https://github.com/ooni/probe-cli/pull/846). We also reviewed the work of an OTF Information Controls Research Fellow that we’re currently hosting ( https://github.com/ooni/minivpn/pull/10, https://github.com/ooni/minivpn/pull/12).
*## OONI Explorer*
As part of our work on improving OONI Explorer, we worked on: * Adding support for localization (https://github.com/ooni/explorer/pull/705 ); * Implementing domain-centric pages ( https://github.com/ooni/explorer/pull/762); * Maintenance, package upgrades, and fixing security vulnerabilities; * Improving the SEO of the Measurement Aggregation Toolkit (MAT).
*## OONI backend*
In July 2022, we continued to work on the migration to the Clickhouse database. This involved a series of backend tasks including the testing and switching to more API entry points, numerous configuration changes, and end-to-end testing of the processing of measurements. We fixed various bugs in the fastpath, including the handling of the measurement start time.
We worked on the reprocessing of legacy measurements, which involved the deletion of old legacy measurements from the fastpath pipeline table, fixing a bug and adding metrics for the fetching of measurements across API hosts, and improving upon JSONL tables, among many other backend tasks ( https://github.com/ooni/api/pull/297). We improved the API performance by enabling gzip compression of measurements.
We created internal dashboards to monitor and test the new ASN-based URL prioritization system (https://github.com/ooni/api/pull/293). We also worked on improvements to the new test helper ( https://github.com/ooni/pipeline/pull/392) and performed security updates to the infrastructure.
*## Interviewing Community Coordinator candidates*
In July 2022, we carried out and completed the second (and final) round of interviews for the OONI Community Coordinator position ( https://ooni.org/post/2022-job-opening-ooni-community-coordinator/).
We were thrilled to have received applications from (and to have the opportunity to meet) many strong candidates. Based on both rounds of interviews, we decided to hire Elizaveta Yachmeneva for the OONI Community Coordinator position (who joined the OONI team in August 2022).
*## Test list updates*
In July 2022, we updated the test list for the Philippines to include websites that were reportedly blocked ( https://github.com/citizenlab/test-lists/pull/1022).
*## New blocks emerged in Indonesia*
On 29th July 2022, new blocks emerged in Indonesia. We shared relevant OONI data (and findings) with the #KeepItOn advocacy community.
Specifically, the blocked services included:
* PayPal: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivi... * Yahoo: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivi... * Steam: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivi... * Origin: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivi... * Epic Games: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivi... * Dota2: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivi...
In many of the above cases, we were able to automatically confirm the blocks based on the IP returned from DNS resolution. In other cases, we were able to confirm the blocks because they were implemented through an HTTP middlebox serving a blockpage ( https://explorer.ooni.org/measurement/20220731T055038Z_webconnectivity_ID_45... ).
To encourage further testing of the blocked services, we shared an OONI Run link with the #KeepItOn community.
*## OONI citations### Digital Defenders Partnership publication*
Digital Defenders Partnership published a guide (in Spanish) describing how to document internet blockages and circumvent internet censorship. This publication, which cites OONI and describes OONI tools, is available here: https://www.digitaldefenders.org/wp-content/uploads/2022/09/2207-ISG_final-e...
*### OPTIMA report on the impact of internet shutdowns on women in Uganda*
Sandra Aceng published a research report (as part of Internews’ OPTIMA project) which documents the impact of internet shutdowns on women in Uganda. This study makes use of OONI data collected from Uganda around the country’s 2021 elections.
The report is available here: https://preparepreventresist.org/wp-content/uploads/2022/07/Impact-of-Shutdo...
*## Community activities### OONI training session for journalists in Kenya*
On 28th July 2022, OONI’s Maria facilitated an OONI training session for journalists in Kenya. The workshop involved introducing participants to OONI tools and data for investigating internet censorship in Kenya and around the world.
*### OONI Community Meeting*
On 26th July 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we discussed the following topics:
1) Using OONI Explorer to group measurements by region
2) Request for community feedback: Do users want to have the option to delete specific measurements manually or automatically?
3) Request for community feedback on write/erase cycle limits
*## Userbase*
In July 2022, 47,752,189 OONI Probe measurements were collected from 2,892 ASNs in 167 countries around the world.
This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/
~ OONI team.