This email shares OONI's monthly report for June 2023.
# OONI Monthly Report: June 2023
Throughout June 2023, the OONI team worked on the following sprints:
* Sprint 93 (5th - 18th June 2023) * Sprint 94 (19th June - 2nd July 2023)
Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni
Highlights are shared in this report below.
## Launch of OONI Explorer supporting localization
We are thrilled to share that OONI Explorer (https://explorer.ooni.org/) now supports localization! OONI Explorer is an open data resource on internet censorship around the world.
We thank the Localization Lab (https://www.localizationlab.org/) community for making OONI Explorer – and real-time open data on internet censorship worldwide – available in so many new languages!
In addition to the aforementioned patches that we also included in v3.17.x, the v3.18.x series releases the following major features (please, refer to the v3.18.0-alpha and v3.18.0 release notes for the complete changelog):
We started working on dslx in December 2022 and we started working on netemx in March 2023. We are excited to see these features finally landing in a stable release!
A DSL-based design is more flexible because we can naturally compose existing building blocks (e.g., DNS lookups, TCP connect, TLS handshake) with other blocks whose task is that of calculating a nettest’s top level keys (e.g., the `blocking` key of Web Connectivity).
Regarding measuring throttling (https://github.com/ooni/ooni.org/issues/1296), we focused on adding (to the DSL introduced in ooni/probe-cli v3.18.0-alpha (https://github.com/ooni/probe-cli/releases/tag/v3.18.0-alpha)) prototype support for collecting extra statistics during long downloads to Web Connectivity LTE (the current experimental version of Web Connectivity) and to the ooni/2023-05-richer-input (https://github.com/ooni/2023-05-richer-input). Currently, the underlying framework that we use for these three pieces of code collects up to 64 network events during and after the TLS or the QUIC handshake. While these events naturally include the initial download of the body, we wanted to extend this model to collect download speed snapshots.
## Creating a Social Media Censorship Alert System
In May 2023, we continued to make progress towards building a Social Media Censorship Alert System. Specifically, we added support for the reprocessing of existing measurements, and we are currently running the event detector on the backend host. As part of improvements to the overall event detector logic, we fixed a bug affecting how columns are handled in Pandas.
## Creating a Censorship Incident Reporting Platform
We’re excited to share that we’re building a Censorship Incident Reporting Platform!
New censorship events frequently emerge worldwide on an ongoing basis, but producing an in-depth report and analysis on each event (in a timely manner) is not easily scalable. Moreover, rapid response efforts benefit from censorship findings that are easier and quicker to consume.
We therefore aim to create a Censorship Incident Reporting Platform which will regularly provide short reports with summary information on emergent censorship events based on OONI data and charts. This will enable journalists and human rights defenders to more easily learn about and respond to censorship events. Over time, the platform will become an archive of known censorship incidents from around the world based on OONI network measurement data.
In June 2023, we made significant progress towards creating a Censorship Incident Reporting Platform. Specifically, we developed the backend API for creating, updating and retrieving censorship incidents through OONI Explorer (https://github.com/ooni/backend/pull/678). This will allow logged in OONI Explorer users to create incidents that can then be retrieved by any user on the OONI Explorer platform, allowing us to develop views for them. The backend API feature was deployed on testing and production infrastructure, allowing us to start working on the frontend for displaying and editing the censorship incidents.
On the frontend part, we started working on the editor (https://github.com/ooni/explorer/pull/862), which supports an enriched markdown format that allows us to embed MAT charts within markdown formatted text. It also allows us to define some metadata which can be used for retrieving incidents matching a specific search criterion (e.g. all censorship incidents pertaining to a particular country).
## OONI backend
In June 2023, we implemented the extraction of test helper metadata (URL and type) in the fastpath (https://github.com/ooni/backend/pull/682), which will enable us to track which test helper was used in Web Connectivity measurements and to create dashboards to detect correlation between test helper locations and test failures. We updated the test helper rotation tool to deploy Vector and configure log forwarding on test helpers (https://github.com/ooni/backend/pull/692), which will allow collecting logs from the test helpers centrally and storing them. We also implemented an initial form of alarming based on the test helper failure rate.
We added support for GeoIP lookup for the probes (https://github.com/ooni/backend/pull/678), and we implemented an experimental tool and dashboard to monitor the accessibility of our website (ooni.org) through Tor. We created an experimental STUN server to perform initial connectivity tests from countries that block many other protocols.
## Automating censorship detection and characterization based on OONI measurements
We also added support for correctly mapping unknown_failures (caused by software bugs) to known failure strings, which allows us to consider more kinds of measurements as part of the analysis (which would have otherwise been considered as “failed”). This work is documented here: https://github.com/ooni/data/pull/29
Finally, we added support for skipping the use of the dask parallel processing engine, when no parallelism is needed, thereby increasing performance (https://github.com/ooni/data/pull/31).
## Interviewed Project Manager candidates
In June 2023, we completed the first round of interviews for the OONI Project Manager position (https://ooni.org/post/2023-job-opening-ooni-project-manager/). Based on these interviews, we created a shortlist for the second (and final) round of interviews and coordinated with candidates. We also started the second round of interviews with shortlisted candidates.
## Blocking of social media in Senegal
Following unrest erupted over the sentencing of opposition leader Ousmane Sonko, OONI data from Senegal shows that ISPs started blocking access to several social media platforms on 1st June 2023.
OONI data collected from Senegal shows that access to social media and messaging platforms is blocked by means of TLS interference (i.e. TLS handshakes timing out).
As of 22nd June 2023, OONI data collected from China shows the blocking of F-Droid by means of DNS injection, where an unrelated IP address (e.g. DropBox) is returned to the client (https://explorer.ooni.org/m/20230622211826.539282_CN_webconnectivity_87bc828c0c6e5758). This technique is commonly used in China to restrict access to services, and OONI data shows that F-Droid is blocked on at least 4 networks in China.
In response to the block, we shared OONI data (and charts) in both English and Chinese on our social media platforms (https://twitter.com/OpenObservatory/status/1673659531639894019). We also reached out to F-Droid developers to share relevant data that could potentially help with circumvention.
## Community use of OONI data
### Civil society response to the social media blocks in Senegal
Between 5th-8th June 20213, OONI’s Elizaveta, Maria, and Arturo traveled to Costa Rica to attend RightsCon 2023 (https://www.rightscon.org/).
As part of their participation, they presented OONI as part of the following 5 sessions:
1) Session: “KeepItOn in 2022: successes, opportunities, struggles, and lessons learned in the fight against election-related shutdowns” (hosted by Access Now and KICTANET), 6th June 2023, OONI speaker: Elizaveta Yachmeneva
2) Session: “Data and Action: Using data analysis for incident response and threat prevention” (hosted by the Center for Digital Resilience), 7th June 2023, OONI speaker: Arturo Filastò
3) Workshop: “Learn to investigate internet shutdowns through an Iran 2022 case study” (hosted by OONI and IODA), 7th June 2023, OONI speaker: Maria Xynou
4) Session: “Internet blocking and the consequences for human rights?” (hosted by Cloudflare), 8th June 2023, OONI speaker: Arturo Filastò
5) Lightning talk: “Investigating Internet shutdowns through open data” (hosted by OONI, IODA, M-Lab), 8th June 2023, OONI speaker: Maria Xynou
We also participated in a private #KeepItOn meeting to discuss how telcos can support the fight against internet shutdowns.
Overall, participating at RightsCon provided a great opportunity to reconnect with many of our partners and community members in person, and to learn from many of the sessions that we attended!
### OONI workshop for journalists and civil society in Sierra Leone
On 21st June 2023, OONI’s Elizaveta facilitated an online OONI workshop for journalists and civil society organizations in Sierra Leone as part of the hybrid Stakeholders Dialogue on Internet Shutdowns event organized by Access Now and our Sierra Leone partner, CHRDI (https://ooni.org/partners/chrdi/), ahead of Sierra Leone’s 2023 elections.
### OONI webinar for civil society in Sierra Leone
On 22nd June 2023, OONI’s Elizaveta presented OONI’s tools and methods to civil society groups in Sierra Leone as part of Paradigm Initiative’s “Defend, Connect, Report: Navigating Internet Censorship and Defending Digital Rights in Sierra Leone’s Election” webinar in preparation for Sierra Leone’s 2023 elections.
Between 28th-29th June 2023, OONI’s Maria traveled to Johannesburg, South Africa, to participate at DSAConnectCon, a 2-day community convening organized by our partner, Digital Society of Africa (DSA). Learn about DSA’s important work here: https://digitalsociety.africa/
As part of her participation, Maria co-facilitated 2 training sessions in collaboration with IODA’s Amanda. These training sessions (“Investigating Internet shutdowns through open data”) introduced participants to both OONI and IODA tools, methods, and datasets, including hands-on exercises based on real-world internet shutdown scenarios.
Maria also assisted participants with localizing OONI Probe into 7 additional African languages (Zulu, Ndebele, Nyanja, Swati, Mozambican Portuguese, Tumbuka, Shona) during the Localization Lab’s (https://www.localizationlab.org/) hands-on workshop on localizing internet freedom tools.
### OONI Community Meeting
On 27th June 2023, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we briefly discussed:
1) Social media blocks in Senegal. 2) F-Droid block in China. 3) Monitoring internet censorship in Sierra Leone during the country’s 2023 elections. 4) OONI Explorer being released in 8 new languages (encouraging further localization efforts).
## Measurement coverage
In June 2023, 56,948,483 OONI Probe measurements were collected from 2,897 networks in 165 countries around the world.
This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/