Hello,
OONI's February 2022 status report is shared below.
*# OONI Monthly Report: February 2022*
Throughout February 2022, the OONI team worked on the following sprints:
* Sprint 58 (1st - 13th February 2022) * Sprint 59 (14th - 27th February 2022)
Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni
Highlights are shared in this report below.
*## OONI submission for the OHCHR report on internet shutdowns*
On 10th February 2022, we sent a submission in response to the UN High Commissioner for Human Rights’s call for submissions in support of OHCHR report on internet shutdowns and human rights to the fiftieth session of the Human Rights Council in June 2022 ( https://www.ohchr.org/en/Issues/CivicSpace/Pages/cfi-report-hrc-50.aspx).
In particular, the OONI submission provides information on the occurrence of mandated disruptions of access to social media and messaging platforms over the past 5 years based on empirical OONI network measurement data.
Through our submission, we share information on social media blocks that occurred during elections in Uganda, Tanzania, Mali, Benin, Togo, Burundi, and Zambia over the last 5 years (based on OONI data). We also share information on social media blocks that occurred during protests in Pakistan, Jordan, Iran, Zimbabwe, and Cuba. Moreover, we share OONI data and information on social media blocks that emerged during sensitive political time periods in Ethiopia, Myanmar, and Venezuela. In our submission, we also recommend that the OHCHR considers expanding this scope to also report on cases that involve the blocking of websites of marginalized communities.
*## OONI Probe Mobile*
Throughout February 2022, we worked towards the OONI Probe Android 3.6.0 release.
In particular, we worked on the following OONI Probe Android improvements: * Worked on ensuring that the VPN modal is displayed prior to the running of tests (https://github.com/ooni/probe/issues/1982); * Improved the UI for minimizing a running test ( https://github.com/ooni/probe/issues/1998); * Fixed a bug affecting the display of the indication that a proxy is being used (https://github.com/ooni/probe/issues/1919); * Fixed a bug affecting Web Connectivity tests ( https://github.com/ooni/probe/issues/1328); * Improved the performance of the test startup time ( https://github.com/ooni/probe/issues/2012); * Improved the progress bar of tests ( https://github.com/ooni/probe/issues/2021, https://github.com/ooni/probe/issues/1936); * Fixed a bug affecting the OONI backend ( https://github.com/ooni/probe/issues/1922); * Disabled the RiseupVPN test from OONI Probe ( https://github.com/ooni/probe/issues/2001).
We also worked on the following OONI Probe iOS improvements: * Added links to the OONI blog and research reports in OONI Probe iOS ( https://github.com/ooni/probe/issues/1850); * Fixed a bug that caused the OONI Probe iOS app to crash with a malformed URL (https://github.com/ooni/probe/issues/1978).
In preparation for the upcoming release of our new Tor Snowflake experiment, we wrote a test description, which we published on our website: https://ooni.org/nettest/tor-snowflake/
*## OONI Run*
We continued to make incremental improvements to OONI Run based on community feedback. In particular, we fixed a bug affecting the copy-pasting of URLs into the OONI Run platform ( https://github.com/ooni/run/issues/82).
*## OONI Probe Desktop*
Throughout February 2022, we worked towards the OONI Probe Desktop 3.7.0 release. In late February 2022, we released the release candidate for OONI Probe Desktop 3.7.0: https://github.com/ooni/probe-desktop/releases/tag/v3.7.0-rc.1
In particular, we worked on the following OONI Probe Desktop improvements: * Fixed a bug affecting website testing ( https://github.com/ooni/probe/issues/1848); * Fixed a bug affecting the “test duration limit” (following recent OONI Probe CLI changes); * Fixed a bug affecting the RiseupVPN test result screen ( https://github.com/ooni/probe-desktop/pull/272); * Ensured that OONI Probe Desktop defaults to English on unsupported languages (https://github.com/ooni/probe/issues/1914); * Disabled the RiseupVPN test from OONI Probe Desktop ( https://github.com/ooni/probe/issues/2002).
*## OONI Probe Command Line Interface (CLI)*
We released OONI Probe CLI 3.14.1 along with new mobile libraries for Android and iOS. You can read the changelog here: https://github.com/ooni/probe-cli/releases/tag/v3.14.1
Some highlights:
* We added the DNS Check test to the experimental test suite;
* We added the Tor Snowflake test to the experimental test suite;
* We made `ooniprobe list` approximately 7 times faster than before;
* We fixed our mobile libraries to ensure that all measurements are submitted. Previously, there were cases when the last measurement in a Web Connectivity run would not be submitted when the experiment timeout occurred right before an attempt to submit this measurement (see: https://github.com/ooni/probe/issues/2037);
* We fixed a bug which prevented the NDT test from working on Android 6 phones due to the bundling of a very old certification authority ( https://github.com/ooni/probe/issues/2031);
* We added a workaround to handle an inconsistency in how Android 10 NXDOMAIN errors are reported which would otherwise cause measurements to be marked as failed. This was done by tweaking our error classifier for this error condition (https://github.com/ooni/probe/issues/2029) and we started working on a more comprehensive fix for v3.15.
*## Expanding censorship measurement methodologies*
In preparation for the OONI Probe CLI 3.14 release, we performed extensive QA of the new Tor Snowflake experiment ( https://github.com/ooni/probe/issues/1917 and https://github.com/ooni/probe/issues/2004) where we discussed the proper experiment configuration with Tor Snowflake developers. We eventually settled for a configuration where we allow tor to cache on disk its descriptors to increase the chances that tor could successfully bootstrap using Snowflake. We determined that bootstrapping issues are probably caused by the Snowflake bridge being at capacity, which is a problem being addressed upstream.
We started researching how we could use data we already collect to detect cases of heavy throttling. We developed a prototype experiment, called “download”, to explicitly investigate these conditions ( https://github.com/ooni/probe/issues/2025). We applied some of the preliminary knowledge we gathered from this new experiment to detect cases of Twitter throttling using data already collected by Web Connectivity in our report on censorship in Russia amid the conflict in Ukraine ( https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/#twitter-thrott... ).
We discovered and documented issues with the RiseupVPN experiment ( https://github.com/ooni/probe/issues/1999) and decided to temporarily disable the experiment from the OONI Probe apps ( https://github.com/ooni/probe/issues/2000) until those fundamental issues have been resolved.
We also discovered an issue with the tor experiment, where security changes in the upstream obfs4 library caused the NTOR handshake to fail with some of the bridges we are currently testing ( https://github.com/ooni/probe/issues/1970 and https://github.com/ooni/probe/issues/1986). We discussed this issue with the obfs4 maintainer, which helped us understand the extent of the change. We concluded that for the OONI Probe CLI 3.14 release, we would not use the most updated version of the obfs4 library, in order to provide bridge operators some extra time to update obfs4. We aim to upgrade obfs4 in the next OONI Probe release.
We helped our OTF Information Controls Research Fellow Kathrin Elmenhorst to implement a QUIC Ping experiment aimed at better characterizing cases of QUIC blocking (https://github.com/ooni/probe/issues/1994). The core idea of the experiment is to initiate a QUIC session using an invalid version number, to elicit a version negotiation message from a server. The message sent to initiate the session does not contain valid TLS data, therefore this experiment helps us to distinguish cases where QUIC is blocked from cases where the blocking depends on the TLS data contained in the initial QUIC datagram.
*## OONI Explorer localization*
Through collaboration with the Localization Lab, OONI Explorer copy was added to Transifex to enable translation of the platform by the global Localization Lab community.
OONI Explorer can now be translated through the following Transifex project: https://www.transifex.com/otf/ooni-explorer/
Thanks to the Localization Lab community, OONI Explorer is being translated to 20 languages, and is already fully translated to Persian, Turkish, and French.
*## Circumvention Tool Reachability Dashboard*
Leading up to the launch (early March 2022), we worked on final improvements to our new Circumvention Tool Reachability Dashboard ( https://explorer.ooni.org/chart/circumvention). This included a series of UI improvements based on review and testing of the platform ( https://github.com/ooni/explorer/pull/653#pullrequestreview-877501889).
*## OONI Measurement Aggregation Toolkit (MAT)*
We continued to make improvements to our new Measurement Aggregation Toolkit (MAT) in preparation for the public launch (early April 2022). These improvements included improvements to the readability of x-axis charts, and layout changes in charts for longer date ranges (expanding beyond 2 months). We also made backend improvements to the aggregation API to improve how anomalous, confirmed, and failed measurements are grouped and counted in the API (https://github.com/ooni/backend/issues/486).
*## OONI backend*
We continued to work on migrating the OONI API to the new Clickhouse-based backend in production (https://github.com/ooni/pipeline/pull/371). This involved testing the new Clickhouse-based API ( https://github.com/ooni/backend/issues/552), re-importing the fastpath table with a domain column, comparing statistics generated by the private API, extensive monitoring, testing API endpoints and migrating them, and fixing the counts inside the aggregation API ( https://github.com/ooni/backend/issues/486).
We also continued working on a number of other backend activities, such as improving the Prometheus and Grafana dashboards and alerting to monitor the Clickhouse database and its server. We cleaned up various spurious alerts and alerts that emerged due to limited memory on the ams-pg-test host. We ran an experiment predicting incoming traffic using sklearn's (machine learning library) linear regression. The experiment was successful.
We set up log forwarding and ingestion using the monitoring server and created an example dashboard on Jupyter. We implemented a tool to run Jupyter notebooks automatically, which allows us to produce charts on an hourly, daily or weekly basis (https://github.com/ooni/backend/issues/564). We created a dashboard to monitor API check-ins. We also worked on adding support to the check-in API for returning geolocation information ( https://github.com/ooni/backend/issues/555).
We started a thorough review of measurements that were not processed successfully in the past due to unsupported formats or parsing errors ( https://github.com/ooni/backend/issues/538). As part of this, we also created Jupyter dashboards and started a design document to plan the reprocessing of measurements.
We also investigated a bug impacting RiseupVPN measurements ( https://github.com/ooni/pipeline/pull/379). In preparation for a security audit, we also continued to work on extensive threat modeling ( https://github.com/ooni/backend/issues/556).
*## Improving OONI data analysis capabilities*
We worked on data analysis and research related to extracting per website metrics (https://github.com/ooni/backend/issues/330). As part of this, we wrote an internal design document that outlines the features of various OONI measurements that are useful for performing data analysis tasks ( https://github.com/ooni/ooni.org/issues/1112). This document will support our efforts in extracting data from OONI measurements that is useful for characterizing different forms of internet censorship.
We also worked towards consolidating our block page fingerprint collection efforts with the Citizen Lab filtering annotations ( https://github.com/ooni/backend/issues/516). This will enhance our ability to automatically detect and confirm more cases of website blocking around the world.
*## Improving third party usage of OONI data*
To enable third parties (such as researchers) to make use of OONI data, we worked on implementing a tool for third party download of OONI data. To avoid drifting out of sync, we determined in our design goals that this tool should use the same codebase as the OONI data processing pipeline.
Initial work on this new tool was published as a branch of the OONI pipeline repository: https://github.com/ooni/pipeline/pull/377
We also created an internal design document, which outlines some of the current challenges in third party OONI data consumption and shared it for feedback with some key users of OONI data.
*## Creating an online OONI Training Course for Advocacy Assembly*
Small Media has provided us the opportunity to create an online OONI Training Course for their Advocacy Assembly project ( https://www.advocacyassembly.org/), which features a variety of training courses for human rights defenders.
In February 2022, we started working on creating the online OONI training course. We started off by reviewing several past Advocacy Assembly training course scripts and videos in order to better understand how these courses are structured. Based on this, we finalized the structure for the OONI training course.
Structurally, the online OONI training course will entail the following 7 chapters: * Introduction to the course * Introduction to internet censorship * Measuring internet censorship * Measuring internet censorship with OONI Probe * Understanding OONI censorship measurement data * OONI Explorer: Accessing real-time censorship measurement data * Conclusion
Each chapter will dive into each topic in depth, providing participants with videos, screencasts, slides, quizzes, and hands-on exercises.
Throughout February 2022, we created several hands-on exercises for the course, and we wrote the scripts for 8 direct-to-camera videos for the following (sub-)chapters: * Introduction to the course * What is internet censorship? * The problem of internet censorship * What is OONI Probe? * Interpreting OONI data * What is OONI Explorer? * Looking at OONI data in aggregate * Conclusion
In April 2022, we will travel to London to record the above videos (based on the scripts we wrote) at a studio with Small Media.
*## ISOC Pulse project on Internet shutdowns*
OONI is a data partner for the Internet Society (ISOC) Pulse project on Internet shutdowns (https://pulse.internetsociety.org/partners). We aim to contribute OONI data (along with relevant charts and information) for all the “content blocking” events listed on the ISOC Pulse shutdowns timeline ( https://pulse.internetsociety.org/shutdowns).
To this end, we started off by creating an inventory of all ISOC Pulse “content blocking” entries listed per year in their timeline (which span from 2018 to date). We then created content for the “content blocking” entries listed for 2022, including relevant OONI data, charts, and interpretation, which we shared with ISOC (to include in their timeline). We also started creating content (short reports) for the 2021 entries listed in the ISOC Pulse shutdown timeline.
*## OTF Information Controls Research Fellows*
In February 2022, OONI started serving as the host organization for OTF Information Controls Research Fellow, Ain Ghazal. Throughout their 1-year fellowship, Ain will study censorship resistance systems in global VPN infrastructure. Further information about their research fellowship is available here: https://www.opentech.fund/about/people/ain-ghazal/
As of February 2022, OONI is excited to serve as the host organization for a total of 3 OTF Information Controls Research Fellows. We are also hosting: * Kathrin Elmenhorst (investigating HTTP/3 censorship): https://www.opentech.fund/about/people/kathrin-elmenhorst/ * Gurshabad Grover (investigating the role of private ISPs in exacerbating or minimizing the effects of state-ordered censorship): https://www.opentech.fund/about/people/gurshabad-grover/
*## Google Summer of Code (GSoC) 2022 OONI project ideas*
Over the past years, we have submitted Google Summer of Code (GSoC) project ideas for OONI via the Tor Project ( https://summerofcode.withgoogle.com/programs/2022/organizations/the-tor-proj... ).
For the summer of 2022, we submitted the following OONI project ideas for GSoC students:
OONI Probe Network Experiments OONI Probe CLI Improvements OONI Explorer Improvements
Further information is available here: https://gitlab.torproject.org/tpo/team/-/wikis/GSoC
*## Collaboration with Netalitica*
Netalitica researchers continued to do excellent work in reviewing and updating the Citizen Lab test lists. In February 2022, we reviewed their updates to the test lists for Hungary ( https://github.com/citizenlab/test-lists/pull/892) and Poland.
*## Test list updates*
In response to (potential) censorship events, we contributed updates to the following Citizen Lab test lists: * Global: https://github.com/citizenlab/test-lists/pull/889, https://github.com/citizenlab/test-lists/pull/890, https://github.com/citizenlab/test-lists/pull/915 * South Korea: https://github.com/citizenlab/test-lists/pull/891 * Turkey: https://github.com/citizenlab/test-lists/pull/911
We also reviewed and merged several test list pull requests contributed by community members.
*## Emerging censorship events### New censorship events in Russia following the invasion of Ukraine*
Following the invasion of Ukraine, Russia ramped up its censorship. Real-time OONI data collected from Russia shed light on these new censorship events as they emerged.
Starting from 26th February 2022, Russia started to throttle access to Twitter and to block access to several independent news media websites. We started off by sharing OONI findings on social media ( https://twitter.com/OpenObservatory/status/1497951299593281538) and with mailing lists, while performing more extensive data analysis as new blocks emerged. Based on our analysis, we wrote a research report, which we published in early March 2022.
*### Twitch blocked in Iran*
Twitch was temporarily blocked in Iran on 27th and 28th February 2022. This was detected and reported by community members in Iran who shared relevant OONI data (https://twitter.com/xhdix/status/1497961275225214980/).
OONI data involving the block can be accessed and viewed through the following MAT chart: https://explorer.ooni.org/experimental/mat?probe_cc=IR&test_name=web_con...
*## Notable community use of OONI Probe and OONI data### VEsinFiltro report on the blocking of news media websites in Venezuela*
On 1st February 2022, our Venezuelan partner, VEsinFiltro, published a report documenting the blocking of news media websites in Venezuela: https://vesinfiltro.com/noticias/2022-02-01-bloqueo-Noticias/
Their report makes use of OONI data, as well as network measurement data collected from their own custom tools.
*### VEsinFiltro report on the blocking of circumvention tools in Venezuela*
On 7th February 2022, VEsinFiltro published a report documenting the blocking of circumvention tools in Venezuela: https://vesinfiltro.com/noticias/2022-02-07-restriction-circumvention-tools/
Their report makes use of OONI data, involving the analysis of measurements collected from the OONI Probe Tor and Web Connectivity experiments.
*### ECOWAS court ruling on internet shutdown in Togo*
Togo’s internet shutdown amid protests in September 2017 was challenged in the “Amnesty International Togo and Ors v. The Togolese Republic” case ( https://globalfreedomofexpression.columbia.edu/cases/amnesty-international-t...) brought to the Community Court of Justice of the Economic Community of West African States (ECOWAS). In preparation for this case, the OONI team was asked to share relevant technical expertise, along with other experts. We analyzed the internet shutdown and shared a technical analysis in support of the court case ( https://twitter.com/FrancoisPatuel/status/1494030238388563968).
As an outcome, the Community Court of Justice of ECOWAS held that the Togolese government violated the applicants’ right to freedom of expression by shutting down the internet during protests in September 2017, and ordered the Respondent State of Togo to take measures to guarantee the “non-occurrence” of a future similar situation.
The ECOWAS court ruling on the September 2017 internet shutdown in Togo won the Columbia University Global Freedom of Expression 2022 Significant Legal Ruling Award ( https://globalfreedomofexpression.columbia.edu/prizewinners2022/).
*## Community activities### OONI workshop in Tanzania by Zaina Foundation*
On 28th February 2022, Zaina Foundation (our partner in Tanzania) facilitated an OONI workshop as part of their broader Digital Security Training for women journalists in Dar es Salaam ( https://twitter.com/ZainaFoundation/status/1498330087258742789).
*### OONI FAQ translated to Khmer by DigitalHub101*
Cambodian civil society group, DigitalHub101, published a version of the OONI FAQ (https://ooni.org/support/faq/) in Khmer: https://digitalhub101.com/ooni/
This new page features information in Khmer about OONI, OONI Probe, OONI Explorer, and OONI research reports. We linked to their translated version in the main OONI FAQ page to enable more community members to benefit from the translation.
*### OONI Community Meeting*
On 22nd February 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we discussed the following topics:
1) “Internet Protection Bill” in Iran 2) OONI measurement coverage from Zimbabwe
*## Userbase*
In February 2022, 36,927,939 OONI Probe measurements were collected from 2,556 AS networks in 158 countries around the world.
This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/
~ OONI team.