Hello,
These days we're primarily supported by the DRL, which is why we do quarterly reporting. That said, we'd like to share monthly updates from the OONI team with the community, hence our August 2021 report shared below. I'll follow-up to share our September 2021 report.
# OONI Monthly Report: August 2021
Throughout August 2021, the OONI team worked on the following sprints:
* Sprint 45 - Antarctic Krill (2nd - 15th August 2021)
* Sprint 46 - Happy Oyster (16th - 29th August 2021)
Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni
Highlights are shared in this report below.
## Published research report on the blocking of LGBTIQ websites in six countries
In collaboration with the Citizen Lab and OutRight Action International, we co-published a new research report, "No Access: LGBTIQ Website Censorship in Six Countries", which examines the blocking of LGBTIQ websites in Indonesia, Malaysia, Iran, Russia, Saudi Arabia, and the United Arab Emirates (UAE).
The report is available on each of our websites:
* OONI: https://ooni.org/post/2021-no-access-lgbtiq-website-censorship-six-countries/
* Citizen Lab: https://citizenlab.ca/2021/08/no-access-lgbtiq-website-censorship-in-six-countries/
* OutRight Action International: https://outrightinternational.org/content/no-access-lgbtiq-website-censorship-six-countries
Download the full (203-page) report here: https://ooni.org/documents/2021-lgbtiq-website-censorship-report/2021-lgbtiq-website-censorship-report.pdf
We adopted a mixed methods research approach, combining OONI network measurement analysis with interviews and literature research. The timeframe that we selected for OONI data analysis was June 1, 2016 to July 31, 2020. To examine the impact of online LGBTIQ censorship, OutRight Action International and the Citizen Lab interviewed LGBTIQ communities in the six countries.
Our research report received the following press coverage:
https://www.openlynews.com/i/?id=3ff85ecb-ac2a-4a84-a451-876598f90a4f
https://www.reuters.com/article/idUSL8N2Q23D4
https://www.freewebturkey.com/dunyada-sansur-gundemi-28-agustos-3-eylul/
https://www.opendemocracy.net/en/5050/tech-firms-facilitating-censorship-of-lgbtiq-people-must-be-held-to-account/
https://www.gaytimes.co.uk/life/invisible-and-silenced-lgbtq-online-content-is-being-censored-across-the-globe/
https://76crimes.com/2021/09/15/how-online-censors-muzzle-queer-voices-in-russia-middle-east-indonesia-malaysia/
## Published research report on social media blocks amid Zambia’s 2021 election
We published a new research report titled: "Zambia: Social media blocked amid 2021 general elections".
The report is available here: https://ooni.org/post/2021-zambia-social-media-blocks-amid-elections/
In the report, we share OONI data (and technical observations) on the blocking of WhatsApp, Twitter, and Facebook amid Zambia's 2021 general election.
## Academic paper on internet censorship in Myanmar published by FOCI
In collaboration with the Center for Applied Internet Data Analysis (CAIDA) of UC San Diego, Kentik, and other researchers from UC San Diego and the University of Michigan, we co-authored an academic research paper titled: “A multi-perspective view of Internet censorship in Myanmar”.
We submitted this paper to the ACM SIGCOMM 2021 Workshop on Free and Open Communications on the Internet (FOCI 2021), who published our paper on 27th August 2021: https://dl.acm.org/doi/10.1145/3473604.3474562
Our paper can be read through the following link: https://www.ramapad.com/papers/myanmar_foci21_cr.pdf
Ramakrishna Padmanabhan (CAIDA, UC San Diego) presented the paper at the FOCI 2021 workshop (https://conferences.sigcomm.org/sigcomm/2021/workshop-foci.html) and published a video presentation which summarizes our findings: https://www.youtube.com/watch?v=ABcswfBO2RY
As part of our work on this paper, we analyzed OONI measurements collected from Myanmar between 1st February 2021 to 30th April 2021, building upon our previous research report on Myanmar (https://ooni.org/post/2021-myanmar-internet-blocks-and-outages/). Notably, we found two cases of collateral damage caused by IP blocks in Myanmar. Based on our analysis, we contributed to the writing of the paper, and helped produce relevant charts.
## OONI blog post for TunnelBear Tales blog series
We had the opportunity to write about OONI as part of the TunnelBear Tale blog series.
On 3rd August 2021, TunnelBear published our blog post (“TunnelBear tales: OONI”), which is the form of an interview. In this blog post, we discuss how OONI came to be, why people run OONI Probe, the role of network measurement in the fight for internet freedom, community stories, and what is needed to further support the censorship measurement community.
Our blog post for TunnelBear is available here: https://www.tunnelbear.com/blog/tunnelbear-tales-ooni/
## Published blog post on mining OONI data
To enable community members to access and analyze OONI data, we published a blog post which explains the various ways (S3 bucket, PostgreSQL database, OONI API, OONI Explorer) through which you can access raw OONI measurement data.
This blog post (“Mining OONI data”) is available here: https://ooni.org/post/mining-ooni-data/
## Published blog post on a brief introduction to OONI
To enable new community members to learn about OONI, we published a blog post which provides a gentle introduction to OONI, some basics about how the internet works, and how OONI Probe detects blocking.
This blog post (“A brief introduction to OONI”) is available here: https://ooni.org/post/brief-introduction-to-ooni/
## Expanding OONI censorship measurement methodologies
In August 2021, we started working on rewriting our flagship experiment (Web Connectivity) for measuring the blocking of websites. We first released the Web Connectivity test 5 years ago (https://ooni.org/post/web-connectivity/), and it has served us quite well in measuring the blocking of websites worldwide (https://ooni.org/reports/). But internet censorship has evolved over the last years (for example, SNI based filtering is increasingly being observed), requiring us to rethink (and rewrite) how we measure the accessibility of websites.
Some of the main improvements (https://github.com/ooni/probe/issues/1714) we would like to make to the Web Connectivity experiment include:
* QUIC support
* TLS parroting
* Using both the DNS of the local resolver and that of the test helper
* Reducing the number of false positives by changing the logic by which measurements are run
To incorporate such changes, we have decided to re-write our website measurement experiment entirely from scratch.
Our new experiment is tentatively called “Websteps”, and throughout August 2021, we wrote a series of design documents to describe this new experiment.
These (currently internal) design documents for our new Websteps experiment include:
* High-level description of the Websteps experiment
* Evaluation criteria
* Underlying library: Data format and algorithms library
* Prototype: Where to find a code prototype and how to use it
* Data format: Specification of measurements data format
* Example measurements: How measurements look like with specific errors
* Known-data quality issues
We started off by reviewing these documents internally in our team. Based on internal review, we made a series of iterations and improvements. The next steps of the review process involve sharing these documents with certain community members (who have relevant expertise), and eventually sharing them publicly to encourage public review and feedback.
The new websteps experiment depends on the use of a new type of test-helper which assists it in performing discovery of the testing targets and is also used as a baseline for comparison. We wrote a specification for this new test helper in the ooni/spec repo: https://github.com/ooni/spec/pull/219
In August 2021, we also reviewed the pull requests by Google Summer of Code (GSoC) student Kathrin Elmenhorst, who added support for using the utls library for TLS parroting (https://github.com/ooni/probe-cli/pull/442). She also worked with us on implementing a prototype for the new Websteps experiment (https://github.com/ooni/probe-cli/pull/432) and fixed a bug in it (https://github.com/ooni/probe-cli/pull/447).
## OONI Probe Mobile
We worked on identifying the priorities and next steps for the development of the OONI Probe mobile app. To this end, we went through community feedback (through tickets and internal documents) to identify feature requests, bug reports, and other areas that need improvement. Based on this, we wrote an internal document where we map out the priorities and next steps for the development of OONI Probe Mobile over the next 6 months, based on which we opened relevant tickets.
## OONI Run
Based on community feedback, we worked on making some improvements to the OONI Run (https://run.ooni.io/) platform. In particular, we worked on ensuring that HTTPS is used as the default prefix when URLs are added to the platform, that the HTTPS prefix is not duplicated when URLs are pasted in bulk, and we added a note informing users that they can copy-paste multiple URLs into the platform. This work is tracked here: https://github.com/ooni/run/pull/74
## OONI Probe Desktop
We made progress on improving the end-to-end testing of the OONI Probe Desktop app. Specifically, tests were added for ensuring that the onboarding process, the dashboard, settings, and the test results screens work as expected: https://github.com/ooni/probe-desktop/pull/243
We also worked on implementing a fix to a bug that led to the autorun settings not being preserved across updates: https://github.com/ooni/probe-desktop/pull/237
## OONI backend
In August 2021, we worked on the following OONI backend activities:
* Developed beta quality data analysis capabilities for pluggable transport tests: https://github.com/ooni/ooni.org/issues/766
* Added support to the OONI API for displaying PT measurements: https://github.com/ooni/ooni.org/issues/771
* API: Searched and fetched measurements by uid in addition to rid/input (fixes issues around paging and rid/input collision): https://github.com/ooni/api/pull/263
* Monitored the growth of unattended OONI Probe runs and test list coverage
* Investigated an odd decrease in the number of received measurements from OONI Probe Android and unattended OONI Probe desktop runs, and added new metrics to our internal dashboard.
* Tested the URL prioritization backend: https://github.com/ooni/backend/issues/418
* Fixed URL prioritization and added end-to-end testing: https://github.com/ooni/api/pull/264
* Added support for using the new OONI measurement data format in s3 for end-to-end testing
* Created admin accounts for our test-list web UI
* Prioritized the testing of the GRP Citizen Lab category code in Zambia in response to the blocking of social media (amid elections)
* Looked into OONI API logs to detect popular queries that take a long time to run (heavy hitters)
* Deployed Nginx caching
* Worked on improvements to the system used to predict the expected measurement volume from the API
* Experimented with different database indexes on our fastpath table
* Worked on updating our fast-path pipeline
* Worked on updating the logic used to populate the counter tables which are used for performing aggregation queries (counters_table_updater)
## Google Summer of Code (GSoC)
Throughout the summer of 2021, we had the opportunity to host two Google Summer of Code (GSoC) students: Kathrin Elmenhorst and Shivam Shekhar.
Between June 2021 to August 2021, Kathrin worked with us on developing OONI Probe network experiments (https://community.torproject.org/gsoc/ooni-probe-experiments/), while Shivam worked with us on developing integration and unit tests for the OONI Probe desktop app (https://community.torproject.org/gsoc/ooni-probe-testing/).
In August 2021, Kathrin and Shivam completed their GSoC projects with us, and we submitted a review of their performance. Overall, we were thrilled to work with Kathrin and Shivam, and we thank them for their excellent contributions to OONI projects!
## Test list updates
In August 2021, we made a minor update to the Citizen Lab test lists for Cuba and Venezuela. In particular, we added a site (to both lists) which tracks fake news in Venezuela, but which was reportedly blocked in Cuba.
Pull request: https://github.com/citizenlab/test-lists/pull/814
## Community activities
### OONI Community Meeting
On 31st August 2021, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we discussed the following topics:
1. Tor usage in Turkmenistan is decreasing. How can we increase OONI Probe testing of Tor in Turkmenistan?
2. Preparing for Russia’s upcoming 2021 election: Measurement and advocacy.
3. Reviewing the test list for Afghanistan in accordance with possible new regulations.
## Userbase
In August 2021, 13,304,893 OONI Probe measurements were collected from 4,938 AS networks in 194 countries around the world.
This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/
~ OONI team.