Hello,
Last night the OONI team published a new report, documenting the *ongoing
blocking of Telegram in Brazil*.
You can read the report here: https://ooni.org/post/2023-brazil-telegram/
OONI data shows that some ISPs in Brazil immediately complied with the
federal judge’s decision to suspend Telegram, as the blocking of Telegram
started on 26th April 2023 (same day as the judge's decision).
However, the block was not implemented by all ISPs in Brazil, nor was it
implemented in the same way.
Some ISPs blocked access to all tested Telegram endpoints, while others
only blocked some of them. This suggests lack of coordination between
providers, and that each ISP implemented the block autonomously. Learn more
details through our report.
You can continue to monitor the blocking of Telegram in Brazil through OONI
data, which is automatically published in real-time:
https://explorer.ooni.org/chart/mat?probe_cc=BR&since=2023-03-29&until=2023…
We thank OONI Probe users in Brazil who contributed measurements,
supporting this study. <3
~ OONI team.
Hello,
Today the OONI team published a new research report documenting the t*hrottling
of news media websites in Kazakhstan*.
You can read the report here:
https://ooni.org/post/2023-throttling-kz-elections/
In our latest report, we analyse the ongoing throttling of Radio Free
Europe/Radio Liberty (RFE/RL) Kazakhstan’s service websites in Kazakh and
Russian (www.azattyq.org and rus.azattyq.org) which started on 27th
September 2022.
We also document the temporary throttling of Current Time TV during
Kazakhstan's 2022 presidential election.
OONI data shows the ongoing throttling of RFE/RL Kazakhstan’s service
websites on most tested Kazakh providers, while Current Time TV was mainly
throttled on AS9198 (JSC Kazakhtelecom).
This report documents how OONI data can be used to investigate and document
cases of throttling.
We thank OONI Probe users in Kazakhstan for contributing measurements,
making this study possible. <3
~ OONI team.
Hi friends,
eQualit.ie would like to invite you to "Internet Without Borders", a hybrid
conference and hackathon that will take place in Berlin on April 28-30th:
https://internetborders.net/
It's an in-person event at Forum Factory, Besselstraße 13-14, 10969 Berlin.
Internet Without Borders is a transdisciplinary laboratory, uniting a
critical mass of developers, journalists, circumvention tech experts, OSINT
researchers, activists and designers. The main mission of this event is to
fight internet censorship and other forms of digital and infrastructural
oppression spread by the Russian government.
OONI is presenting an important challenge at the hackathon, with a $600 USD
prize to win! The task is to develop a VPN measurement research methodology
for Russia and temporarily occupied territories of Ukraine.
Our fellow Ain Ghazal will be curating this challenge on site, and you will
have an opportunity to hack around with some of the OONI solutions and meet
other interesting tech organizations, such as Mozilla, Lantern,
Censorship.no, Amnezia VPN, LEAP and many other open source projects
focused on Internet Freedom. Drinks, lunch and coffee breaks are provided
to all conference guests.
See more information about the hackathon here:
https://internetborders.net/digital-resistance-lab-challenges/
The event is off-the-record, no photo and video is allowed. The event
respects Chatham House rules and tries to guarantee security and privacy to
participants. All applications are undergoing verification.
Participation at Internet Without Borders is free of charge! However,
registration is mandatory.
To register, please fill in this form:
https://internetborders.net/registration-berlin/
Thanks,
Maria.
Hi everyone,
Next Tuesday we will be hosting our monthly community meeting and we are welcoming everyone to join us there.
=> Where? OONI Slack channel: https://slack.ooni.org/ <https://slack.ooni.org/> (bridged with IRC: ircs://irc.oftc.net:6697/#ooni)
=> When? Tuesday, 25th April 2023 at 14:00 UTC (for 1 hour)
Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep <https://pad.riseup.net/p/ooni-community-meeting-keep>
The monthly OONI community meetings aim to:
* Collect community feedback on OONI tools & methodologies
* Address questions in relation to the use of OONI tools and OONI data
* Foster discussions on internet censorship issues
* Receive updates from the community
We hope you can join us!
All the best,
~ OONI team
Hello,
This email shares OONI's monthly report for March 2023.
*# OONI Monthly Report: March 2023*
Throughout March 2023, the OONI team worked on the following sprints:
* Sprint 86 (1st-12th March 2023)
* Sprint 87 (13th-31st March 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Public launch of OONI Probe Web*
In March 2023, the OONI team released OONI Probe Web: a new browser-based
tool for measuring the blocking of websites.
Run OONI Probe Web: https://probe-web.ooni.org/
We built OONI Probe Web in response to long-term community feedback,
requesting a censorship measurement tool that can be run from a browser,
without requiring the installation of any software.
With OONI Probe Web, you will test the same types of websites as with the
OONI Probe apps (
https://ooni.org/support/faq#which-websites-will-i-test-for-censorship-with…)
Upon running OONI Probe Web, your test results will be published as open
data in near real-time (https://ooni.org/data/) However, please note that
OONI Probe Web is very limited in comparison to the OONI Probe apps (
https://ooni.org/install/) Due to limitations in the API available to web
browsers, it’s not possible to run advanced experiments necessary for
accurately detecting and characterizing website blocks from a browser.
We therefore kindly ask you to not treat OONI Probe Web as a replacement
for the OONI Probe apps, but as a complementary tool. We encourage you to
run OONI Probe Web and contribute measurements, the analysis of which will
help us improve upon OONI Probe Web on an ongoing basis.
You can learn more about OONI Probe Web in our blog post:
https://ooni.org/post/introducing-ooni-probe-web/
In preparation for the launch, we:
* Switched the endpoints used by OONI Probe Web to production (
https://github.com/ooni/probe-web/pull/6)
* Made several improvements to the copy and to the layout of the pages (
https://github.com/ooni/probe-web/pull/7) with special focus on the
informed consent procedure;
* Made changes to OONI Probe Web to ensure that it works better on mobile
platforms (https://github.com/ooni/probe-web/pull/9)
* Added support to the backend for analyzing OONI Probe Web measurements (
https://github.com/ooni/backend/pull/648)
* Added support for filtering OONI Probe Web measurements inside of OONI
Explorer (https://github.com/ooni/explorer/pull/847)
* Updated the OONI Data Policy (https://github.com/ooni/ooni.org/pull/1347)
*## Launched new OONI Explorer features and pages*
In March 2023, we launched several new OONI Explorer features and pages.
These include:
* New domain-centric pages: https://explorer.ooni.org/domains
* New network-centric pages: https://explorer.ooni.org/networks
* New “Internet outage” charts (integrating IODA, Google traffic, and
Cloudflare Radar data), available in each country-specific (
https://explorer.ooni.org/countries) and network-specific page (
https://explorer.ooni.org/networks)
* New user feedback reporting mechanism, available through the “Verify”
button in the banner of each OONI measurement page
The user feedback reporting mechanism enables OONI Explorer users to submit
feedback for each OONI measurement, helping the OONI team to improve OONI
data quality. Using the user feedback reporting mechanism requires logging
in with an email address (OONI Explorer user account). Email addresses are
not stored. They are just required so that we can send you a login link.
Leading up to the launch of the above, we wrapped up the relevant work.
Specifically, we:
* Added index pages for domains and networks that we have measurements for (
https://github.com/ooni/explorer/pull/830)
* Finished applying the designs for the new domain and network pages (
https://github.com/ooni/explorer/pull/830)
* Unified the appearance of the pre-existent pages by implementing new
designs created by a UI/UX designer (updated the general countries page,
each country page and several other components);
* Released third-party data integration on network and country pages (
https://github.com/ooni/explorer/pull/831)
* Released OONI Explorer user accounts and the user feedback reporting
mechanism for measurements (https://github.com/ooni/explorer/pull/790)
*## Published new OONI Explorer user guide*
In March 2023, we published a new user guide for OONI Explorer:
https://ooni.org/support/ooni-explorer
OONI Explorer (https://explorer.ooni.org/) is the world's largest open
dataset on Internet censorship, with new measurements published in near
real-time.
This user guide provides step-by-step instructions (with screenshots) on
how to use OONI Explorer to investigate internet censorship worldwide.
Through this guide, you will learn how to use OONI Explorer to:
* Discover blocked websites and apps around the world
* Access measurement data that can serve as evidence of internet censorship
* Generate charts based on aggregate views of OONI data
* Compare censorship across countries and networks
* Filter measurements based on various parameters (such as country, ASN,
date range, OONI Probe test, websites categories, domains, etc.)
Upon reading this guide, we hope you will feel empowered to investigate
internet censorship through the use of OONI Explorer.
We thank all OONI Probe (https://ooni.org/install/) users worldwide who
have contributed – and continue to contribute – measurements, shedding
light on internet censorship.
*## Published new documentation on interpreting OONI data*
In March 2023, we published new documentation which explains how to
interpret OONI data.
You can access this documentation here:
https://ooni.org/support/interpreting-ooni-data/
OONI data is open data on Internet censorship around the world, updated in
real-time.
With our new guide, you can learn how to interpret OONI data to monitor and
respond to Internet censorship worldwide.
*## Published new documentation on coordinating censorship measurement
campaigns*
In March 2023, we published a new guide through which you can learn how to
coordinate OONI censorship measurement campaigns with your community.
Read the guide here:
https://ooni.org/support/ooni-censorship-measurement-campaigns
Is your country preparing for elections? Did new blocks emerge during
protests? Would you like to measure censorship in different regions in your
country, or in different countries around the world?
Community coordination can help with rapidly measuring and responding to
Internet censorship events.
Through our new guide, you can learn which steps you can take to organize
censorship testing with other OONI Probe users. We also share a few
examples of OONI censorship measurement campaigns led by groups in the
internet freedom community.
*## Published new OONI Outreach Kit*
In March 2023, we published a new OONI Outreach Kit:
https://ooni.org/support/ooni-outreach-kit/
Our Outreach Kit includes flyers, brochures, workshop slides & other
resources that you can use as part of your OONI community engagement
efforts!
Specifically, the Outreach Kit includes:
* OONI brochures: https://ooni.org/support/ooni-outreach-kit/#ooni-brochures
* OONI Probe flyers and leaflets:
https://ooni.org/support/ooni-outreach-kit/#ooni-probe-flyers-and-leaflets
* OONI Explorer flyers:
https://ooni.org/support/ooni-outreach-kit/#ooni-explorer-flyers
* Internet censorship flyers:
https://ooni.org/support/ooni-outreach-kit/#internet-censorship-flyers
* OONI screencasts:
https://ooni.org/support/ooni-outreach-kit/#ooni-screencasts
* OONI workshop slides:
https://ooni.org/support/ooni-outreach-kit/#ooni-workshop-slides
* Other OONI resources:
https://ooni.org/support/ooni-outreach-kit/#ooni-resources
We hope you find these materials useful, and we thank you for your OONI
community engagement efforts! We also thank Ura Design (https://ura.design/)
for the beautiful design of the OONI Outreach Kit.
*## Updated the Get Involved section of the OONI website*
In light of the aforementioned publications, we edited the copy of the Get
Involved page on the OONI website to provide updated information and links (
https://github.com/ooni/ooni.org/pull/1366/files)
The updated version of the OONI Get Involved page can be viewed here:
https://ooni.org/get-involved/
*## Published OONI documents translated into 5 languages*
In March 2023, we published several OONI documents translated into 5
languages: Arabic, Farsi, Russian, Swahili, and Spanish.
The translated OONI documents are:
* OONI Frequently Asked Questions (FAQ): https://ooni.org/support/faq
* OONI Glossary: https://ooni.org/support/glossary
* Potential risks: https://ooni.org/about/risks/
Below we share links for the translated documents in each of the 5
languages.
*### Arabic*
Thanks to Ahmed Gharbeia, the following OONI documents are now available in
Arabic:
* OONI Frequently Asked Questions (FAQ): https://ooni.org/ar/support/faq
* OONI Glossary: https://ooni.org/ar/support/glossary
* Potential risks: https://ooni.org/ar/about/risks
*### Farsi*
Thanks to Miaan Group, the following OONI documents are now available in
Farsi:
* OONI Frequently Asked Questions (FAQ): https://ooni.org/fa/support/faq
* OONI Glossary: https://ooni.org/fa/support/glossary
* Potential risks: https://ooni.org/fa/about/risks
*### Russian*
Thanks to Tatyana Boldyreva, the following OONI documents are now available
in Russian:
* OONI Frequently Asked Questions (FAQ): https://ooni.org/ru/support/faq
* OONI Glossary: https://ooni.org/ru/support/glossary
* Potential risks: https://ooni.org/ru/about/risks
*### Swahili*
Thanks to Zaina Foundation, the following OONI documents are now available
in Swahili:
* OONI Frequently Asked Questions (FAQ): https://ooni.org/sw/support/faq
* OONI Glossary: https://ooni.org/sw/support/glossary
* Potential risks: https://ooni.org/sw/about/risks
*### Spanish*
Thanks to Katherine Pennacchio and Mariengracia Chirinos, the following
OONI documents are now available in Spanish:
* OONI Frequently Asked Questions (FAQ): https://ooni.org/es/support/faq
* OONI Glossary: https://ooni.org/es/support/glossary
The Risks document had already been translated to Spanish by Derechos
Digitales: https://ooni.org/es/about/risks
Huge thanks to the translators who helped make important OONI documents
accessible to more communities worldwide!
*## Published blog post on the analysis of failed OONI measurements*
In March 2023, Gurshabad Grover (OTF Information Controls Research Fellow
at OONI) and Simone Basso (OONI engineer) published a new post titled: "How
we're improving OONI data quality: An analysis of failed measurements".
Read this post here:
https://ooni.org/post/improving-data-quality-analysis-of-failed-measurement…
You may have noticed that some OONI measurements are annotated as "failed".
This happens when the OONI Probe experiment fails to perform as expected
(e.g due to bugs).
To better understand why some OONI measurements fail, Gurshabad and Simone
analyzed 100 of the most failed Web Connectivity measurements collected
(during 3 days in June 2022) from India, Indonesia, and Pakistan (300
measurements in total).
They found that Web Connectivity measurements failed to the following
reasons:
* Instance of internet censorship
* Tested website was down
* Tested website was misconfigured
* Bugs
Out of the 3 countries, they found that most failed measurements (more than
75%) from Pakistan and Indonesia were symptomatic of internet censorship.
Based on this study, the authors identified some next steps for improving
OONI data quality:
https://ooni.org/post/improving-data-quality-analysis-of-failed-measurement…
We encourage researchers to analyze failed OONI measurements and help us
further improve OONI data quality.
*## Published URL prioritization web interface*
OONI’s smart URL list system (
https://ooni.org/post/ooni-smart-url-list-system/) prioritizes the testing
of certain URLs over others. Currently, popular social media websites
(which are frequently blocked around the world) are prioritized for testing
the most globally.
To enable the internet freedom community to review URL priorities and
contribute feedback (i.e suggest URLs for testing prioritization), we
published a new URL prioritization web interface:
https://test-lists.ooni.org/prioritization
To this end, we added support for displaying URL priorities in the Test
Lists Editor interface (https://github.com/ooni/backend/pull/646) and we
added a web form that enables community members to suggest URLs for
prioritization (https://github.com/ooni/test-lists-ui/pull/72)
*## OONI Probe Mobile*
In March 2023, we continued to improve upon the OONI Probe mobile app.
Specifically, we:
* Continued to make improvements to the next OONI Probe Mobile release
(3.8.0) based on feedback from internal testing;
* Made the Android Manifest general enough based on community feedback (
https://github.com/ooni/probe/issues/1989)
* Continued working towards ensuring that the OONI Probe iOS app does not
crash when OONI backend services are down (
https://github.com/ooni/probe/issues/2201)
* Continued working towards rewriting OONI Probe in Flutter (
https://github.com/ooni/probe/issues/2359)
* Worked towards addressing a bug impacting status information in dark mode
(https://github.com/ooni/probe/issues/2423)
* Fixed bugs (https://github.com/ooni/probe/issues/2427,
https://github.com/ooni/probe/issues/2428)
*## OONI Run*
We released a new stable version of OONI Run (v1.0.0:
https://github.com/ooni/run/releases/tag/v1.0.0) which includes some of
the improvements which are backward compatible with older versions of OONI
Probe. These include support for using translations with a language
selector, copy edits, updating software dependencies and several bug fixes.
Following the release of OONI Run v1.0.0, we deployed it and made some
improvements to the deployment process.
We also merged the specification for the next generation of OONI Run which
is going to be implemented over the course of the next year:
https://github.com/ooni/spec/pull/249
*## OONI Probe CLI*
We continued working on the OONI Probe CLI 3.17 release. While performing
quality assurance, we noticed and fixed an issue that prevented mobile apps
from taking advantage of the prototype richer input functionality exported
by the OONI backend. We fixed this issue (
https://github.com/ooni/probe-cli/pull/1123) and released OONI Probe CLI
3.17.1 (https://github.com/ooni/probe-cli/releases/tag/v3.17.1)
Based on a community request, we exported all the internal Go packages used
by OONI Probe CLI (https://github.com/ooni/probe-cli) in the OONI Probe
Engine (https://github.com/ooni/probe-engine) repository. We will
semi-automatically update this repository after each minor release, to
allow community members to import our internal implementation details from
their applications and libraries written in Go.
We merged the dslx package, which provides a composable high-level API for
writing OONI experiments (https://github.com/ooni/probe/issues/2402) For
additional information on this package, please take a look at the design
document that explains this functionality in detail:
https://github.com/ooni/probe-cli/blob/master/docs/design/dd-005-dslx.md
We continued to investigate Android crashes caused by tor. We updated the
fdsan-related issue we previously opened in February 2023 (
https://gitlab.torproject.org/tpo/core/tor/-/issues/40747) providing
additional information on how the problem could be fixed in a way that
works on both Unix and Windows systems. We gathered additional evidence
regarding the pubsub_install crash and opened another issue in the tor
issue tracker (https://gitlab.torproject.org/tpo/core/tor/-/issues/40774)
*## Expanding OONI’s testing model to support richer testing input*
We continued exploring the design space around passing richer input to
experiments (https://github.com/ooni/ooni.org/issues/1295) by discussing
internally a way to update the check-in API and the OONI Run v2
specification to support richer testing input. We aim to review and
finalize this design document in the next development sprints and update
the existing PoC.
*## Creating a throttling measurement methodology*
We specified and implemented an integration testing tool, called “netem”,
to facilitate writing integration tests that simulate extreme throttling
conditions (https://github.com/ooni/probe/issues/2424) The netem tool,
which lives in its own GitHub repository (https://github.com/ooni/netem)
uses GVisor to create user-space TCP/IP network stacks. When running
integration tests, the traffic generated by OONI Probe uses its own GVisor
stack. We route this traffic through network links written in software that
simulate latency and losses. We started integrating netem (
https://github.com/ooni/probe/issues/2431) with the OONI Probe CLI
repository (https://github.com/ooni/probe-cli/)
*## OONI backend*
Throughout March 2023, we continued to work on a series of backend
improvements.
Specifically, we:
* Deployed the change to the fastpath to use fingerprints from GitHub (
https://github.com/ooni/pipeline/pull/407)
* Implemented the publication of the URL prioritization listing in the API (
https://github.com/ooni/backend/pull/646)
* Added an updater for ASN metadata to the pipeline (
https://github.com/ooni/pipeline/pull/414)
* Implemented an API endpoint for mapping ASNs to AS organization names (
https://github.com/ooni/api/pull/326,
https://github.com/ooni/backend/issues/635)
* Added domain and network endpoints to the API (
https://github.com/ooni/backend/pull/641)
* Added backend support to the MAT to accept multiple values in input
parameters (https://github.com/ooni/api/pull/324,
https://github.com/ooni/backend/issues/584)
* Removed the “test_name” check from the list of measurements in the API (
https://github.com/ooni/backend/pull/647)
* Merged the pipeline and API git repositories into one to simplify and
speed up development (https://github.com/ooni/backend/pull/638)
* Fixed the API integration test (after adding new columns to the fastpath)
(https://github.com/ooni/api/pull/325)
* Ensured that the rotation table history is maintained (
https://github.com/ooni/pipeline/pull/412)
* Increased the worker count and simplified the queue in the fastpath (
https://github.com/ooni/backend/pull/650)
* Fixed a bug affecting the platform field extraction in the fastpath (
https://github.com/ooni/backend/pull/652)
* Ensured that the API uses the fastpath repo in CI (
https://github.com/ooni/backend/pull/649)
* Cleaned and refactored the codebase (
https://github.com/ooni/backend/pull/645)
* Implemented log collection on the monitoring host using Vector.
*## Automating censorship detection and characterization based on OONI
measurements*
We made stable progress on the front of automating the detection and
characterization of censorship based on OONI measurements.
In March 2023, we focused on analyzing DNS level interference, as
understanding the outcome of the DNS measurement is a prerequisite for the
interpretation of the following measurements. Specifically, we implemented
a DNS blocking analysis that is purely based on SQL queries and we set up a
system to assess its accuracy by comparing it with blocking fingerprints.
We experimented with applying machine learning algorithms to the DNS
analysis. They present very promising cursory results and we will be
exploring this avenue further.
The outcomes of this experimentation and DNS analysis were presented
internally to our team, as well as to external researchers in the network
measurement field to request feedback.
We also improved the validation of blocking fingerprints (
https://github.com/ooni/blocking-fingerprints/pull/6) and we removed a
fingerprint that was leading to false positives (
https://github.com/ooni/blocking-fingerprints/pull/7)
*## Improving the quality of test lists*
To improve the quality of test lists, we created a prototype script for
detecting and optionally automatically removing expired domains from the
lists. We implemented this functionality inside the OONI Probe CLI
repository (https://github.com/ooni/probe-cli/pull/1114) We will test and
discuss this prototype with our partners to gather feedback and evaluate
the next steps.
For the time being, we decided to refrain from automatically removing
expired domains because we found cases where expired domains redirect to
other domains that are blocked (
https://github.com/citizenlab/test-lists/pull/1247) and, therefore, their
automatic removal would result in missing cases of censorship.
We also mined OONI data to identify URLs that were manually tested by OONI
Probe users, and which are not currently included in the Citizen Lab test
lists (https://github.com/ooni/ooni.org/issues/1228) Based on our
analysis, we identified more than 1,000 URLs (from the last 3 months) which
should be peer-reviewed (and categorized by country researchers/experts)
before added to the test lists (
https://gist.github.com/hellais/14e75ce8b9d5cf04f78644c8e9f66554#file-mine-…)
In the meanwhile, we added a few of those URLs to test lists (
https://github.com/citizenlab/test-lists/pull/1250)
*## OONI interview for Roskomsvoboda’s Privacy Day event 2023*
In January 2023, OONI’s Maria provided an interview for Roskomsvoboda’s
Privacy Day 2023 event (https://2023.privacyday.net/) as part of which she
discussed how people in Russia can participate in OONI censorship
measurement.
In March 2023, Roskomsvoboda published the interview in a blog post (
https://roskomsvoboda.org/post/maria-xynou-ooni/) and on their YouTube
channel (https://www.youtube.com/watch?v=_yugAH26CK0)
*## Community use of OONI data### Access Now’s press release on ongoing
social media blocks in Ethiopia*
In March 2023, Access Now published a press release urging Ethiopian
authorities to stop blocking access to social media platforms. As part of
this press release, they cite OONI measurements on the ongoing blocking of
social media platforms in Ethiopia.
Access Now’s press release is available here:
https://www.accessnow.org/press-release/ethiopia-social-media-protest/
*### Sinar Project’s article on verifying website blocks with OONI Explorer*
Our partner, Sinar Project, published an article (“Is the Website Blocked?
Verifying Internet Censorship with OONI Explorer”) which explains how to
verify the blocking of websites through the use of OONI Explorer.
Their article is available here:
https://imap.sinarproject.org/news/is-the-website-blocked-verifying-interne…
*## Community activities### OONI workshop for election observers in
Kazakhstan*
On 9th March 2023, OONI’s Elizaveta facilitated an online workshop on
coordinating OONI censorship measurement campaigns for election observers
in Kazakhstan.
Following this workshop, the participants created a Telegram chat to
coordinate on OONI censorship measurement during Kazakhstan’s elections on
19th March 2023.
*### MozFest 2023*
On 22nd March 2023, OONI’s Arturo attended MozFest 2023 (
https://www.mozillafestival.org/) where he participated as a speaker on
the panel “Co-Creating A Better Tech Future: Mozilla Data Futures Lab” (
https://schedule.mozillafestival.org/session/EMSW79-1)
As part of his presentation, Arturo shared the activities OONI plans to do
(in creating the next generation OONI Run) as part of the Mozilla Data
Futures Lab 2023 cohort (https://foundation.mozilla.org/en/data-futures-lab/)
*### DEMHACK 6 hackathon*
On 25th and 26th March 2023, OONI’s Simone and Elizaveta mentored the team
working on OONI’s task at the DEMHACK 6 hackathon (https://6.demhack.org/)
As part of this hackathon, the team of participants worked on developing a
script for the automatic identification of parked domains in the Citizen
Lab test lists.
*### OONI workshop for digital rights organizations from Central Asia*
Between 27th-29th March 2023, OONI’s Elizaveta participated in an in-person
training organized by CIPI and Internews for digital rights organizations
from Central Asia (Kazakhstan, Kyrgyzstan, Tajikistan, Uzbekistan).
As part of this training, Elizaveta facilitated an OONI workshop and helped
the organizations determine how OONI data may be useful to their work.
*### RWC 2023*
Between 27th-29th March 2023, OONI’s Arturo traveled to Japan to attend the
Real World Crypto (RWC) 2023 conference (https://rwc.iacr.org/2023/)
The goal of attending this conference was to discuss with cryptographers a
problem OONI is trying to solve related to implementing a privacy
preserving way to authenticate probes and which might benefit from some
sort of anonymous credential system.
*### OONI Community Meeting*
On 28th March 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we discussed the
following topics:
1. Updates from the OONI team.
2. The effects of Russia's invasion of Ukraine on Internet censorship in
Russia.
3. Enabling more extensive beta testing of OONI tools by the community.
4. OONI Probe Web: Community questions about the “URL Limit” feature.
*## Measurement coverage*
In March 2023, 61,415,821 OONI Probe measurements were collected from 3,058
AS networks in 167 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hello,
This email shares OONI's monthly report for February 2023.
*# OONI Monthly Report: February 2023*
Throughout February 2023, the OONI team worked on the following sprints:
* Sprint 84 (1st-12th February 2023)
* Sprint 85 (13th-28th February 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Published research report on internet censorship in Russia*
On 24th February 2023 (1 year since the start of the war in Ukraine), we
co-published a new research report, in collaboration with Roskomsvoboda,
which documents how Internet censorship in Russia changed over the last
year based on OONI data analysis.
Read our report in:
* English: https://ooni.org/post/2023-russia-a-year-after-the-conflict/
* Russian: https://ooni.org/ru/post/2023-russia-a-year-after-the-conflict/
You can also read Roskomsvoboda's publication about the report here:
https://roskomsvoboda.org/post/rks-ooni-odin-god/
Overall, OONI data analysis shows the blocking of 494 domains in Russia.
Many more websites are also blocked, beyond those tested. The blocked
domains fall under 28 categories, suggesting pervasive levels of internet
censorship in Russia.
New blocks that emerged in Russia over the last year include:
* Blocking of Russian & international human rights websites (e.g. Human
Rights Watch & Amnesty International)
* Blocking of investigative journalism & news media sites
* Blocking of Instagram, SoundCloud and Patreon
Access to the Tor Project's website was temporarily unblocked in Russia
between 15th to 28th July 2022. Several discontinued sites have been
unblocked over the last year (possibly as part of a "blocklist cleanup").
We confirmed the blocking of 48 domains (including a Let's Encrypt domain)
based on OONI data, which do not appear to be included in Russia's official
blocking registry. OONI data also suggests that access to a major CDN was
blocked in the past year.
*## Published research report on internet censorship in Azerbaijan*
On 28th February 2023, we co-published a new research report on internet
censorship in Azerbaijan, in collaboration with our partner, Azerbaijan
Internet Watch. As part of this research, we analyzed OONI data collected
from Azerbaijan over the last year.
The report was published by:
* OONI: https://ooni.org/post/2023-azerbaijan-internet-censorship/
* Azerbaijan Internet Watch:
https://www.az-netwatch.org/news/ooni-measurements-show-ongoing-internet-ce…
As part of our analysis of OONI measurements collected from Azerbaijan
between January 2022 to February 2023, we found:
** Blocking of news media websites. *Azerbaijan continues to block access
to several independent news media websites. OONI data also suggests that
some ISPs in Azerbaijan may have started blocking access to the Guardian on
25th December 2022.
** Azerbaijan and Russia block each other’s news media. *In early June
2022, Azerbaijan started blocking access to Russia’s state-run RIA Novosti
media website. Since (at least) 7th June 2022, Russian ISPs have been
blocking access to Azerbaijani news media websites (haqqin.az, minval.az,
oxu.az, ru.oxu.az, ru.baku.ws). These blocks remain ongoing.
** Temporary blocking of TikTok amid border clashes with Armenia. *During
the September 2022 border clashes, both Azerbaijan and Armenia blocked
access to TikTok. While the TikTok block was lifted in Armenia by 21st
September 2022 (only lasting about a week), the TikTok block remained in
place in Azerbaijan until November 2022 (lasting about 2 months).
** Blocking of circumvention tool sites.* Azerbaijan continues to block
access to numerous circumvention tool websites, potentially limiting the
ability to circumvent internet censorship in Azerbaijan. However, most OONI
measurements suggest that tested circumvention tools (Tor and Psiphon)
appear to be reachable.
** Variance of censorship across networks. *While most ISPs in Azerbaijan
appear to adopt similar censorship techniques (as we continued to observe
connection timeouts in most anomalous measurements across ASNs), different
ISPs block access to different websites over time.
*## Published report on social media blocks in Ethiopia*
On 15th February 2023, we published a report on the blocking of social
media platforms in Ethiopia.
OONI data from Ethiopia showed the ongoing blocking of:
* Facebook and Telegram (since 9th February 2023)
* YouTube (since 10th February 2023)
Read the report here:
https://ooni.org/post/2023-ethiopia-blocks-social-media/
*## Published report on DW and Wikipedia blocking in Pakistan*
On 15th February 2023, we published a report on the blocking of Wikipedia
and Deutsche Welle (DW) in Pakistan.
OONI data from Pakistan showed:
* Access to Wikipedia restricted between 1st-6th February 2023 (48-hour
degradation, followed by block)
* Ongoing blocking of Deutsche Welle (DW) since (at least) 16th January 2023
Read the report here:
https://ooni.org/post/2023-pakistan-blocks-wikipedia-and-dw/
*## Published report on Twitter throttling in Turkey*
On 15th February 2023, we published a report on the throttling and DNS
blocking of Twitter in Turkey in the aftermath of a deadly earthquake.
On 8th February 2023, OONI data from Turkey showed that access to Twitter
was throttled on at least 4 networks, and blocked by means of DNS
interference on other networks.
Read the report here:
https://ooni.org/post/2023-turkey-throttling-blocking-twitter/
*## OONI reports for the ISOC Pulse project on Internet shutdowns*
As a data partner for ISOC’s Pulse Shutdown project (
https://pulse.internetsociety.org/shutdowns) we have been working on
creating “OONI reports” (with relevant OONI data, interpretation, and
charts) for all of the “service blocking” entries listed on the Pulse
shutdown timeline (ranging from March 2018 to February 2023) over the last
year.
In February 2023, we completed this work, having written the “Data and
analysis” section for all “service blocking” entries of the ISOC Pulse
Shutdown timeline (where relevant OONI data exists). As part of this work,
we found that OONI data is available for almost all blocking events
reported worldwide over the last 5 years (at least based on those listed on
the ISOC Pulse shutdown timeline).
In total, we wrote 49 OONI reports for the ISOC Pulse shutdown timeline.
These reports will be included in the “Data and analysis” section of
“service blocking” entries.
*## OONI Probe Mobile*
In February 2023, we continued to improve upon the OONI Probe mobile app.
Specifically, we:
* Prepared the next OONI Probe Mobile release (3.8.0) for internal testing;
* Fixed a bug ensuring that the network name is displayed along with the
ASN (https://github.com/ooni/probe/issues/2371)
* Worked towards addressing a bug affecting automated testing on OONI Probe
iOS (https://github.com/ooni/probe/issues/2259)
* Worked towards enabling the transfer of the OONI Probe Android app to a
SD card (https://github.com/ooni/probe/issues/2019)
* Worked towards ensuring that the OONI Probe iOS app does not crash when
OONI backend services are down (https://github.com/ooni/probe/issues/2201)
*## OONI Run*
OONI Run (https://run.ooni.io/) can be translated via the Transifex (
https://explore.transifex.com/otf/ooni-run/) platform.
In February 2023, thanks to the Localization Lab community, OONI Run was
translated into 6 languages: Chinese, German, Thai, Russian, Spanish, and
Turkish.
*## OONI Probe CLI*
In February 2023, we released OONI Probe CLI 3.17.0:
https://github.com/ooni/probe-cli/releases/tag/v3.17.0
Notably, OONI Probe CLI 3.17.0 includes an updated version (v0.5.20) of the
Web Connectivity experiment, a TLS middlebox experiment, backend proxy
support for Tor Snowflake, OONI Run improvements, and many other
improvements (https://github.com/ooni/probe-cli/releases/tag/v3.17.0)
This new release will be the first one to support providing richer input to
experiments. We now support dynamically instructing the probes to enable
experimental Web Connectivity functionality through the OONI backend.
As part of this release cycle, we investigated two issues related to
running tor on Android devices. The first issue (
https://gitlab.torproject.org/tpo/core/tor/-/issues/40747,
https://github.com/ooni/probe/issues/2405) deals with the OONI Probe
Android app receiving a SIGABRT signal because tor closes a file descriptor
twice, thus triggering Android’s fdsan (
https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md)
We patched the problem as part of the OONI Probe CLI 3.17.0 release. The
second issue deals with a mysterious SIGABRT crash that we are still
investigating.
While testing OONI Probe CLI 3.17.0, we also noticed and investigated
excessive CPU usage in the Web Connectivity test helper (
https://github.com/ooni/probe/issues/2413) We modified the codebase to add
support for profiling, started fixing the most obvious issues, and
increased the number of test helper hosts to spread the load among them.
As part of testing, we also triaged the following data quality issues:
https://github.com/ooni/probe/issues/2410https://github.com/ooni/probe/issues/2411https://github.com/ooni/probe/issues/1925#issuecomment-1429373491https://github.com/ooni/probe/issues/2412https://github.com/ooni/probe/issues/2420https://github.com/ooni/probe/issues/2421
We fixed some of these data quality issues, as documented below:
https://github.com/ooni/probe-cli/pull/1110https://github.com/ooni/probe-cli/pull/1111https://github.com/ooni/probe/issues/2293
Where feasible, we backported to the OONI Probe CLI 3.17.0 branch.
After M-Lab deprecated their locatev1 API, we also migrated the DASH
experiment to use the locatev2 API and included this patch into the 3.17.0
branch (https://github.com/ooni/probe/issues/2398)
We also started working on exposing the OONI engine API, which is part of
the probe-cli repository, as a dynamic library (
https://github.com/ooni/probe/issues/2414)
*## Expanding OONI’s testing model to support richer testing input*
The OONI Probe CLI 3.17.0 release (
https://github.com/ooni/probe-cli/releases/tag/v3.17.0) starts adding
support for richer testing input (
https://github.com/ooni/ooni.org/issues/1291) We also continued to
improve such support in the main development branch of OONI Probe CLI and
OONI API. Specifically, we modified the API to directly return test helper
information inside the check-in API response (
https://github.com/ooni/probe/issues/2392)
We also started working on a design document regarding the richer testing
input functionality and we began working on the related PoC (
https://github.com/ooni/probe-cli/pull/1075)
*## Creating a throttling measurement methodology*
As part of our plans to create a methodology for measuring throttling (
https://github.com/ooni/ooni.org/issues/1296) we started sketching out a
possible integration testing strategy.
At the end of this reporting period, we wrote a diff for probe-cli (
https://gist.github.com/bassosimone/3ce52a37fc7394f7cce82391685c5477) which
works as follows:
* We hijack the core networking code to use a GVisor-based (
https://gvisor.dev/) TCP/IP stack in userspace;
* We connect this stack, representing the OONI Probe client with other
GVisor stacks representing servers;
* We add latency and throttling on those virtual channels when the network
traffic meets specified conditions (e.g., depending on the SNI used during
the TLS handshake).
Writing good integration tests for this upcoming functionality is crucial
to ensure that we do not break the Web Connectivity experiment while
applying the necessary changes to add support for collecting network
performance metrics (as planned:
https://github.com/ooni/ooni.org/issues/1297)
*## OONI Explorer*
We continued to work with a designer on the new OONI Explorer domain and
network centric pages (https://github.com/ooni/explorer/pull/830) We
updated the copy for the user feedback reporting system and we worked on
end-to-end tests. We made improvements to the third-party data integration,
notably adding support for the caching of proxied Cloudflare requests and
other UI changes (https://github.com/ooni/explorer/pull/831) We also
released a new MAT filter that allows showing data for different time
intervals: hourly, daily, weekly and monthly.
*## OONI backend*
Through February 2023, we continued to work on a series of backend
improvements.
Specifically, we:
* Implemented datacenter filtering in rotation (
https://github.com/ooni/pipeline/pull/408)
* Scaled up our infrastructure to 4 test helpers (
https://github.com/ooni/api/pull/320) and correlated the test runtime with
the deployment of the new test helpers to measure their effectiveness;
* Added more networks to the Web Connectivity v0.5 feature flag (
https://github.com/ooni/api/pull/318)
* Fixed a bug affecting authentication in the Test Lists Editor (
https://github.com/ooni/test-lists-ui/issues/71)
* Fixed the pipeline CI permissions (
https://github.com/ooni/pipeline/pull/409)
* Extracted more values from measurements including software and test
version, engine details, CPU architecture and test run time (
https://github.com/ooni/pipeline/pull/411,
https://github.com/ooni/pipeline/pull/410)
* Implemented an initial database backup tool.
*## Automating censorship detection and characterization based on OONI
measurements*
We made stable progress on the OONI data analysis tool (
https://github.com/ooni/data) Specifically, we:
* Made several improvements to the tooling that can be used for creating
plots that allow us to assess how well the analysis performs (
https://github.com/ooni/data/pull/25)
* Added support for mapping several “unknown_failures” that are a result of
software bugs, ensuring that the data is more clean and can be better
analyzed (
https://github.com/ooni/data/pull/25/commits/dc6517e0148be6c571a9e0768b50b5…
):
* Started exploring the possibility of performing the analysis directly
inside of the database engine (as opposed to doing in python code). This
resulted in a 10x performance increase, allowing us to analyze one day's
worth of OONI data in just over 5 minutes (compared to 2 hours of the
python implementation).
We also ingested new blocking fingerprints (
https://github.com/ooni/pipeline/pull/407) into the fastpath data
processing pipeline, thereby automatically detecting and confirming more
cases of website blocking around the world.
To evaluate how effectively OONI Probe measures and detects cases of
internet censorship, we compared measurements from Indonesia (that show
blocking) with Indonesia’s (“DNS-Trust”) blocking registry (
https://github.com/alsyundawy/dnstrust-apjii) We found that OONI
measurements that show DNS-based censorship are consistent with Indonesia’s
blocklist.
*## Creating a Social Media Censorship Alert System*
We continued to make progress on the initial event detector developed for
the new Social Media Censorship Alert System (
https://github.com/ooni/backend/issues/629) Specifically, we ran
experiments using Pandas, added more tests, and generated PNG/SVGs as part
of the testing.
*## Creating a new OONI Outreach Kit*
Throughout February 2023, we coordinated with Ura Design (
https://ura.design/) who worked on designing the new OONI Outreach Kit
materials. We are aiming to publish the Outreach Kit in March 2023.
*## Coordination of translation of OONI documents*
Throughout February 2023, we coordinated with translators with regards to
the translation of several OONI documents into 5 languages: Arabic, Farsi,
Russia, Swahili, and Spanish. We are aiming to publish the translated
documents in March 2023.
*## Test list updates*
We coordinated with our partner, TEDIC (https://ooni.org/partners/tedic/)
who provided extensive updates for the test list of Paraguay:
https://github.com/citizenlab/test-lists/pull/1224
The test list for Cambodia was also updated extensively by API Cambodia
(iMAP partner): https://github.com/citizenlab/test-lists/pull/1214
We reviewed and merged many other test list pull requests contributed by
community members:
https://github.com/citizenlab/test-lists/pulls?q=is%3Apr+is%3Aclosed
*## Rapid response### Wikipedia blocking in Pakistan*
Access to Wikipedia was temporarily restricted in Pakistan between 1st to
6th February 2023.
We rapidly responded by sharing relevant OONI data collected from Pakistan
(and encouraging further testing) on Twitter:
https://twitter.com/OpenObservatory/status/1620845976645963778 and
https://twitter.com/OpenObservatory/status/1621449233852252160
We subsequently published a report:
https://ooni.org/post/2023-pakistan-blocks-wikipedia-and-dw/
*### Social media blocks in Iraq*
On 6th February 2023, community members in Iraq reported to the #KeepItOn
mailing list that access to social media platforms (WhatsApp, Facebook,
Twitter, Instagram) would be banned during the country’s final exams,
starting on 5th February 2023 and ending on 13th February 2023 (from 4 am
to 12 pm local time). The decision was made by the PMO based on the
ministry of education recommendation.
We shared relevant OONI data with the #KeepItOn campaign on these blocks.
*### Twitter throttling in Turkey*
On 8th February 2023, in the aftermath of the earthquakes, Turkey throttled
access to Twitter.
Real-time OONI data collected from Turkey showed the block:
https://explorer.ooni.org/chart/mat?probe_cc=TR&test_name=web_connectivity&…
We rapidly responded by analyzing relevant OONI data collected from Turkey
and sharing our analysis through a Twitter thread:
https://twitter.com/OpenObservatory/status/1623398643595182083
OONI data showed that access to Twitter was throttled on at least 4
networks, and blocked by means of DNS on other networks.
We subsequently published a report:
https://ooni.org/post/2023-turkey-throttling-blocking-twitter/
*### Social media blocks in Ethiopia*
On 9th February 2023, amid church split tensions and calls for
anti-government protests, access to social media platforms was reportedly
blocked in Ethiopia.
We rapidly responded by sharing relevant OONI data collected from Ethiopia
(and encouraging further testing) on Twitter:
https://twitter.com/OpenObservatory/status/1624045402679873540 and
https://twitter.com/OpenObservatory/status/1624761465453379584
We subsequently published a report:
https://ooni.org/post/2023-ethiopia-blocks-social-media/
*## Community use of OONI data### Paper on internet censorship in Russia*
Censored Planet (in collaboration with other researchers) published a paper
(“Network Responses to Russia’s Invasion of Ukraine in 2022: A Cautionary
Tale for Internet Freedom”) which analyzes OONI data (along with Censored
Planet data) to examine internet censorship in Russia following the
February 2022 invasion of Ukraine.
Their paper (which was accepted to appear in USENIX Security 2023) is
available here:
https://censoredplanet.org/assets/russia-ukraine-invasion.pdf
*### Paper on censorship data analysis*
Censored Planet and Jigsaw published a paper (“Advancing the Art of
Censorship Data Analysis”) which discusses the challenges involved in
analyzing censorship measurement data. This paper shares examples from OONI
data and other public censorship datasets (such as Censored Planet).
Their paper (which was published by FOCI) is available here:
https://www.petsymposium.org/foci/2023/foci-2023-0003.pdf
*### Data Journalism guide for measuring internet shutdowns*
Journalist Sabrina Faramarzi published an article (“Kill switch: reporting
on and during internet shutdowns”) which aims to serve as a guide for data
journalists interested in measuring internet shutdowns. This article
features an interview with OONI’s Maria, where she discusses why
journalists should use OONI data to investigate internet censorship around
the world.
The article is available here:
https://datajournalism.com/read/longreads/internet-shutdowns-data-reporting
*## Community activities### OONI workshop for civil society in Azerbaijan*
On 1st February 2023, OONI’s Elizaveta facilitated an OONI workshop for
civil society groups in Azerbaijan. This workshop was hosted in
collaboration with our partner, Azerbaijan Internet Watch (
https://ooni.org/partners/azerbaijan-internet-watch/) The goal of the
workshop was to introduce civil society groups in Azerbaijan to OONI tools
to help boost OONI measurement coverage in Azerbaijan.
*### FOSDEM 2023*
On 4th and 5th February 2023, OONI’s Federico attended FOSDEM (
https://fosdem.org/2023/) in Brussels.
*### OONI workshop for human rights defenders in Nigeria*
On 16th February 2023, OONI’s Elizaveta facilitated an OONI workshop for
human rights defenders in Nigeria. This workshop was hosted in
collaboration with our partner, Paradigm Initiative (
https://ooni.org/partners/paradigm-initiative/) The goal of the workshop
was to introduce participants to OONI tools and data, encouraging
censorship measurement leading up to and during Nigeria’s 2023 elections
(which may trigger new censorship events).
*### OONI Run demo for the Localization Lab community*
On 22nd February 2023, OONI’s Elizaveta provided an OONI Run (
https://run.ooni.io/) demo for the Localization Lab community to encourage
the translation of the platform.
*### Live OONI session for the OPTIMA community in Senegal*
On 23rd February 2023, OONI’s Elizaveta briefly presented OONI and
addressed questions as part of Internews’ live OPTIMA session for community
members in Senegal.
*### OONI Community Meeting*
On 28th February 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we discussed the
following topics:
1. Updates from the OONI team.
2. Upcoming conferences and opportunities for censorship measurement
sessions.
3. Community feedback on early beta testing of OONI tools.
*## Measurement coverage*
In February 2023, 57,841,940 OONI Probe measurements were collected from
2,844 AS networks in 164 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hello,
This email shares OONI's monthly report for January 2023.
*# OONI Monthly Report: January 2023*
Throughout January 2023, the OONI team worked on the following sprints:
* Sprint 82 (2nd-15th January 2023)
* Sprint 83 (16th-29th January 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## OONI Probe Mobile*
We added support to OONI Probe Mobile for the re-testing of URLs that
presented anomalies (https://github.com/ooni/probe-android/pull/554) This
new feature enables OONI Probe users to more easily re-test URLs that
present signs of blocking, thereby contributing more measurements that
potentially contain evidence of internet censorship.
We also made improvements to RTL support for both OONI Probe Mobile and
Desktop (https://github.com/ooni/probe/issues/2346) we removed the
RiseupVPN test option from the app settings (
https://github.com/ooni/probe/issues/2204) we improved upon how tests are
run on iOS (https://github.com/ooni/probe/issues/2175) and we continued
working towards rewriting OONI Probe Mobile in Flutter (
https://github.com/ooni/probe/issues/2359)
*## OONI Run*
We worked towards ensuring that OONI Run links continue performing tests on
iOS, even when the device is locked (
https://github.com/ooni/probe/issues/2361)
Notably, we collaborated with the Localization Lab (
https://www.localizationlab.org/) on creating a new localization project
for OONI Run on Transifex (https://explore.transifex.com/otf/ooni-run/) and
we uploaded the relevant strings for translation. This enabled the
localization community to start translating the OONI Run platform (
https://run.ooni.io/) making it more accessible to communities worldwide.
*## Published OONI Run screencast*
To enable communities to use OONI Run (https://run.ooni.io/) to coordinate
website censorship testing, we created a screencast which shows how to use
the platform. In creating this screencast, we wrote a script (which appears
as subtitles, which can be translated) and we created a screen recording
illustrating how to use OONI Run.
We published the OONI Run screencast on our YouTube channel:
https://www.youtube.com/watch?v=OGRN7ve6cIA
*## OONI Probe CLI*
We carried out development and quality assurance work to prepare the OONI
Probe CLI 3.17.x release. To this end, we released OONI Probe ClI
v3.17.0-alpha (https://github.com/ooni/probe-cli/releases/tag/v3.17.0-alpha)
and OONI Probe CLI v3.17.0-alpha.1 (
https://github.com/ooni/probe-cli/releases/tag/v3.17.0-alpha.1) As part of
this work, we merged a fix that prevents the Web Connectivity v0.5
experiment from fetching data from the localhost (
https://github.com/ooni/probe-cli/pull/1027)
We also added support for performing A/B testing between Web Connectivity
v0.4 (the current stable release) and v0.5 (the new Web Connectivity
version). Once OONI Probe CLI 3.17.0 is released, we will start using this
mechanism to enable the Web Connectivity v0.5 experiment for some users and
compare the measurement results it collects with the ones collected by the
previous version.
This release also refactors the build system to verify we are building with
the expected compiler flags through unit tests. As part of this work, we
rewrote how we compile tor for Android to add support for patching and
hardening flags.
*## Expanding OONI’s testing model to support richer testing input*
Supporting a richer testing input model requires exposing the full check-in
API response to experiments. Such a response, in fact, potentially includes
configuration flags and other ancillary information that, along with the
input URL, provides richer input.
To move this objective forward, we fixed ooniprobe’s backend client to
observe the full check-in response, rather than just extracting the URLs (
https://github.com/ooni/probe-cli/pull/1037)
We then started using richer input when instantiating experiments by
implementing code to A/B test Web Connectivity implementations. This
feature inspects the check-in API response to extract feature flags (
https://github.com/ooni/probe-cli/pull/1039) We also extended the check-in
API itself to provide a feature flag that selects the Web Connectivity v0.5
experiment for some users.
*## OONI Probe Web*
We worked towards finalizing the development of our new browser-based
censorship measurement tool: OONI Probe Web (https://probe-web.ooni.org/)
In working towards the first beta release, we added support for detecting
the browser being used by the user and displaying a warning text if their
browser configuration could lead to false positives (
https://github.com/ooni/probe-web/pull/5/files)
*## OONI Explorer*
Through January 2023, we worked on the following new OONI Explorer (
https://explorer.ooni.org/) features:
* Improving the domain-centric pages (
https://github.com/ooni/explorer/pull/830)
* Improving the UI of the network-centric pages (
https://github.com/ooni/explorer/issues/744,
https://github.com/ooni/explorer/pull/830)
* Finalizing the user feedback reporting mechanism (
https://github.com/ooni/explorer/issues/811)
* Integrating third-party data (IODA, Cloudflare Radar, Google traffic
data) for the monitoring of internet outages (
https://github.com/ooni/explorer/pull/831)
As part of this work, we worked closely with a designer who carried out UX
research with our community, helping to improve the design and usability of
the above new features and pages.
*## OONI backend*
Through January 2023, we continued to work on a series of backend
improvements.
Specifically, we:
* Added a feature flag for the Web Connectivity v0.5 experiment (
https://github.com/ooni/api/pull/315)
* Added support for returning the test helper addresses as part of the
check-in API (https://github.com/ooni/api/pull/316)
* Merged the initial VPN observation table and extraction into the fastpath
pipeline (https://github.com/ooni/pipeline/pull/399)
* Removed the Tor and Psiphon configurations from the check-in API (
https://github.com/ooni/api/pull/317)
* Wrote an internal design document for the check-in API authentication;
* Wrote an internal design document for our database backup strategy.
*## Automating censorship detection and characterization based on OONI
measurements*
We made progress on the characterization of website censorship through our
OONI data analysis tool (https://github.com/ooni/data/pull/23)
Specifically, we:
* Improved the DNS analysis;
* Made some tweaks to the TCP and NXDOMAIN scoring;
* Improved the TLS analysis to make use of HTTP measurements;
* Improved the code quality.
Moreover, we started implementing a set of data visualizations to be
displayed from the OONI data web interface by integrating the vega
generated charts into react. This lays the foundation for eventually
integrating these charts into OONI Explorer (
https://github.com/ooni/data/pull/25)
*## Creating a Social Media Censorship Alert System*
We continued to make progress on the initial event detector developed for
the new Social Media Censorship Alert System (
https://github.com/ooni/backend/issues/629) As part of this, we monitored
our internal dashboard, ran experiments using the Pandas library, and
created a CSV dump on S3 for reproducible testing. Based on the
experiments, we made performance improvements and we implemented various
changes to improve how cases of potential censorship are detected.
*## Test list updates*
We continued to work towards improving the test lists. Specifically, we
updated the test lists for Tajikistan (
https://github.com/citizenlab/test-lists/pull/1211) Nigeria (
https://github.com/citizenlab/test-lists/pull/1208) and Russia (
https://github.com/citizenlab/test-lists/pull/1206,
https://github.com/citizenlab/test-lists/pull/1207)
*## OONI reports for the ISOC Pulse project on Internet shutdowns*
As a data partner for the ISOC Pulse project on Internet shutdowns, we
contributed OONI data (along with relevant charts and information) for the
“service blocking” events listed on the ISOC Pulse shutdowns timeline:
https://pulse.internetsociety.org/shutdowns
Specifically, we created OONI reports (including OONI data, charts, and
relevant interpretation of the findings) for many of the “service blocking”
entries listed in the ISOC Pulse shutdown timeline. These reports will be
added under the “Data and Analysis” section of relevant ISOC Pulse shutdown
entries.
Throughout January 2023, we wrote OONI reports for the “service blocking”
entries (where relevant OONI data exists) from 2018, 2019, 2021, and 2022
on the ISOC Pulse shutdown timeline.
*## OONI data analysis*
Throughout January 2023, we provided OONI data analysis support to
community members, and we carried out various data analysis tasks in
support of our own research. This includes a year-long analysis of OONI
data collected from Russia (https://github.com/ooni/ooni.org/issues/1331)
as well as analysis of OONI data collected from Venezuela (
https://github.com/ooni/ooni.org/issues/1310)
*## Creating a new OONI Outreach Kit*
In January 2023, we finalized all of the materials for the new OONI
Outreach Kit.
Specifically, we created and finalized the content for the following:
* 3 versions of an (1-page) OONI Probe Fact Sheet
* Long version (2-pages) of an OONI Probe Fact Sheet
* OONI Probe Fact Sheet for Elections
* Document with OONI Probe testing instructions
* 2 versions of an (1-page) OONI Explorer Fact Sheet
* OONI Explorer Fact Sheet for Researchers
* OONI Explorer Fact Sheet for Human Rights Advocates
* OONI Explorer Fact Sheet for Journalists
* Internet Censorship Fact Sheet
* Fact Sheet on How Websites are Blocked
* 2 versions of OONI Brochures
* OONI Resources document
Upon finalizing the Outreach Kit content, we contracted Ura Design (
https://ura.design/) for the design of the materials.
*## Coordination of translation of OONI documents*
To enable more communities around the world to learn about OONI, we aim to
have the following OONI documents translated:
* OONI Frequently Asked Questions (FAQ): https://ooni.org/support/faq
* OONI Glossary: https://ooni.org/support/glossary
* Potential risks associated with censorship measurement:
https://ooni.org/about/risks/
To this end, we contracted translators for the translation of the above
documents into the following 5 languages: Arabic, Farsi, Russia, Swahili,
and Spanish.
We prioritized these 5 languages because they are spoken in many countries
that experience high levels of internet censorship. We worked with
translators who were already active in the OONI community and therefore
more familiar with OONI’s terminologies, tools, and goals.
In preparation for these translations, we wrote relevant translation
guidelines, which we shared with each of the translators. We subsequently
coordinated with the translators.
*## Updated the OONI FAQ*
On our website, we have an OONI Frequently Asked Questions (FAQ) section
which aims to address the questions that we are frequently asked by the
community (https://ooni.org/support/faq/)
In January 2023, we updated the OONI FAQ page to reflect changes in our
tools and provide up-to-date information and links (
https://github.com/ooni/ooni.org/pull/1334) – particularly in preparation
for the translation of this content.
*## Community use of OONI data### Research report on VPN blocking in Uganda*
Christopher Kalema (OPTIMA research fellow) published a research report
investigating VPN blocking in Uganda and its impact on the country’s
preparations for the 2021 general elections. This study made use of OONI
data to investigate VPN blocking in Uganda.
The research report is available here:
https://internews.org/wp-content/uploads/2022/08/VPN-Blocking-During-the-20…
*### Pakistan started blocking Deutsche Welle (DW)*
On 16th January 2023, OONI data shows that Pakistan started blocking access
to Deutsche Welle (DW):
https://explorer.ooni.org/chart/mat?probe_cc=PK&test_name=web_connectivity&…
This was reported by Oliver Linow (Internet Freedom Specialist at DW), who
shared OONI data on the blocking of DW:
https://twitter.com/OliverLinow/status/1619282417629675522
*## Community activities*
*### SFLC.in panel for the launch of a new report on website blocks in
India*
On 12th January 2023, OONI’s Maria participated as a panelist on SFLC.in’s
panel “Error 404 Website Cannot Be Found”, which launched their new
research report on website blocking in India (
https://sflc.in/finding-404-report-website-blocking-india/)
Information about the event can be found here:
https://sflc.in/error-404-website-cannot-be-found
The panel discussion was live-streamed here:
https://www.youtube.com/watch?v=aB5hlX6dSoI
*### OTF Summit*
Between 24th-26th January 2023, OONI’s Arturo traveled to Texas to
participate in the OTF Summit. As part of his participation, Arturo
co-facilitated a session (in collaboration with Netalitica’s Igor
Valentovitch) on investigating information controls through the use of OONI
tools and the Citizen Lab’s test lists.
*### Roskomsvoboda’s Privacy Day 2023 event*
On 27th January 2023, OONI’s Maria participated as a speaker in
Roskomsvoboda’s Privacy Day 2023 event (https://2023.privacyday.net/) as
part of which she discussed how people in Russia can participate in OONI
censorship measurement.
The event was live-streamed here:
https://www.youtube.com/watch?v=zUPYMW_tsUo
*### OONI Community Meeting*
On 31st January 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we discussed the
following topics:
1. Updates from the OONI team.
2. Updating the OONI Probe test for RiseupVPN.
3. OONI workshops and participation in events in the Global South in 2023.
*## Measurement coverage*
In January 2023, 60,858,923 OONI Probe measurements were collected from
2,899 AS networks in 168 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
