Hello,
This email shares OONI's monthly report for February 2023.
*# OONI Monthly Report: February 2023*
Throughout February 2023, the OONI team worked on the following sprints:
* Sprint 84 (1st-12th February 2023)
* Sprint 85 (13th-28th February 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Published research report on internet censorship in Russia*
On 24th February 2023 (1 year since the start of the war in Ukraine), we
co-published a new research report, in collaboration with Roskomsvoboda,
which documents how Internet censorship in Russia changed over the last
year based on OONI data analysis.
Read our report in:
* English: https://ooni.org/post/2023-russia-a-year-after-the-conflict/
* Russian: https://ooni.org/ru/post/2023-russia-a-year-after-the-conflict/
You can also read Roskomsvoboda's publication about the report here:
https://roskomsvoboda.org/post/rks-ooni-odin-god/
Overall, OONI data analysis shows the blocking of 494 domains in Russia.
Many more websites are also blocked, beyond those tested. The blocked
domains fall under 28 categories, suggesting pervasive levels of internet
censorship in Russia.
New blocks that emerged in Russia over the last year include:
* Blocking of Russian & international human rights websites (e.g. Human
Rights Watch & Amnesty International)
* Blocking of investigative journalism & news media sites
* Blocking of Instagram, SoundCloud and Patreon
Access to the Tor Project's website was temporarily unblocked in Russia
between 15th to 28th July 2022. Several discontinued sites have been
unblocked over the last year (possibly as part of a "blocklist cleanup").
We confirmed the blocking of 48 domains (including a Let's Encrypt domain)
based on OONI data, which do not appear to be included in Russia's official
blocking registry. OONI data also suggests that access to a major CDN was
blocked in the past year.
*## Published research report on internet censorship in Azerbaijan*
On 28th February 2023, we co-published a new research report on internet
censorship in Azerbaijan, in collaboration with our partner, Azerbaijan
Internet Watch. As part of this research, we analyzed OONI data collected
from Azerbaijan over the last year.
The report was published by:
* OONI: https://ooni.org/post/2023-azerbaijan-internet-censorship/
* Azerbaijan Internet Watch:
https://www.az-netwatch.org/news/ooni-measurements-show-ongoing-internet-ce…
As part of our analysis of OONI measurements collected from Azerbaijan
between January 2022 to February 2023, we found:
** Blocking of news media websites. *Azerbaijan continues to block access
to several independent news media websites. OONI data also suggests that
some ISPs in Azerbaijan may have started blocking access to the Guardian on
25th December 2022.
** Azerbaijan and Russia block each other’s news media. *In early June
2022, Azerbaijan started blocking access to Russia’s state-run RIA Novosti
media website. Since (at least) 7th June 2022, Russian ISPs have been
blocking access to Azerbaijani news media websites (haqqin.az, minval.az,
oxu.az, ru.oxu.az, ru.baku.ws). These blocks remain ongoing.
** Temporary blocking of TikTok amid border clashes with Armenia. *During
the September 2022 border clashes, both Azerbaijan and Armenia blocked
access to TikTok. While the TikTok block was lifted in Armenia by 21st
September 2022 (only lasting about a week), the TikTok block remained in
place in Azerbaijan until November 2022 (lasting about 2 months).
** Blocking of circumvention tool sites.* Azerbaijan continues to block
access to numerous circumvention tool websites, potentially limiting the
ability to circumvent internet censorship in Azerbaijan. However, most OONI
measurements suggest that tested circumvention tools (Tor and Psiphon)
appear to be reachable.
** Variance of censorship across networks. *While most ISPs in Azerbaijan
appear to adopt similar censorship techniques (as we continued to observe
connection timeouts in most anomalous measurements across ASNs), different
ISPs block access to different websites over time.
*## Published report on social media blocks in Ethiopia*
On 15th February 2023, we published a report on the blocking of social
media platforms in Ethiopia.
OONI data from Ethiopia showed the ongoing blocking of:
* Facebook and Telegram (since 9th February 2023)
* YouTube (since 10th February 2023)
Read the report here:
https://ooni.org/post/2023-ethiopia-blocks-social-media/
*## Published report on DW and Wikipedia blocking in Pakistan*
On 15th February 2023, we published a report on the blocking of Wikipedia
and Deutsche Welle (DW) in Pakistan.
OONI data from Pakistan showed:
* Access to Wikipedia restricted between 1st-6th February 2023 (48-hour
degradation, followed by block)
* Ongoing blocking of Deutsche Welle (DW) since (at least) 16th January 2023
Read the report here:
https://ooni.org/post/2023-pakistan-blocks-wikipedia-and-dw/
*## Published report on Twitter throttling in Turkey*
On 15th February 2023, we published a report on the throttling and DNS
blocking of Twitter in Turkey in the aftermath of a deadly earthquake.
On 8th February 2023, OONI data from Turkey showed that access to Twitter
was throttled on at least 4 networks, and blocked by means of DNS
interference on other networks.
Read the report here:
https://ooni.org/post/2023-turkey-throttling-blocking-twitter/
*## OONI reports for the ISOC Pulse project on Internet shutdowns*
As a data partner for ISOC’s Pulse Shutdown project (
https://pulse.internetsociety.org/shutdowns), we have been working on
creating “OONI reports” (with relevant OONI data, interpretation, and
charts) for all of the “service blocking” entries listed on the Pulse
shutdown timeline (ranging from March 2018 to February 2023) over the last
year.
In February 2023, we completed this work, having written the “Data and
analysis” section for all “service blocking” entries of the ISOC Pulse
Shutdown timeline (where relevant OONI data exists). As part of this work,
we found that OONI data is available for almost all blocking events
reported worldwide over the last 5 years (at least based on those listed on
the ISOC Pulse shutdown timeline).
In total, we wrote 49 OONI reports for the ISOC Pulse shutdown timeline.
These reports will be included in the “Data and analysis” section of
“service blocking” entries.
*## OONI Probe Mobile*
In February 2023, we continued to improve upon the OONI Probe mobile app.
Specifically, we:
* Prepared the next OONI Probe Mobile release (3.8.0) for internal testing;
* Fixed a bug ensuring that the network name is displayed along with the
ASN (https://github.com/ooni/probe/issues/2371);
* Worked towards addressing a bug affecting automated testing on OONI Probe
iOS (https://github.com/ooni/probe/issues/2259);
* Worked towards enabling the transfer of the OONI Probe Android app to a
SD card (https://github.com/ooni/probe/issues/2019);
* Worked towards ensuring that the OONI Probe iOS app does not crash when
OONI backend services are down (https://github.com/ooni/probe/issues/2201).
*## OONI Run*
OONI Run (https://run.ooni.io/) can be translated via the Transifex (
https://explore.transifex.com/otf/ooni-run/) platform.
In February 2023, thanks to the Localization Lab community, OONI Run was
translated into 6 languages: Chinese, German, Thai, Russian, Spanish, and
Turkish.
*## OONI Probe CLI*
In February 2023, we released OONI Probe CLI 3.17.0:
https://github.com/ooni/probe-cli/releases/tag/v3.17.0
Notably, OONI Probe CLI 3.17.0 includes an updated version (v0.5.20) of the
Web Connectivity experiment, a TLS middlebox experiment, backend proxy
support for Tor Snowflake, OONI Run improvements, and many other
improvements (https://github.com/ooni/probe-cli/releases/tag/v3.17.0).
This new release will be the first one to support providing richer input to
experiments. We now support dynamically instructing the probes to enable
experimental Web Connectivity functionality through the OONI backend.
As part of this release cycle, we investigated two issues related to
running tor on Android devices. The first issue (
https://gitlab.torproject.org/tpo/core/tor/-/issues/40747,
https://github.com/ooni/probe/issues/2405) deals with the OONI Probe
Android app receiving a SIGABRT signal because tor closes a file descriptor
twice, thus triggering Android’s fdsan (
https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md).
We patched the problem as part of the OONI Probe CLI 3.17.0 release. The
second issue deals with a mysterious SIGABRT crash that we are still
investigating.
While testing OONI Probe CLI 3.17.0, we also noticed and investigated
excessive CPU usage in the Web Connectivity test helper (
https://github.com/ooni/probe/issues/2413). We modified the codebase to add
support for profiling, started fixing the most obvious issues, and
increased the number of test helper hosts to spread the load among them.
As part of testing, we also triaged the following data quality issues:
https://github.com/ooni/probe/issues/2410https://github.com/ooni/probe/issues/2411https://github.com/ooni/probe/issues/1925#issuecomment-1429373491https://github.com/ooni/probe/issues/2412https://github.com/ooni/probe/issues/2420https://github.com/ooni/probe/issues/2421
We fixed some of these data quality issues, as documented below:
https://github.com/ooni/probe-cli/pull/1110https://github.com/ooni/probe-cli/pull/1111https://github.com/ooni/probe/issues/2293
Where feasible, we backported to the OONI Probe CLI 3.17.0 branch.
After M-Lab deprecated their locatev1 API, we also migrated the DASH
experiment to use the locatev2 API and included this patch into the 3.17.0
branch (https://github.com/ooni/probe/issues/2398).
We also started working on exposing the OONI engine API, which is part of
the probe-cli repository, as a dynamic library (
https://github.com/ooni/probe/issues/2414).
*## Expanding OONI’s testing model to support richer testing input*
The OONI Probe CLI 3.17.0 release (
https://github.com/ooni/probe-cli/releases/tag/v3.17.0) starts adding
support for richer testing input (
https://github.com/ooni/ooni.org/issues/1291). We also continued to
improve such support in the main development branch of OONI Probe CLI and
OONI API. Specifically, we modified the API to directly return test helper
information inside the check-in API response (
https://github.com/ooni/probe/issues/2392).
We also started working on a design document regarding the richer testing
input functionality and we began working on the related PoC (
https://github.com/ooni/probe-cli/pull/1075).
*## Creating a throttling measurement methodology*
As part of our plans to create a methodology for measuring throttling (
https://github.com/ooni/ooni.org/issues/1296), we started sketching out a
possible integration testing strategy.
At the end of this reporting period, we wrote a diff for probe-cli (
https://gist.github.com/bassosimone/3ce52a37fc7394f7cce82391685c5477) which
works as follows:
* We hijack the core networking code to use a GVisor-based (
https://gvisor.dev/) TCP/IP stack in userspace;
* We connect this stack, representing the OONI Probe client with other
GVisor stacks representing servers;
* We add latency and throttling on those virtual channels when the network
traffic meets specified conditions (e.g., depending on the SNI used during
the TLS handshake).
Writing good integration tests for this upcoming functionality is crucial
to ensure that we do not break the Web Connectivity experiment while
applying the necessary changes to add support for collecting network
performance metrics (as planned:
https://github.com/ooni/ooni.org/issues/1297).
*## OONI Explorer*
We continued to work with a designer on the new OONI Explorer domain and
network centric pages (https://github.com/ooni/explorer/pull/830). We
updated the copy for the user feedback reporting system and we worked on
end-to-end tests. We made improvements to the third-party data integration,
notably adding support for the caching of proxied Cloudflare requests and
other UI changes (https://github.com/ooni/explorer/pull/831). We also
released a new MAT filter that allows showing data for different time
intervals: hourly, daily, weekly and monthly.
*## OONI backend*
Through February 2023, we continued to work on a series of backend
improvements.
Specifically, we:
* Implemented datacenter filtering in rotation (
https://github.com/ooni/pipeline/pull/408);
* Scaled up our infrastructure to 4 test helpers (
https://github.com/ooni/api/pull/320) and correlated the test runtime with
the deployment of the new test helpers to measure their effectiveness;
* Added more networks to the Web Connectivity v0.5 feature flag (
https://github.com/ooni/api/pull/318);
* Fixed a bug affecting authentication in the Test Lists Editor (
https://github.com/ooni/test-lists-ui/issues/71);
* Fixed the pipeline CI permissions (
https://github.com/ooni/pipeline/pull/409);
* Extracted more values from measurements including software and test
version, engine details, CPU architecture and test run time (
https://github.com/ooni/pipeline/pull/411,
https://github.com/ooni/pipeline/pull/410);
* Implemented an initial database backup tool.
*## Automating censorship detection and characterization based on OONI
measurements*
We made stable progress on the OONI data analysis tool (
https://github.com/ooni/data). Specifically, we:
* Made several improvements to the tooling that can be used for creating
plots that allow us to assess how well the analysis performs (
https://github.com/ooni/data/pull/25);
* Added support for mapping several “unknown_failures” that are a result of
software bugs, ensuring that the data is more clean and can be better
analyzed (
https://github.com/ooni/data/pull/25/commits/dc6517e0148be6c571a9e0768b50b5…
):
* Started exploring the possibility of performing the analysis directly
inside of the database engine (as opposed to doing in python code). This
resulted in a 10x performance increase, allowing us to analyze one day's
worth of OONI data in just over 5 minutes (compared to 2 hours of the
python implementation).
We also ingested new blocking fingerprints (
https://github.com/ooni/pipeline/pull/407) into the fastpath data
processing pipeline, thereby automatically detecting and confirming more
cases of website blocking around the world.
To evaluate how effectively OONI Probe measures and detects cases of
internet censorship, we compared measurements from Indonesia (that show
blocking) with Indonesia’s (“DNS-Trust”) blocking registry (
https://github.com/alsyundawy/dnstrust-apjii). We found that OONI
measurements that show DNS-based censorship are consistent with Indonesia’s
blocklist.
*## Creating a Social Media Censorship Alert System*
We continued to make progress on the initial event detector developed for
the new Social Media Censorship Alert System (
https://github.com/ooni/backend/issues/629). Specifically, we ran
experiments using Pandas, added more tests, and generated PNG/SVGs as part
of the testing.
*## Creating a new OONI Outreach Kit*
Throughout February 2023, we coordinated with Ura Design (
https://ura.design/), who worked on designing the new OONI Outreach Kit
materials. We are aiming to publish the Outreach Kit in March 2023.
*## Coordination of translation of OONI documents*
Throughout February 2023, we coordinated with translators with regards to
the translation of several OONI documents into 5 languages: Arabic, Farsi,
Russia, Swahili, and Spanish. We are aiming to publish the translated
documents in March 2023.
*## Test list updates*
We coordinated with our partner, TEDIC (https://ooni.org/partners/tedic/),
who provided extensive updates for the test list of Paraguay:
https://github.com/citizenlab/test-lists/pull/1224
The test list for Cambodia was also updated extensively by API Cambodia
(iMAP partner): https://github.com/citizenlab/test-lists/pull/1214
We reviewed and merged many other test list pull requests contributed by
community members:
https://github.com/citizenlab/test-lists/pulls?q=is%3Apr+is%3Aclosed
*## Rapid response### Wikipedia blocking in Pakistan*
Access to Wikipedia was temporarily restricted in Pakistan between 1st to
6th February 2023.
We rapidly responded by sharing relevant OONI data collected from Pakistan
(and encouraging further testing) on Twitter:
https://twitter.com/OpenObservatory/status/1620845976645963778 and
https://twitter.com/OpenObservatory/status/1621449233852252160
We subsequently published a report:
https://ooni.org/post/2023-pakistan-blocks-wikipedia-and-dw/
*### Social media blocks in Iraq*
On 6th February 2023, community members in Iraq reported to the #KeepItOn
mailing list that access to social media platforms (WhatsApp, Facebook,
Twitter, Instagram) would be banned during the country’s final exams,
starting on 5th February 2023 and ending on 13th February 2023 (from 4 am
to 12 pm local time). The decision was made by the PMO based on the
ministry of education recommendation.
We shared relevant OONI data with the #KeepItOn campaign on these blocks.
*### Twitter throttling in Turkey*
On 8th February 2023, in the aftermath of the earthquakes, Turkey throttled
access to Twitter.
Real-time OONI data collected from Turkey showed the block:
https://explorer.ooni.org/chart/mat?probe_cc=TR&test_name=web_connectivity&…
We rapidly responded by analyzing relevant OONI data collected from Turkey
and sharing our analysis through a Twitter thread:
https://twitter.com/OpenObservatory/status/1623398643595182083
OONI data showed that access to Twitter was throttled on at least 4
networks, and blocked by means of DNS on other networks.
We subsequently published a report:
https://ooni.org/post/2023-turkey-throttling-blocking-twitter/
*### Social media blocks in Ethiopia*
On 9th February 2023, amid church split tensions and calls for
anti-government protests, access to social media platforms was reportedly
blocked in Ethiopia.
We rapidly responded by sharing relevant OONI data collected from Ethiopia
(and encouraging further testing) on Twitter:
https://twitter.com/OpenObservatory/status/1624045402679873540 and
https://twitter.com/OpenObservatory/status/1624761465453379584
We subsequently published a report:
https://ooni.org/post/2023-ethiopia-blocks-social-media/
*## Community use of OONI data### Paper on internet censorship in Russia*
Censored Planet (in collaboration with other researchers) published a paper
(“Network Responses to Russia’s Invasion of Ukraine in 2022: A Cautionary
Tale for Internet Freedom”) which analyzes OONI data (along with Censored
Planet data) to examine internet censorship in Russia following the
February 2022 invasion of Ukraine.
Their paper (which was accepted to appear in USENIX Security 2023) is
available here:
https://censoredplanet.org/assets/russia-ukraine-invasion.pdf
*### Paper on censorship data analysis*
Censored Planet and Jigsaw published a paper (“Advancing the Art of
Censorship Data Analysis”) which discusses the challenges involved in
analyzing censorship measurement data. This paper shares examples from OONI
data and other public censorship datasets (such as Censored Planet).
Their paper (which was published by FOCI) is available here:
https://www.petsymposium.org/foci/2023/foci-2023-0003.pdf
*### Data Journalism guide for measuring internet shutdowns*
Journalist Sabrina Faramarzi published an article (“Kill switch: reporting
on and during internet shutdowns”) which aims to serve as a guide for data
journalists interested in measuring internet shutdowns. This article
features an interview with OONI’s Maria, where she discusses why
journalists should use OONI data to investigate internet censorship around
the world.
The article is available here:
https://datajournalism.com/read/longreads/internet-shutdowns-data-reporting
*## Community activities### OONI workshop for civil society in Azerbaijan*
On 1st February 2023, OONI’s Elizaveta facilitated an OONI workshop for
civil society groups in Azerbaijan. This workshop was hosted in
collaboration with our partner, Azerbaijan Internet Watch (
https://ooni.org/partners/azerbaijan-internet-watch/). The goal of the
workshop was to introduce civil society groups in Azerbaijan to OONI tools
to help boost OONI measurement coverage in Azerbaijan.
*### FOSDEM 2023*
On 4th and 5th February 2023, OONI’s Federico attended FOSDEM (
https://fosdem.org/2023/) in Brussels.
*### OONI workshop for human rights defenders in Nigeria*
On 16th February 2023, OONI’s Elizaveta facilitated an OONI workshop for
human rights defenders in Nigeria. This workshop was hosted in
collaboration with our partner, Paradigm Initiative (
https://ooni.org/partners/paradigm-initiative/). The goal of the workshop
was to introduce participants to OONI tools and data, encouraging
censorship measurement leading up to and during Nigeria’s 2023 elections
(which may trigger new censorship events).
*### OONI Run demo for the Localization Lab community*
On 22nd February 2023, OONI’s Elizaveta provided an OONI Run (
https://run.ooni.io/) demo for the Localization Lab community to encourage
the translation of the platform.
*### Live OONI session for the OPTIMA community in Senegal*
On 23rd February 2023, OONI’s Elizaveta briefly presented OONI and
addressed questions as part of Internews’ live OPTIMA session for community
members in Senegal.
*### OONI Community Meeting*
On 28th February 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:
1. Updates from the OONI team.
2. Upcoming conferences and opportunities for censorship measurement
sessions.
3. Community feedback on early beta testing of OONI tools.
*## Measurement coverage*
In February 2023, 57,841,940 OONI Probe measurements were collected from
2,844 AS networks in 164 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hello,
This email shares OONI's monthly report for January 2023.
*# OONI Monthly Report: January 2023*
Throughout January 2023, the OONI team worked on the following sprints:
* Sprint 82 (2nd-15th January 2023)
* Sprint 83 (16th-29th January 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## OONI Probe Mobile*
We added support to OONI Probe Mobile for the re-testing of URLs that
presented anomalies (https://github.com/ooni/probe-android/pull/554). This
new feature enables OONI Probe users to more easily re-test URLs that
present signs of blocking, thereby contributing more measurements that
potentially contain evidence of internet censorship.
We also made improvements to RTL support for both OONI Probe Mobile and
Desktop (https://github.com/ooni/probe/issues/2346), we removed the
RiseupVPN test option from the app settings (
https://github.com/ooni/probe/issues/2204), we improved upon how tests are
run on iOS (https://github.com/ooni/probe/issues/2175), and we continued
working towards rewriting OONI Probe Mobile in Flutter (
https://github.com/ooni/probe/issues/2359).
*## OONI Run*
We worked towards ensuring that OONI Run links continue performing tests on
iOS, even when the device is locked (
https://github.com/ooni/probe/issues/2361).
Notably, we collaborated with the Localization Lab (
https://www.localizationlab.org/) on creating a new localization project
for OONI Run on Transifex (https://explore.transifex.com/otf/ooni-run/) and
we uploaded the relevant strings for translation. This enabled the
localization community to start translating the OONI Run platform (
https://run.ooni.io/), making it more accessible to communities worldwide.
*## Published OONI Run screencast*
To enable communities to use OONI Run (https://run.ooni.io/) to coordinate
website censorship testing, we created a screencast which shows how to use
the platform. In creating this screencast, we wrote a script (which appears
as subtitles, which can be translated) and we created a screen recording
illustrating how to use OONI Run.
We published the OONI Run screencast on our YouTube channel:
https://www.youtube.com/watch?v=OGRN7ve6cIA
*## OONI Probe CLI*
We carried out development and quality assurance work to prepare the OONI
Probe CLI 3.17.x release. To this end, we released OONI Probe ClI
v3.17.0-alpha (https://github.com/ooni/probe-cli/releases/tag/v3.17.0-alpha)
and OONI Probe CLI v3.17.0-alpha.1 (
https://github.com/ooni/probe-cli/releases/tag/v3.17.0-alpha.1). As part of
this work, we merged a fix that prevents the Web Connectivity v0.5
experiment from fetching data from the localhost (
https://github.com/ooni/probe-cli/pull/1027).
We also added support for performing A/B testing between Web Connectivity
v0.4 (the current stable release) and v0.5 (the new Web Connectivity
version). Once OONI Probe CLI 3.17.0 is released, we will start using this
mechanism to enable the Web Connectivity v0.5 experiment for some users and
compare the measurement results it collects with the ones collected by the
previous version.
This release also refactors the build system to verify we are building with
the expected compiler flags through unit tests. As part of this work, we
rewrote how we compile tor for Android to add support for patching and
hardening flags.
*## Expanding OONI’s testing model to support richer testing input*
Supporting a richer testing input model requires exposing the full check-in
API response to experiments. Such a response, in fact, potentially includes
configuration flags and other ancillary information that, along with the
input URL, provides richer input.
To move this objective forward, we fixed ooniprobe’s backend client to
observe the full check-in response, rather than just extracting the URLs (
https://github.com/ooni/probe-cli/pull/1037).
We then started using richer input when instantiating experiments by
implementing code to A/B test Web Connectivity implementations. This
feature inspects the check-in API response to extract feature flags (
https://github.com/ooni/probe-cli/pull/1039). We also extended the check-in
API itself to provide a feature flag that selects the Web Connectivity v0.5
experiment for some users.
*## OONI Probe Web*
We worked towards finalizing the development of our new browser-based
censorship measurement tool: OONI Probe Web (https://probe-web.ooni.org/).
In working towards the first beta release, we added support for detecting
the browser being used by the user and displaying a warning text if their
browser configuration could lead to false positives (
https://github.com/ooni/probe-web/pull/5/files).
*## OONI Explorer*
Through January 2023, we worked on the following new OONI Explorer (
https://explorer.ooni.org/) features:
* Improving the domain-centric pages (
https://github.com/ooni/explorer/pull/830);
* Improving the UI of the network-centric pages (
https://github.com/ooni/explorer/issues/744,
https://github.com/ooni/explorer/pull/830);
* Finalizing the user feedback reporting mechanism (
https://github.com/ooni/explorer/issues/811);
* Integrating third-party data (IODA, Cloudflare Radar, Google traffic
data) for the monitoring of internet outages (
https://github.com/ooni/explorer/pull/831).
As part of this work, we worked closely with a designer who carried out UX
research with our community, helping to improve the design and usability of
the above new features and pages.
*## OONI backend*
Through January 2023, we continued to work on a series of backend
improvements.
Specifically, we:
* Added a feature flag for the Web Connectivity v0.5 experiment (
https://github.com/ooni/api/pull/315);
* Added support for returning the test helper addresses as part of the
check-in API (https://github.com/ooni/api/pull/316);
* Merged the initial VPN observation table and extraction into the fastpath
pipeline (https://github.com/ooni/pipeline/pull/399);
* Removed the Tor and Psiphon configurations from the check-in API (
https://github.com/ooni/api/pull/317);
* Wrote an internal design document for the check-in API authentication;
* Wrote an internal design document for our database backup strategy.
*## Automating censorship detection and characterization based on OONI
measurements*
We made progress on the characterization of website censorship through our
OONI data analysis tool (https://github.com/ooni/data/pull/23).
Specifically, we:
* Improved the DNS analysis;
* Made some tweaks to the TCP and NXDOMAIN scoring;
* Improved the TLS analysis to make use of HTTP measurements;
* Improved the code quality.
Moreover, we started implementing a set of data visualizations to be
displayed from the OONI data web interface by integrating the vega
generated charts into react. This lays the foundation for eventually
integrating these charts into OONI Explorer (
https://github.com/ooni/data/pull/25).
*## Creating a Social Media Censorship Alert System*
We continued to make progress on the initial event detector developed for
the new Social Media Censorship Alert System (
https://github.com/ooni/backend/issues/629). As part of this, we monitored
our internal dashboard, ran experiments using the Pandas library, and
created a CSV dump on S3 for reproducible testing. Based on the
experiments, we made performance improvements and we implemented various
changes to improve how cases of potential censorship are detected.
*## Test list updates*
We continued to work towards improving the test lists. Specifically, we
updated the test lists for Tajikistan (
https://github.com/citizenlab/test-lists/pull/1211), Nigeria (
https://github.com/citizenlab/test-lists/pull/1208), and Russia (
https://github.com/citizenlab/test-lists/pull/1206,
https://github.com/citizenlab/test-lists/pull/1207).
*## OONI reports for the ISOC Pulse project on Internet shutdowns*
As a data partner for the ISOC Pulse project on Internet shutdowns, we
contributed OONI data (along with relevant charts and information) for the
“service blocking” events listed on the ISOC Pulse shutdowns timeline:
https://pulse.internetsociety.org/shutdowns
Specifically, we created OONI reports (including OONI data, charts, and
relevant interpretation of the findings) for many of the “service blocking”
entries listed in the ISOC Pulse shutdown timeline. These reports will be
added under the “Data and Analysis” section of relevant ISOC Pulse shutdown
entries.
Throughout January 2023, we wrote OONI reports for the “service blocking”
entries (where relevant OONI data exists) from 2018, 2019, 2021, and 2022
on the ISOC Pulse shutdown timeline.
*## OONI data analysis*
Throughout January 2023, we provided OONI data analysis support to
community members, and we carried out various data analysis tasks in
support of our own research. This includes a year-long analysis of OONI
data collected from Russia (https://github.com/ooni/ooni.org/issues/1331),
as well as analysis of OONI data collected from Venezuela (
https://github.com/ooni/ooni.org/issues/1310).
*## Creating a new OONI Outreach Kit*
In January 2023, we finalized all of the materials for the new OONI
Outreach Kit.
Specifically, we created and finalized the content for the following:
* 3 versions of an (1-page) OONI Probe Fact Sheet
* Long version (2-pages) of an OONI Probe Fact Sheet
* OONI Probe Fact Sheet for Elections
* Document with OONI Probe testing instructions
* 2 versions of an (1-page) OONI Explorer Fact Sheet
* OONI Explorer Fact Sheet for Researchers
* OONI Explorer Fact Sheet for Human Rights Advocates
* OONI Explorer Fact Sheet for Journalists
* Internet Censorship Fact Sheet
* Fact Sheet on How Websites are Blocked
* 2 versions of OONI Brochures
* OONI Resources document
Upon finalizing the Outreach Kit content, we contracted Ura Design (
https://ura.design/) for the design of the materials.
*## Coordination of translation of OONI documents*
To enable more communities around the world to learn about OONI, we aim to
have the following OONI documents translated:
* OONI Frequently Asked Questions (FAQ): https://ooni.org/support/faq
* OONI Glossary: https://ooni.org/support/glossary
* Potential risks associated with censorship measurement:
https://ooni.org/about/risks/
To this end, we contracted translators for the translation of the above
documents into the following 5 languages: Arabic, Farsi, Russia, Swahili,
and Spanish.
We prioritized these 5 languages because they are spoken in many countries
that experience high levels of internet censorship. We worked with
translators who were already active in the OONI community and therefore
more familiar with OONI’s terminologies, tools, and goals.
In preparation for these translations, we wrote relevant translation
guidelines, which we shared with each of the translators. We subsequently
coordinated with the translators.
*## Updated the OONI FAQ*
On our website, we have an OONI Frequently Asked Questions (FAQ) section
which aims to address the questions that we are frequently asked by the
community (https://ooni.org/support/faq/).
In January 2023, we updated the OONI FAQ page to reflect changes in our
tools and provide up-to-date information and links (
https://github.com/ooni/ooni.org/pull/1334) – particularly in preparation
for the translation of this content.
*## Community use of OONI data### Research report on VPN blocking in Uganda*
Christopher Kalema (OPTIMA research fellow) published a research report
investigating VPN blocking in Uganda and its impact on the country’s
preparations for the 2021 general elections. This study made use of OONI
data to investigate VPN blocking in Uganda.
The research report is available here:
https://internews.org/wp-content/uploads/2022/08/VPN-Blocking-During-the-20…
*### Pakistan started blocking Deutsche Welle (DW)*
On 16th January 2023, OONI data shows that Pakistan started blocking access
to Deutsche Welle (DW):
https://explorer.ooni.org/chart/mat?probe_cc=PK&test_name=web_connectivity&…
This was reported by Oliver Linow (Internet Freedom Specialist at DW), who
shared OONI data on the blocking of DW:
https://twitter.com/OliverLinow/status/1619282417629675522
*## Community activities*
*### SFLC.in panel for the launch of a new report on website blocks in
India*
On 12th January 2023, OONI’s Maria participated as a panelist on SFLC.in’s
panel “Error 404 Website Cannot Be Found”, which launched their new
research report on website blocking in India (
https://sflc.in/finding-404-report-website-blocking-india/).
Information about the event can be found here:
https://sflc.in/error-404-website-cannot-be-found
The panel discussion was live-streamed here:
https://www.youtube.com/watch?v=aB5hlX6dSoI
*### OTF Summit*
Between 24th-26th January 2023, OONI’s Arturo traveled to Texas to
participate in the OTF Summit. As part of his participation, Arturo
co-facilitated a session (in collaboration with Netalitica’s Igor
Valentovitch) on investigating information controls through the use of OONI
tools and the Citizen Lab’s test lists.
*### Roskomsvoboda’s Privacy Day 2023 event*
On 27th January 2023, OONI’s Maria participated as a speaker in
Roskomsvoboda’s Privacy Day 2023 event (https://2023.privacyday.net/), as
part of which she discussed how people in Russia can participate in OONI
censorship measurement.
The event was live-streamed here:
https://www.youtube.com/watch?v=zUPYMW_tsUo
*### OONI Community Meeting*
On 31st January 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:
1. Updates from the OONI team.
2. Updating the OONI Probe test for RiseupVPN.
3. OONI workshops and participation in events in the Global South in 2023.
*## Measurement coverage*
In January 2023, 60,858,923 OONI Probe measurements were collected from
2,899 AS networks in 168 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hi everyone,
Next Tuesday we will be hosting our monthly community meeting and we are welcoming everyone to join us there.
=> Where? OONI Slack channel: https://slack.ooni.org/ <https://slack.ooni.org/> (bridged with IRC: ircs://irc.oftc.net:6697/#ooni)
=> When? Tuesday, 28th March 2023 at 14:00 UTC (for 1 hour)
Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep <https://pad.riseup.net/p/ooni-community-meeting-keep>
The monthly OONI community meetings aim to:
* Collect community feedback on OONI tools & methodologies
* Address questions in relation to the use of OONI tools and OONI data
* Foster discussions on internet censorship issues
* Receive updates from the community
We hope you can join us!
All the best,
~ OONI team
Hello,
We're excited to share that today we published *new documentation which
explains how to interpret OONI data*.
You can access this documentation here:
https://ooni.org/support/interpreting-ooni-data/
OONI data is open data on Internet censorship around the world, updated in
real-time.
With our new guide, you can learn how to interpret OONI data to monitor and
respond to Internet censorship worldwide.
We hope you find this documentation useful!
Please feel encouraged to reach out with any questions or feedback you may
have.
Thanks,
Maria.
Hello,
Today the OONI team published a new guide through which you can learn *how
to coordinate OONI censorship measurement campaigns* with your community.
Read the guide here:
https://ooni.org/support/ooni-censorship-measurement-campaigns
Is your country preparing for elections? Did new blocks emerge during
protests? Would you like to measure censorship in different regions in your
country, or in different countries around the world?
Community coordination can help with rapidly measuring and responding to
Internet censorship events.
Through our new guide, you can learn which steps you can take to organize
censorship testing with other OONI Probe users.
We also share a few examples of OONI censorship measurement campaigns led
by groups in the internet freedom community.
We hope you find this documentation useful. Please feel encouraged to reach
out with any questions or feedback.
Thanks,
Maria.
Hello,
Today, the OONI team released OONI Probe Web: a new browser-based tool for
measuring the blocking of websites.
*Run OONI Probe Web*: https://probe-web.ooni.org/
We built OONI Probe Web in response to long-term community feedback,
requesting a censorship measurement tool that can be run from a browser,
without requiring the installation of any software.
With OONI Probe Web, you will test the same types of websites as with the
OONI Probe apps. You can find which websites you would test here:
https://ooni.org/support/faq#which-websites-will-i-test-for-censorship-with…
Upon running OONI Probe Web, your test results will be published as open
data in near real-time: https://ooni.org/data/
However, please note that *OONI Probe Web is very limited in comparison to
the OONI Probe apps* (https://ooni.org/install/).
Due to limitations in the API available to web browsers, it’s not possible
to run advanced experiments necessary for accurately detecting and
characterizing website blocks from a browser.
We therefore kindly ask you to not treat OONI Probe Web as a replacement
for the OONI Probe apps, but as a complementary tool.
We encourage you to run OONI Probe Web and contribute measurements, the
analysis of which will help us improve upon OONI Probe Web on an ongoing
basis.
You can learn more about OONI Probe Web in our blog post:
https://ooni.org/post/introducing-ooni-probe-web/
We welcome your feedback, and we're happy to address any questions.
Thanks,
Maria (on behalf of the OONI team).
Hello,
Today we are excited to share that we published a new *user guide for OONI
Explorer*: https://ooni.org/support/ooni-explorer/
OONI Explorer (https://explorer.ooni.org/) is the world's largest open
dataset on Internet censorship, with new measurements published in near
real-time.
This user guide provides *step-by-step instructions* (with screenshots) on
how to use OONI Explorer to investigate internet censorship worldwide.
*# What you will get from this guide*
Learn how to use OONI Explorer to:
* Discover *blocked websites and apps* around the world
* Access *measurement data* that can serve as evidence of internet
censorship
* Generate *charts* based on aggregate views of OONI data
* *Compare censorship* across countries and networks
* Filter measurements based on various parameters (such as country, ASN,
date range, OONI Probe test, websites categories, domains, etc.)
Upon reading this guide, we hope you will feel empowered to investigate
internet censorship through the use of OONI Explorer.
We thank all OONI Probe (https://ooni.org/install/) users worldwide who
have contributed – and continue to contribute – measurements, shedding
light on internet censorship.
If you have any questions or feedback, please let us know.
Thanks,
Maria (on behalf of the OONI team).