Hello,
The OONI team warmly welcomes you to join us next Tuesday for our monthly community meeting.
=> Where? OONI Slack channel: https://slack.ooni.org/ <https://slack.ooni.org/> (bridged with IRC:
ircs://irc.oftc.net:6697/#ooni)
=> When? Tuesday, 29th August 2023 at 14:00 UTC (for 1 hour)
Please add topics, questions and updates that you'd like to discuss during the meeting in this pad:
https://pad.riseup.net/p/ooni-community-meeting-keep <https://pad.riseup.net/p/ooni-community-meeting-keep>
The monthly OONI community meetings aim to:
* Collect community feedback on OONI tools & methodologies
* Address questions in relation to the use of OONI tools and OONI data
* Foster discussions on internet censorship issues
* Receive updates from the community
We hope you can join us!
All the best,
~ OONI team
Hello,
Today OONI, Cloudflare, Access Now and Laura Schwartz-Henderson published a
report documenting the social media blocks and network outages in Senegal
over the past 2 months.
You can read the report here:
https://ooni.org/post/2023-senegal-social-media-blocks/
*# Key Findings*
Amid political unrest in Senegal, OONI data shows:
* Blocking of WhatsApp, Facebook, Instagram, Twitter, and YouTube between
1st-7th June 2023
* Temporary blocking of TikTok between 5th-7th June 2023
* Ongoing blocking of Telegram Web (since 1st June 2023)
It remains unclear to us why ISPs in Senegal unblocked social media &
messaging apps (such as WhatsApp) by 8th June 2023, but continue blocking
Telegram Web -- on Sonatel (AS8346) -- to this date:
https://explorer.ooni.org/chart/mat?test_name=telegram&axis_x=measurement_s…
Data from Cloudflare Radar shows 3 disruptions to traffic from Tigo, and 2
disruptions at Sudatel Senegal between 3rd-6th June 2023. Cloudflare data
also shows traffic drops at Sudatel Senegal and Tigo on 31st July 2023.
Senegalese survey respondents who experienced the outages and blocks
described that they had an impact on the country's economic activity and
political stability, on their professional endeavours, and on the
vulnerable or marginalised groups in Senegal.
Learn more through our report:
https://ooni.org/post/2023-senegal-social-media-blocks/
Thanks,
Maria.
Hello,
Unfortunately, China started blocking OONI a few weeks ago.
Specifically, as of 7th July 2023, we have started to observe the blocking
of our website (ooni.org) and our censorship measurement app (OONI Probe)
in China. The block appears to be implemented by means of DNS injection and
TLS interference.
This is the first time that we're observing a large and persistent drop in
OONI measurement coverage as a result of OONI Probe blocking.
We just published a *report, documenting the blocking of OONI in China*
with OONI data: https://ooni.org/post/2023-china-blocks-ooni/
The report also includes advice for circumventing the blocking of OONI
Probe in China. However, running OONI Probe in China is now probably
riskier.
~ OONI team.
Hello,
Our next community meeting is already next Tuesday!
Please come and join us to share news and feedback, ask questions and discuss internet censorship.
=> Where? OONI Slack channel: https://slack.ooni.org/ <https://slack.ooni.org/> (bridged with IRC:
ircs://irc.oftc.net:6697/#ooni <ircs://irc.oftc.net:6697/#ooni>)
=> When? Tuesday, 25th July 2023 at 14:00 UTC (for 1 hour)
Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep <https://pad.riseup.net/p/ooni-community-meeting-keep>
The monthly OONI community meetings aim to:
* Collect community feedback on OONI tools & methodologies
* Address questions in relation to the use of OONI tools and OONI data
* Foster discussions on internet censorship issues
* Receive updates from the community
We hope you can join us!
All the best,
~ OONI team
Hi folks!
We are happy to announce that thanks to the Localization Lab community, OONI Run is now available in 9 languages:
Chinese Simplified
Chinese Traditional
Farsi
German
Portuguese
Russian
Spanish
Thai
Turkish
OONI Run (https://run.ooni.io/ <https://run.ooni.io/>) is a web platform that you can use to generate mobile deep links and widget code to coordinate OONI Probe website testing in your country or region. Please find a user guide for this tool here: https://ooni.org/support/ooni-run/ <https://ooni.org/support/ooni-run/>
OONI Run is specifically useful to coordinate measurement campaigns during elections or protests. We provide a short guide on how to coordinate such campaigns here: https://ooni.org/support/ooni-censorship-measurement-campaigns <https://ooni.org/support/ooni-censorship-measurement-campaigns>
Please encourage your partners and colleagues to use the localized OONI Run and let us know if you have any feedback or questions.
Best wishes,
~OONI Team
Hello,
This email shares OONI's monthly report for June 2023.
*# OONI Monthly Report: June 2023*
Throughout June 2023, the OONI team worked on the following sprints:
* Sprint 93 (5th - 18th June 2023)
* Sprint 94 (19th June - 2nd July 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Launch of OONI Explorer supporting localization*
We are thrilled to share that OONI Explorer (https://explorer.ooni.org/)
now supports localization! OONI Explorer is an open data resource on
internet censorship around the world.
As of June 2023, OONI Explorer is available in 8 new languages:
* Chinese: https://explorer.ooni.org/zh-cn
* Farsi: https://explorer.ooni.org/fa
* French: https://explorer.ooni.org/fr
* German: https://explorer.ooni.org/de
* Russian: https://explorer.ooni.org/ru
* Spanish: https://explorer.ooni.org/es
* Thai: https://explorer.ooni.org/th
* Turkish: https://explorer.ooni.org/tr
We thank the Localization Lab (https://www.localizationlab.org/) community
for making OONI Explorer – and real-time open data on internet censorship
worldwide – available in so many new languages!
*## OONI Probe Mobile*
We made some improvement to the code quality of OONI Probe Android by
making progress on de-duplicating the code used for running background
tests (https://github.com/ooni/probe-android/pull/572) and we made
improvements to some of the unit tests (
https://github.com/ooni/probe-android/pull/575) We also added the ability
to “select all”, “deselect all” category codes in the websites categories
to test settings (https://github.com/ooni/probe-android/pull/576)
*## OONI Probe user guides translated to German *
Thanks to a community member (Curtis Baltimore), the OONI Probe user guides
are now available in German!
The German user guides are available through the following links:
* OONI Probe Mobile User Guide:
https://ooni.org/de/support/ooni-probe-mobile
* OONI Probe Desktop User Guide:
https://ooni.org/de/support/ooni-probe-desktop
*## OONI Run*
We have started adding support to OONI Probe Android for parsing and
displaying OONI Run v2 descriptors (
https://github.com/ooni/probe-android/pull/577) We also made progress on
the web editor for OONI Run v2 links (https://github.com/ooni/run/pull/131)
and on the backend for adding support for storing and serving them, by
deploying it to the testing infrastructure (
https://github.com/ooni/backend/pull/678)
*## OONI Probe CLI*
We released OONI Probe CLI v3.17.4 (
https://github.com/ooni/probe-cli/releases/tag/v3.17.4) This release
updates to OpenSSL 1.1.1u (https://github.com/ooni/probe-cli/pull/1148)
includes a test helper fix to correctly measure Twitter (
https://github.com/ooni/probe/issues/2488) and updates the GeoIP database
to 2023-06 to address a Facebook Messenger issue (
https://github.com/ooni/probe/issues/2485) We also released OONI Probe
v3.17.5 (https://github.com/ooni/probe-cli/releases/tag/v3.17.5) which
updates to go1.19.10.
Moreover, we upgraded the main development branch to go1.20.5 (
https://github.com/ooni/probe/issues/2478) and we released OONI Probe CLI
v3.18.0-alpha (https://github.com/ooni/probe-cli/releases/tag/v3.18.0-alpha)
and v3.18.0 (https://github.com/ooni/probe-cli/releases/tag/v3.18.0)
In addition to the aforementioned patches that we also included in v3.17.x,
the v3.18.x series releases the following major features (please, refer to
the v3.18.0-alpha and v3.18.0 release notes for the complete changelog):
* dslx (
https://pkg.go.dev/github.com/ooni/probe-cli/v3@v3.18.0/internal/dslx) an
internal DSL for simplifying the process of writing and maintaining network
experiments;
* netemx (
https://pkg.go.dev/github.com/ooni/probe-cli/v3@v3.18.0/internal/netemx)
support for writing integration testing using Gvisor (a TCP/IP stack in
userspace that allows us to simulate censorship and throttling in a
predictable environment).
We started working on dslx in December 2022 and we started working on
netemx in March 2023. We are excited to see these features finally landing
in a stable release!
While working on the v3.18.x release, we also updated the underlying
ooni/netem library (https://github.com/ooni/netem) used by the netemx
feature to emulate DNS spoofing (https://github.com/ooni/netem/pull/27)
By releasing v3.18.0 (https://github.com/ooni/probe-cli/releases/tag/v3.18.0)
which uses go1.20.5, we also addressed a community raised issue (
https://github.com/ooni/probe/issues/2478) where it was impossible to
compile OONI Probe using the go1.20.x compiler series.
*## Expanding OONI’s testing model to support richer testing input*
We continued to work on supporting richer testing input (
https://github.com/ooni/ooni.org/issues/1291) We tagged version v0.1.0 (
https://github.com/ooni/2023-05-richer-input/releases/tag/v0.1.0) of our
ooni/2023-05-richer-input prototype (
https://github.com/ooni/2023-05-richer-input) This release saves the stage
of the repository at the end of May 2023. In this initial prototype, we
implemented richer input as a list of flat “mini nettests” that an existing
nettest (e.g., Facebook Messenger) should execute to obtain a complete
measurement of all the relevant targets.
While this design was adequate for an initial prototype, in June 2023 we
spent additional time trying to engineer and simplify this design. In
particular, we investigated whether we could reuse the DSL introduced in
ooni/probe-cli v3.18.0-alpha (
https://github.com/ooni/probe-cli/releases/tag/v3.18.0-alpha) to express
richer inputs in a more flexible and composable way (
https://github.com/ooni/probe/issues/2494) This culminated in releasing
v0.2.0 (https://github.com/ooni/2023-05-richer-input/releases/tag/v0.2.0)
of the prototype, which included a possible richer input implementation
that was using a DSL instead of flat nettest targets.
A DSL-based design is more flexible because we can naturally compose
existing building blocks (e.g., DNS lookups, TCP connect, TLS handshake)
with other blocks whose task is that of calculating a nettest’s top level
keys (e.g., the `blocking` key of Web Connectivity).
We introduced a complete DSL for richer input and adapted Facebook
Messenger to use such a DSL (
https://github.com/ooni/2023-05-richer-input/pull/5,
https://github.com/ooni/2023-05-richer-input/pull/6)
*## Creating a throttling measurement methodology*
Regarding measuring throttling (https://github.com/ooni/ooni.org/issues/1296)
we focused on adding (to the DSL introduced in ooni/probe-cli v3.18.0-alpha
(https://github.com/ooni/probe-cli/releases/tag/v3.18.0-alpha)) prototype
support for collecting extra statistics during long downloads to Web
Connectivity LTE (the current experimental version of Web Connectivity) and
to the ooni/2023-05-richer-input (
https://github.com/ooni/2023-05-richer-input) Currently, the underlying
framework that we use for these three pieces of code collects up to 64
network events during and after the TLS or the QUIC handshake. While these
events naturally include the initial download of the body, we wanted to
extend this model to collect download speed snapshots.
To this end, we imported memoryless from m-lab/go (
https://github.com/ooni/probe-cli/pull/1163) to have support for sampling
at exponentially distributed (and truncated) intervals, which provides
PASTA properties (https://en.wikipedia.org/wiki/Arrival_theorem) when
observing network events. We then used this functionality to implement
collecting download speed snapshots (
https://github.com/ooni/probe-cli/pull/1166) and we added initial support
for measuring the download speed (
https://github.com/ooni/2023-05-richer-input/pull/4)
*## Creating a Social Media Censorship Alert System*
In May 2023, we continued to make progress towards building a Social Media
Censorship Alert System. Specifically, we added support for the
reprocessing of existing measurements, and we are currently running the
event detector on the backend host. As part of improvements to the overall
event detector logic, we fixed a bug affecting how columns are handled in
Pandas.
This work is documented through the following pull request:
https://github.com/ooni/backend/pull/651
*## Creating a Censorship Incident Reporting Platform*
We’re excited to share that we’re building a Censorship Incident Reporting
Platform!
New censorship events frequently emerge worldwide on an ongoing basis, but
producing an in-depth report and analysis on each event (in a timely
manner) is not easily scalable. Moreover, rapid response efforts benefit
from censorship findings that are easier and quicker to consume.
We therefore aim to create a Censorship Incident Reporting Platform which
will regularly provide short reports with summary information on emergent
censorship events based on OONI data and charts. This will enable
journalists and human rights defenders to more easily learn about and
respond to censorship events. Over time, the platform will become an
archive of known censorship incidents from around the world based on OONI
network measurement data.
In June 2023, we made significant progress towards creating a Censorship
Incident Reporting Platform. Specifically, we developed the backend API for
creating, updating and retrieving censorship incidents through OONI
Explorer (https://github.com/ooni/backend/pull/678) This will allow logged
in OONI Explorer users to create incidents that can then be retrieved by
any user on the OONI Explorer platform, allowing us to develop views for
them. The backend API feature was deployed on testing and production
infrastructure, allowing us to start working on the frontend for displaying
and editing the censorship incidents.
On the frontend part, we started working on the editor (
https://github.com/ooni/explorer/pull/862) which supports an enriched
markdown format that allows us to embed MAT charts within markdown
formatted text. It also allows us to define some metadata which can be used
for retrieving incidents matching a specific search criterion (e.g. all
censorship incidents pertaining to a particular country).
*## OONI backend*
In June 2023, we implemented the extraction of test helper metadata (URL
and type) in the fastpath (https://github.com/ooni/backend/pull/682) which
will enable us to track which test helper was used in Web Connectivity
measurements and to create dashboards to detect correlation between test
helper locations and test failures. We updated the test helper rotation
tool to deploy Vector and configure log forwarding on test helpers (
https://github.com/ooni/backend/pull/692) which will allow collecting logs
from the test helpers centrally and storing them. We also implemented an
initial form of alarming based on the test helper failure rate.
We added support for GeoIP lookup for the probes (
https://github.com/ooni/backend/pull/678) and we implemented an
experimental tool and dashboard to monitor the accessibility of our website
(ooni.org) through Tor. We created an experimental STUN server to perform
initial connectivity tests from countries that block many other protocols.
*## Automating censorship detection and characterization based on OONI
measurements*
In June 2023, we added support for storing IP metadata related to control
measurements (e.g. AS and country information) to be used as part of the
DNS analysis (https://github.com/ooni/data/pull/30) We also added support
for making it easier to perform migrations of the database SQL schema,
allowing us to more quickly enrich the database table with more features (
https://github.com/ooni/data/pull/30/commits/7c9b6a5c68841c86e553c2d60d5fbf…
).
We also added support for correctly mapping unknown_failures (caused by
software bugs) to known failure strings, which allows us to consider more
kinds of measurements as part of the analysis (which would have otherwise
been considered as “failed”). This work is documented here:
https://github.com/ooni/data/pull/29
Finally, we added support for skipping the use of the dask parallel
processing engine, when no parallelism is needed, thereby increasing
performance (https://github.com/ooni/data/pull/31)
*## Interviewed Project Manager candidates*
In June 2023, we completed the first round of interviews for the OONI
Project Manager position (
https://ooni.org/post/2023-job-opening-ooni-project-manager/) Based on
these interviews, we created a shortlist for the second (and final) round
of interviews and coordinated with candidates. We also started the second
round of interviews with shortlisted candidates.
*## Blocking of social media in Senegal*
Following unrest erupted over the sentencing of opposition leader Ousmane
Sonko, OONI data from Senegal shows that ISPs started blocking access to
several social media platforms on 1st June 2023.
Specifically, OONI data shows the blocking of:
* WhatsApp:
https://explorer.ooni.org/chart/mat?probe_cc=SN&since=2023-05-03&until=2023…
* Telegram:
https://explorer.ooni.org/chart/mat?probe_cc=SN&since=2023-05-03&until=2023…
* Facebook, Instagram, YouTube, Twitter:
https://explorer.ooni.org/chart/mat?probe_cc=SN&since=2023-05-03&until=2023…
OONI data collected from Senegal shows that access to social media and
messaging platforms is blocked by means of TLS interference (i.e. TLS
handshakes timing out).
In response to the blocks, we shared relevant OONI data (and charts) on our
social media platforms (
https://twitter.com/OpenObservatory/status/1664611174854303745) with the
#KeepItOn advocacy community, and with Senegalese advocacy groups.
*## Blocking of F-Droid in China*
As of 22nd June 2023, OONI data collected from China shows the blocking of
F-Droid by means of DNS injection, where an unrelated IP address (e.g.
DropBox) is returned to the client (
https://explorer.ooni.org/m/20230622211826.539282_CN_webconnectivity_87bc82…)
This technique is commonly used in China to restrict access to services,
and OONI data shows that F-Droid is blocked on at least 4 networks in China.
OONI data on the blocking of F-Droid in China is available here:
https://explorer.ooni.org/chart/mat?probe_cc=CN&since=2023-06-13&until=2023…
In response to the block, we shared OONI data (and charts) in both English
and Chinese on our social media platforms (
https://twitter.com/OpenObservatory/status/1673659531639894019) We also
reached out to F-Droid developers to share relevant data that could
potentially help with circumvention.
*## Community use of OONI data### Civil society response to the social
media blocks in Senegal*
As soon as the social media blocks emerged in Senegal on 1st June 2023,
Senegalese civil society groups ran OONI Probe and independently reported
on the blocks, sharing OONI data (
https://twitter.com/aliamsi/status/1664374471849902080) Similarly, other
internet freedom community members, such as Deutsche Welle’s Oliver Linow,
responded to the blocks by sharing OONI data (
https://twitter.com/OliverLinow/status/1664397344111042565)
*### Access Now statement in response to social media blocks in Senegal*
In response to the social media blocks in Senegal, Access Now published a
statement (citing OONI data):
https://www.accessnow.org/press-release/keep-people-connected-senegal/
*## Community activities### RightsCon 2023*
Between 5th-8th June 20213, OONI’s Elizaveta, Maria, and Arturo traveled to
Costa Rica to attend RightsCon 2023 (https://www.rightscon.org/)
As part of their participation, they presented OONI as part of the
following 5 sessions:
1) Session: “KeepItOn in 2022: successes, opportunities, struggles, and
lessons learned in the fight against election-related shutdowns” (hosted by
Access Now and KICTANET), 6th June 2023, OONI speaker: Elizaveta Yachmeneva
2) Session: “Data and Action: Using data analysis for incident response and
threat prevention” (hosted by the Center for Digital Resilience), 7th June
2023, OONI speaker: Arturo Filastò
3) Workshop: “Learn to investigate internet shutdowns through an Iran 2022
case study” (hosted by OONI and IODA), 7th June 2023, OONI speaker: Maria
Xynou
4) Session: “Internet blocking and the consequences for human rights?”
(hosted by Cloudflare), 8th June 2023, OONI speaker: Arturo Filastò
5) Lightning talk: “Investigating Internet shutdowns through open data”
(hosted by OONI, IODA, M-Lab), 8th June 2023, OONI speaker: Maria Xynou
We also participated in a private #KeepItOn meeting to discuss how telcos
can support the fight against internet shutdowns.
Overall, participating at RightsCon provided a great opportunity to
reconnect with many of our partners and community members in person, and to
learn from many of the sessions that we attended!
*### OONI workshop for journalists and civil society in Sierra Leone*
On 21st June 2023, OONI’s Elizaveta facilitated an online OONI workshop for
journalists and civil society organizations in Sierra Leone as part of the
hybrid Stakeholders Dialogue on Internet Shutdowns event organized by
Access Now and our Sierra Leone partner, CHRDI (
https://ooni.org/partners/chrdi/) ahead of Sierra Leone’s 2023 elections.
Information about the event is available through the following links:
https://twitter.com/chrdiorg/status/1671661565068623877https://twitter.com/accessnow/status/1670860527541600256
The event (along with the OONI workshop) can be watched here:
https://www.youtube.com/watch?v=ho1wTgvIE-M
*### OONI webinar for civil society in Sierra Leone*
On 22nd June 2023, OONI’s Elizaveta presented OONI’s tools and methods to
civil society groups in Sierra Leone as part of Paradigm Initiative’s
“Defend, Connect, Report: Navigating Internet Censorship and Defending
Digital Rights in Sierra Leone’s Election” webinar in preparation for
Sierra Leone’s 2023 elections.
Information about the webinar is available here:
https://twitter.com/ParadigmHQ/status/1671091608840990722
*### OONI workshops at DSAConnectCon 2023*
Between 28th-29th June 2023, OONI’s Maria traveled to Johannesburg, South
Africa, to participate at DSAConnectCon, a 2-day community convening
organized by our partner, Digital Society of Africa (DSA). Learn about
DSA’s important work here: https://digitalsociety.africa/
As part of her participation, Maria co-facilitated 2 training sessions in
collaboration with IODA’s Amanda. These training sessions (“Investigating
Internet shutdowns through open data”) introduced participants to both OONI
and IODA tools, methods, and datasets, including hands-on exercises based
on real-world internet shutdown scenarios.
Maria also assisted participants with localizing OONI Probe into 7
additional African languages (Zulu, Ndebele, Nyanja, Swati, Mozambican
Portuguese, Tumbuka, Shona) during the Localization Lab’s (
https://www.localizationlab.org/) hands-on workshop on localizing internet
freedom tools.
*### OONI Community Meeting*
On 27th June 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we briefly discussed:
1) Social media blocks in Senegal.
2) F-Droid block in China.
3) Monitoring internet censorship in Sierra Leone during the country’s 2023
elections.
4) OONI Explorer being released in 8 new languages (encouraging further
localization efforts).
*## Measurement coverage*
In June 2023, 56,948,483 OONI Probe measurements were collected from 2,897
networks in 165 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hello,
This email shares OONI's monthly report for May 2023.
*# OONI Monthly Report: May 2023*
Throughout May 2023, the OONI team worked on the following sprints:
* Sprint 90 (24th April - 7th May 2023)
* Sprint 91 (8th - 21st May 2023)
* Sprint 92 (22nd May - 4th June 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## New partnership*
We are excited to have formed a new partnership with Annir Initiative!
The goal of the partnership is to collaborate on the study of internet
censorship in Libya. We published a dedicated partner page for Annir
Initiative highlighting their important work:
https://ooni.org/partners/annir/
*## Job Opening: OONI Project Manager*
On 2nd May 2023, we published a job opening for hiring an OONI Project
Manager!
The job posting is available here:
https://ooni.org/post/2023-job-opening-ooni-project-manager/
In an attempt to help ensure that our job opening reached interested
candidates around the world, we worked on relevant outreach efforts. In
particular, we posted the job opening on multiple platforms, including:
* All OONI social media channels (Twitter, Mastodon, Facebook, Instagram);
* All OONI mailing lists;
* Community mailing lists (such as those of the #KeepItOn campaign, Global
Voices Advocacy, and NetRights);
* OONI Slack channel (and other relevant Slack, Mattermost, and IRC
channels);
* Digital Rights Job Board (https://www.digitalrights.community/job-board)
* FOSS Job Board (
https://www.fossjobs.net/job/11234/ooni-project-manager-at-open-observatory…)
We also reached out to our partners and many other community members,
requesting that they help share this opportunity with their networks.
Following the publication of this job opening, we worked on tracking and
reviewing incoming applications on a rolling basis, and following-up with
applicants. In May 2023, we started the first round of interviews with
shortlisted candidates.
*## Blog post about new OONI Explorer features*
We published a blog post to share information about new OONI Explorer
features for investigating internet censorship through open data.
This blog post is available here:
https://ooni.org/post/2023-new-explorer-features/
Specifically, OONI Explorer includes the following new features:
* Domain-centric pages: https://explorer.ooni.org/domains
* Network-centric pages: https://explorer.ooni.org/networks
* Charts on internet outages (integrating IODA, Google traffic, and
Cloudflare Radar data), available in each country-specific (
https://explorer.ooni.org/countries) and network-specific (
https://explorer.ooni.org/networks) page
* Mechanism for verifying OONI measurements (available in each OONI
measurement page)
We hope these new features help with monitoring and responding to
censorship events worldwide.
*## OONI Probe Mobile*
We released OONI Probe Android 3.8.2 (
https://github.com/ooni/probe-android/releases/tag/v3.8.2) which includes
a new version of the engine and ships some of the previously developed bug
fixes.
We also worked towards addressing the following bugs:
* OONI Probe installation failure caused by duplicate permission (
https://github.com/ooni/probe/issues/2450)
* Android OONI Probe notification constantly wakes up the mobile screen (
https://github.com/ooni/probe-android/pull/570 ).
*## OONI Probe Desktop*
We released OONI Probe Desktop 3.9.0 (
https://github.com/ooni/probe-desktop/releases/tag/v3.9.0) which contains
the Signal test fix from OONI Probe CLI v3.17.2.
*## OONI Run*
We created a first iteration of the OONI Run v2 link web editor interface.
Specifically, we built the frontend component for the UI, which will enable
users to create and edit existing links (
https://github.com/ooni/run/pull/131)
We added support in the backend API for creating/updating the OONI Run v2
descriptor and serving them to clients (
https://github.com/ooni/backend/pull/678) We also discussed and reviewed
internal design documents for the OONI Run v2 check-in API.
*## OONI Probe CLI*
We released OONI Probe CLI v3.17.3 (
https://github.com/ooni/probe-cli/releases/tag/v3.17.3) This release
updates to go1.19.9, fixes issues with the Signal test, and updates the
bundled geoip database to 2023-05. We did not release this version because
we noticed that we needed to fix more bugs, such as upgrading OpenSSL (
https://github.com/ooni/probe-cli/commit/325a8412999b001f4913ab5c522ed6402c…)
and fixing a test helper issue (
https://github.com/ooni/probe-cli/commit/a3af5542986e314a4e766d36078bedbae5…)
We therefore chose to wait until 2023-06-05 to bundle the 2023-06 geoip
databases.
Following a community request (https://github.com/ooni/probe/issues/2478)
we started preparing for releasing version 3.18, which will include support
for go1.20. In this reporting period, we upgraded the code to use go1.20 (
https://github.com/ooni/probe-cli/pull/1144) and we started following the
usual pre-release check-list (https://github.com/ooni/probe/issues/2417)
*## Expanding OONI’s testing model to support richer testing input*
We continued to work on the richer input design document (
https://github.com/ooni/ooni.org/issues/1295) We open sourced a repository
that contains an initial design document (
https://github.com/ooni/2023-05-richer-input/blob/main/DESIGN.md) and
refactors OONI Probe around the concept of richer input. The code inside of
the repository shows how we can use the concept of richer input to simplify
the probe implementation. We explored, in particular, how we could use
richer input to streamline the implementation of the OONI Probe IM tests.
*## Creating a Social Media Censorship Alert System*
In May 2023, we continued to make progress towards building a Social Media
Censorship Alert System. Specifically, we improved our social media
blocking event detector by adding RSS/Atom feed generation. Additionally,
the event detector was updated to run hourly.
This work is documented through the following pull request:
https://github.com/ooni/backend/pull/651
*## OONI backend*
Throughout May 2023, we worked on the following backend activities:
* Annotated faulty Signal measurements as failed (
https://github.com/ooni/backend/pull/680)
* Monitored incoming measurements following an OONI Probe Android 3.8.0 bug
impacting unattended runs;
* Increased the testing priority for URLs in Timor Leste (amid the
country’s 2023 elections) and created internal dashboards to monitor the
testing coverage;
* Created an MVP for an internal dashboard for long-term measurement
traffic prediction and alarming;
* Fixed a broken dependency in the Altair library used on our Jupyter
Notebook instance;
* Fixed the Jupyter Notebook automatic runner (jupycron) to generate HTML
reports on hourly, daily and weekly intervals;
* Implemented tooling to measure the performance of the probe engine using
valgring / cachegrind (https://github.com/ooni/probe-cli/pull/1137)
* Continued working towards configuring Vector to centralize log management
(setting up SSL certificates);
* Configured the main backend host (backend-FSN) to send logs to the
monitoring host;
* Created initial internal dashboards for log monitoring.
*## Test lists updates*
In preparation for elections which could potentially trigger new censorship
events, we collaborated with our partners and other community members on
updating relevant test lists.
This involved updates to the test lists for the following countries:
* Turkey: https://github.com/citizenlab/test-lists/pull/1290,
https://github.com/citizenlab/test-lists/pull/1291
* Thailand: https://github.com/citizenlab/test-lists/pull/1289,
https://github.com/citizenlab/test-lists/pull/1280,
https://github.com/citizenlab/test-lists/pull/1282
* Timor Leste: https://github.com/citizenlab/test-lists/pull/1296
We thank community members for contributing to these test lists!
*## OONI profile for Mozilla 2023 Data Futures Lab Cohort*
OONI is thrilled to be part of Mozilla’s 2023 Data Futures Lab Cohort (
https://foundation.mozilla.org/en/blog/mozilla-welcomes-2023-data-futures-l…)
As part of this, we’re working towards releasing an improved version of
OONI Run that addresses key community feedback, supporting decentralized
censorship measurement campaigns around the world.
In May 2023, Mozilla published a blog post which provides a profile of OONI
(as a Mozilla Data Futures Lab awardee). This blog post includes quotes
from an interview with OONI’s Arturo, and can be accessed here:
https://foundation.mozilla.org/en/blog/crowdsourcing-data-as-a-tool-to-figh…
*## ClickHouse blog post*
ClickHouse published a blog post describing how we use their database
management system for OONI data:
https://clickhouse.com/blog/ooni-analyzes-internet-censorship-data-with-cli…
We migrated to ClickHouse a few years ago, and that has enabled us to
analyze and publish OONI measurements in real-time (seconds)!
*## Blocking of social media in Pakistan*
In response to the blocking of social media platforms in Pakistan, we
shared relevant OONI data with the #KeepItOn advocacy mailing list and on
social media platforms (
https://twitter.com/OpenObservatory/status/1656417835365588994,
https://twitter.com/OpenObservatory/status/1658489171902885897)
Specifically, starting from around 15:00 UTC on 9th May 2023, OONI data
collected from Pakistan showed that several ISPs started to restrict access
to Facebook, YouTube, and Twitter. You can find relevant OONI data (along
with a chart) here:
https://explorer.ooni.org/chart/mat?test_name=web_connectivity&axis_x=measu…
The blocking of YouTube is further suggested by Google's Transparency
Report, which shows a drop in YouTube traffic from Pakistan:
https://transparencyreport.google.com/traffic/overview?hl=en&fraction_traff…
Starting from around 05:00 UTC on 10th May 2023, OONI data collected from
Pakistan showed that several ISPs started to restrict access to Instagram
as well:
https://explorer.ooni.org/chart/mat?probe_cc=PK&since=2023-04-12&until=2023…
In most measurements (for the restricted social media platforms), OONI data
shows signs of TLS-level interference (as TLS handshakes time out), while
some measurements show DNS-based blocking as well (returning an NXDOMAIN
error). Notably, some providers implement both censorship techniques.
*## Blocking of social media in Guinea*
In response to the blocking of social media platforms in Guinea, we shared
relevant OONI data (
https://twitter.com/OpenObservatory/status/1660591036316499968)
Specifically, OONI data from Guinea suggests that some ISPs started
blocking access to the following social media platforms on 17th May 2023:
* Facebook Messenger:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023…
* Facebook:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023…
* WhatsApp:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023…
* Telegram:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023…
* Instagram:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023…
* YouTube:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023…
* Snapchat:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023…
(we only see blocking on 1 network from 19th May onwards)
For all of the above services, OONI data provides signals suggesting that
access to these platforms started to be blocked on 17th May 2023. However,
the block does not appear to be implemented on all networks in the country.
OONI data suggests that ISPs in Guinea implemented the blocks by means of
TLS interference. Specifically, OONI data shows that the TLS handshakes
result in a timeout error, which is why these social media services appear
to be inaccessible on tested networks.
*## Community use of OONI data*
*### Access Now statement in response to social media blocks in Guinea*
Access Now published a statement (citing OONI data) urging authorities in
Guinea to unblock social media:
https://www.accessnow.org/press-release/stop-shutting-down-the-internet-gui…
*### ISOC Pulse post on social media blocks in Guinea*
As part of the ISOC Pulse shutdowns project, ISOC included OONI data and
analysis in their post about the social media blocks in Guinea:
https://pulse.internetsociety.org/shutdowns/social-media-blockage-observed-…
*## Community activities### OONI workshop for civil society groups in
Liberia*
On 4th May 2023, OONI’s Maria facilitated an online OONI workshop for civil
society groups in Liberia. This workshop was organized by Access Now and
Liberian civil society groups to prepare for Liberia’s 2023 elections.
*### OONI workshop in Tbilisi*
On 9th May 2023, OONI’s Elizaveta facilitated an in-person OONI workshop as
part of a training in Tbilisi (organized by Internews) for content makers
from Turkmenistan.
*### OONI workshop for journalists in Kazakhstan *
On 17th May 2023, OONI’s Elizaveta facilitated an online OONI workshop for
journalists in Kazakhstan as part of the Greater Internet Freedom (GIF)
Convening of Objective Partners in Almaty.
*### OONI Community Meeting*
On 30th May 2023, we hosted the monthly OONI Community Meeting on our Slack
channel (https://slack.ooni.org/) during which we discussed the following
topics:
1) Updates from the OONI team.
2) Addressing false positives in Signal measurements displayed on the MAT (
https://github.com/ooni/backend/issues/679)
3) Using machine learning as part of OONI data analysis (
https://docs.google.com/presentation/d/1rw7a02lpTj4CcguAz_nbqzNzkACKdFvMzTL…)
*## Measurement coverage*
In May 2023, 59,563,089 OONI Probe measurements were collected from 2,990
networks in 159 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hello,
This email shares OONI's monthly report for April 2023.
*# OONI Monthly Report: April 2023*
Throughout April 2023, the OONI team worked on the following sprints:
* Sprint 88 (27th March - 9 April 2023)
* Sprint 89 (10th - 23 April 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## New partnerships*
We are excited to have formed 2 new partnerships!
Specifically, OONI established partnerships with the following digital
rights organizations:
* Roskomsvoboda (Russia): https://ooni.org/partners/roskomsvoboda/
* Miaan Group (Iran): https://ooni.org/partners/miaan/
The goal of the partnerships is to collaborate on the study of internet
censorship (in Russia and Iran), and we published dedicated partner pages
(provided above) to highlight their important work.
*## Published report on the blocking of Telegram in Brazil*
We published a new report documenting the blocking of Telegram in Brazil.
The report is available here: https://ooni.org/post/2023-brazil-telegram/
OONI data shows that some ISPs in Brazil immediately complied with the
federal judge’s decision to suspend Telegram, as the blocking of Telegram
started on 26th April 2023 (same day as the judge's decision). However, the
block was not implemented by all ISPs in Brazil, nor was it implemented in
the same way.
OONI data shows that some ISPs blocked access to all tested Telegram
endpoints, while others only blocked some of them. This suggests lack of
coordination between providers, and that each ISP implemented the block
autonomously.
Our Brazilian partner, Coding Rights, published a translated version of the
report:
https://codingrights.org/library-item/brasil-dados-da-ooni-sobre-o-bloqueio…
*## Published report on throttling of news media in Kazakhstan*
We published a new research report documenting the throttling of news media
websites in Kazakhstan. This report documents how OONI data can be used to
investigate and document cases of throttling.
The report is available here:
https://ooni.org/post/2023-throttling-kz-elections/
In our report, we analyze the ongoing throttling of Radio Free Europe/Radio
Liberty (RFE/RL) Kazakhstan’s service websites in Kazakh and Russian (
www.azattyq.org and rus.azattyq.org) which started on 27th September 2022.
We also document the temporary throttling of Current Time TV during
Kazakhstan's 2022 presidential election. OONI data shows the ongoing
throttling of RFE/RL Kazakhstan’s service websites on most tested Kazakh
providers, while Current Time TV was mainly throttled on AS9198 (JSC
Kazakhtelecom).
Our report received the following press coverage:
https://www.rferl.org/a/kazakhstan-rferl-internet-throttling-ooni/32383892.…https://rus.azattyq.org/a/kazakhstan-throttling-azattyq-currenttime/3238407…https://www.azattyq.org/a/32383510.htmlhttps://www.youtube.com/watch?v=I10nuJBKdUc
*## OONI Probe Mobile*
Throughout April 2023, we worked on OONI Probe Mobile bug fixing and
release engineering.
Specifically, we worked on the following bug fixes:
* Resolved a crash occurring on custom website list test results (
https://github.com/ooni/probe-android/pull/569)
* Fixed a bug related to starting the background service (
https://github.com/ooni/probe-android/pull/567)
* Made progress on improving pagination-based scrolling in the test result
screen (https://github.com/ooni/probe-android/pull/511,
https://github.com/ooni/probe-ios/pull/485)
We also worked on the following release engineering:
* Released OONI Probe version 3.8.0 and 3.8.1 (
https://github.com/ooni/probe-android/pull/560,
https://github.com/ooni/probe-android/pull/568)
* Released beta versions for the Android platform, following the
X.Y.Z-beta.A versioning pattern (
https://github.com/ooni/probe-android/pull/566)
* Worked towards a deployment plan for beta versions on Android.
*## OONI Probe CLI*
In April 2023, we:
* Started working towards optimizing how we manage the Mozilla certificate
pool (https://github.com/ooni/probe/issues/2444)
* Continued investigating issues with our integration of tor (
https://github.com/ooni/probe/issues/2406#issuecomment-1479365235)
* Introduced the optional type to more robustly handle cases where a
specific value is not given (https://github.com/ooni/probe-cli/pull/1131)
*## OONI Run*
As part of our work on improving OONI Run (https://run.ooni.io/) we
understood that the OONI Run v2 and the richer testing input designs should
be mutually compatible to reduce the overall complexity. We therefore
started sketching out a prototype check-in API implementation that allows
these two designs to interoperate (https://github.com/ooni/probe/issues/2445)
This is further discussed below (section on richer testing input).
We also started updating the frontend application in preparation for
upcoming UI changes. This work included migrating the codebase to
TypeScript (https://github.com/ooni/run/pull/128) and adding basic
end-to-end tests (https://github.com/ooni/run/pull/129)
*## Expanding OONI’s testing model to support richer testing input*
In April 2023, we continued working towards a design document (
https://github.com/ooni/ooni.org/issues/1292) for richer testing inputs and
we wrote a prototype (https://github.com/ooni/probe-cli/pull/1130)
implementing some of the ideas in the design document. We realized that the
name “input” is often associated with the string we pass to experiments. We
therefore considered introducing the concept of “target” to describe a
piece of richer input, and we started working on a glossary defining terms
used in the probe implementation.
We also recognized that richer input and OONI Run v2 depend on each other (
https://github.com/ooni/probe/issues/2445) The future check-in v2 API
serving the richer input to the probes could also serve OONI Run v2
descriptors. This realization means that we need to ensure that the richer
input design and the OONI Run v2 design are mutually compatible.
We implemented an experimental version of the check-in API to support
richer testing input (https://github.com/ooni/backend/pull/658)
*## Creating a methodology for measuring throttling*
As part of our research on the throttling of news media websites in
Kazakhstan (https://ooni.org/post/2023-throttling-kz-elections/) we
applied our methods for measuring and evaluating cases of throttling. The
report that we produced documents how OONI data can be analyzed to
investigate cases of throttling. Specifically, our report documents how we
analyzed TLS handshake data from the OONI Probe Web Connectivity
measurements (collected from Kazakhstan). To perform this analysis, we used
the OONI data analysis tool (https://github.com/ooni/data)
*## OONI backend*
We extracted API specs into a JSON file (
https://github.com/ooni/backend/pull/655) we ran experiments around STUN
reachability, and we started moving issues from the old API and pipeline
repositories on GitHub to the backend repository. We also deployed and
configured Vector on our monitoring host to centralize the log management
and implement log analysis.
We detected a sudden drop in incoming measurements from the Android probe
due to a bug in version 3.8.0, and we created internal dashboards for
investigation. We also investigated an issue that incorrectly flagged
Signal measurements as failed.
*## OONI Explorer*
In April 2023, we implemented an API entry point to fetch the measurement
metadata and body by measurement_uid. This will allow OONI Explorer users
to access every measurement even when two different measurements share the
same report ID and URL (https://github.com/ooni/backend/pull/656) Along
with the backend changes, we also added a new measurement page that
requests the measurement based on the measurement_uid and redirects from
the legacy page that requires the report ID and URL (
https://github.com/ooni/explorer/pull/850)
*## Creating a Social Media Censorship Alert System*
In April 2023, we made progress on the event detector developed for the new
Social Media Censorship Alert System. As part of our work on social media
blocking detection, we switched to using the Pandas library and worked on
bug fixes and blocking detection improvements.
This work is documented through the following pull request:
https://github.com/ooni/backend/pull/651
*## Automating censorship detection and characterization based on OONI
measurements*
In April 2023, we made progress on automating the detection and
characterization of censorship based on OONI measurements.
Specifically, we carried out the following activities:
* Continued developing analysis of DNS measurements to determine when DNS
level blocking is happening;
* Built a basic set of UI components for displaying DNS-related analysis
and merged it into the ooni/data tool (https://github.com/ooni/data/pull/25
);
* Added support for running the analysis using dask to improve the
performance and make it easier to scale across multiple machines (
https://github.com/ooni/data/pull/27)
* Added support for storing the probe-generated analysis to make it easier
to compare the new analysis engine with the current probe-based one (
https://github.com/ooni/data/pull/27/commits/eb460a42734fb73f6bb573edd051e0…
);
* Added support for parsing middlebox test results and storing features
related to them in database tables (
https://github.com/ooni/data/pull/27/commits/b8421096bd1d41b823e1a95ec51f3e…
);
* Added support for processing Telegram measurements (
https://github.com/ooni/data/pull/28)
* Added extraction and storing of additional probe-engine related
measurement file keys (
https://github.com/ooni/data/pull/27/commits/f0f65917acb2b0c33208e50673cd6c…
).
*## Community use of OONI data### Sinar Project’s report on censorship
monitoring during Malaysia’s 2022 elections*
Leading up to and during Malaysia’s 2022 elections, Sinar Project
coordinated an OONI measurement campaign to monitor potential censorship
events (
https://sinarproject.org/digital-rights/measuring-and-detecting-network-int…)
Based on the analysis of OONI data collected during Malaysia’s 2022
elections, Sinar Project published a report, documenting their findings.
Their report is available here:
https://imap.sinarproject.org/news/15th-malaysian-general-elections-interne…
*### Article about using OONI Probe in Turkmenistan*
Progres, an online non-profit journal, published an article about OONI in
Turkmen. This article describes OONI Probe, explaining how people in
Turkmenistan could use the tool to document internet censorship. The
article is available here:
https://progres.online/sowatly/name-ucin-web-sahypalar-acylmayar/
*## Community activities### OONI workshop at the Digital Rights and
Inclusion Forum (DRIF) 2023 in Kenya*
OONI’s Elizaveta traveled to Nairobi, Kenya, to participate in the Digital
Rights and Inclusion Forum (DRIF) 2023 organized by Paradigm Initiative.
Information about the event is available here: https://drif.paradigmhq.org/
On 13th April 2023, Elizaveta facilitated an OONI workshop (“Tracking
internet censorship: how to coordinate, document and investigate network
interference in your country”) as part of the event. Information about the
workshop is available here:
https://drif.paradigmhq.org/stec_event/open-observatory-of-network-interfer…
*### OONI MAT demo for Freedom House researchers*
On 17th April 2023, OONI’s Maria provided a live demo of the Measurement
Aggregation Toolkit (MAT) for Freedom House researchers.
Through this session, Maria shared how researchers can use the MAT (
https://explorer.ooni.org/chart/mat) to track (and visualize) internet
censorship around the world based on real-time OONI data, and use the
platform to answer a variety of research questions. The goal of this
session was to enable Freedom House researchers to make use of OONI data,
increasing the use of OONI data as part of the annual Freedom on the Net
reports (https://freedomhouse.org/report/freedom-net)
*### Presentation of Iran research to the EU High Level Group on Internet
Governance*
On 26th April 2023, OONI’s Maria joined IODA and ISOC to present the
findings from our technical multi-stakeholder research report on censorship
events in Iran (
https://ooni.org/post/2022-iran-technical-multistakeholder-report/) to EU
Member States at the morning session (open stakeholder session) of the High
Level Group on Internet Governance (HLIG).
As part of this presentation, Maria shared OONI censorship findings
involving the blocking of encrypted DNS, instant messaging and social media
platforms, app stores, browser extension repositories, and circumvention
tools in Iran.
*### OONI hackathon at Internet Without Borders conference in Berlin*
Between 28th-30th April 2023, the Internet Without Borders conference was
held in Berlin (https://internetborders.net/blog/2023/05/01/berlin-report/)
As part of this event, Ain Ghazal (OONI research fellow) facilitated an
OONI hackathon that involved developing a VPN measurement research
methodology for Russia and the occupied territories of Ukraine.
*### OONI Community Meeting*
On 25th April 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we discussed the
following topics:
1. Updates from the OONI team.
2. Community feedback for improving OONI tools (such as OONI Probe feature
requests).
3. Using OONI Probe to measure internet connectivity shutdowns.
4. Recording automated testing as a signal for identifying internet outages.
*## Measurement coverage*
In April 2023, 53,140,981 OONI Probe measurements were collected from 3,003
networks in 166 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
