[tor-project] OONI Monthly Report: May 2023

[Maria Xynou]

Hello,

This email shares OONI's monthly report for May 2023.

*# OONI Monthly Report: May 2023*

Throughout May 2023, the OONI team worked on the following sprints:

* Sprint 90 (24th April - 7th May 2023)
* Sprint 91 (8th - 21st May 2023)
* Sprint 92 (22nd May - 4th June 2023)

Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni

Highlights are shared in this report below.

*## New partnership*

We are excited to have formed a new partnership with Annir Initiative!

The goal of the partnership is to collaborate on the study of internet
censorship in Libya. We published a dedicated partner page for Annir
Initiative highlighting their important work:
https://ooni.org/partners/annir/

*## Job Opening: OONI Project Manager*

On 2nd May 2023, we published a job opening for hiring an OONI Project
Manager!

The job posting is available here:
https://ooni.org/post/2023-job-opening-ooni-project-manager/

In an attempt to help ensure that our job opening reached interested
candidates around the world, we worked on relevant outreach efforts. In
particular, we posted the job opening on multiple platforms, including:
* All OONI social media channels (Twitter, Mastodon, Facebook, Instagram);
* All OONI mailing lists;
* Community mailing lists (such as those of the #KeepItOn campaign, Global
Voices Advocacy, and NetRights);
* OONI Slack channel (and other relevant Slack, Mattermost, and IRC
channels);
* Digital Rights Job Board (https://www.digitalrights.community/job-board);
* FOSS Job Board (
https://www.fossjobs.net/job/11234/ooni-project-manager-at-open-observatory-of-network-interference-ooni/
).

We also reached out to our partners and many other community members,
requesting that they help share this opportunity with their networks.

Following the publication of this job opening, we worked on tracking and
reviewing incoming applications on a rolling basis, and following-up with
applicants. In May 2023, we started the first round of interviews with
shortlisted candidates.

*## Blog post about new OONI Explorer features*

We published a blog post to share information about new OONI Explorer
features for investigating internet censorship through open data.

This blog post is available here:
https://ooni.org/post/2023-new-explorer-features/

Specifically, OONI Explorer includes the following new features:
* Domain-centric pages: https://explorer.ooni.org/domains
* Network-centric pages: https://explorer.ooni.org/networks
* Charts on internet outages (integrating IODA, Google traffic, and
Cloudflare Radar data), available in each country-specific (
https://explorer.ooni.org/countries) and network-specific (
https://explorer.ooni.org/networks) page
* Mechanism for verifying OONI measurements (available in each OONI
measurement page)

We hope these new features help with monitoring and responding to
censorship events worldwide.

*## OONI Probe Mobile*

We released OONI Probe Android 3.8.2 (
https://github.com/ooni/probe-android/releases/tag/v3.8.2), which includes
a new version of the engine and ships some of the previously developed bug
fixes.

We also worked towards addressing the following bugs:
* OONI Probe installation failure caused by duplicate permission (
https://github.com/ooni/probe/issues/2450);
* Android OONI Probe notification constantly wakes up the mobile screen (
https://github.com/ooni/probe-android/pull/570 ).

*## OONI Probe Desktop*

We released OONI Probe Desktop 3.9.0 (
https://github.com/ooni/probe-desktop/releases/tag/v3.9.0) which contains
the Signal test fix from OONI Probe CLI v3.17.2.

*## OONI Run*

We created a first iteration of the OONI Run v2 link web editor interface.
Specifically, we built the frontend component for the UI, which will enable
users to create and edit existing links (
https://github.com/ooni/run/pull/131).

We added support in the backend API for creating/updating the OONI Run v2
descriptor and serving them to clients (
https://github.com/ooni/backend/pull/678). We also discussed and reviewed
internal design documents for the OONI Run v2 check-in API.

*## OONI Probe CLI*

We released OONI Probe CLI v3.17.3 (
https://github.com/ooni/probe-cli/releases/tag/v3.17.3).  This release
updates to go1.19.9, fixes issues with the Signal test, and updates the
bundled geoip database to 2023-05. We did not release this version because
we noticed that we needed to fix more bugs, such as upgrading OpenSSL (
https://github.com/ooni/probe-cli/commit/325a8412999b001f4913ab5c522ed6402c9d78fe)
 and fixing a test helper issue (
https://github.com/ooni/probe-cli/commit/a3af5542986e314a4e766d36078bedbae5c977ef).
We therefore chose to wait until 2023-06-05 to bundle the 2023-06 geoip
databases.

Following a community request (https://github.com/ooni/probe/issues/2478),
we started preparing for releasing version 3.18, which will include support
for go1.20. In this reporting period, we upgraded the code to use go1.20 (
https://github.com/ooni/probe-cli/pull/1144)  and we started following the
usual pre-release check-list (https://github.com/ooni/probe/issues/2417).

*## Expanding OONI’s testing model to support richer testing input*

We continued to work on the richer input design document (
https://github.com/ooni/ooni.org/issues/1295). We open sourced a repository
that contains an initial design document (
https://github.com/ooni/2023-05-richer-input/blob/main/DESIGN.md) and
refactors OONI Probe around the concept of richer input. The code inside of
the repository shows how we can use the concept of richer input to simplify
the probe implementation. We explored, in particular, how we could use
richer input to streamline the implementation of the OONI Probe IM tests.

*## Creating a Social Media Censorship Alert System*

In May 2023, we continued to make progress towards building a Social Media
Censorship Alert System. Specifically, we improved our social media
blocking event detector by adding RSS/Atom feed generation. Additionally,
the event detector was updated to run hourly.

This work is documented through the following pull request:
https://github.com/ooni/backend/pull/651

*## OONI backend*

Throughout May 2023, we worked on the following backend activities:
* Annotated faulty Signal measurements as failed (
https://github.com/ooni/backend/pull/680);
* Monitored incoming measurements following an OONI Probe Android 3.8.0 bug
impacting unattended runs;
* Increased the testing priority for URLs in Timor Leste (amid the
country’s 2023 elections) and created internal dashboards to monitor the
testing coverage;
* Created an MVP for an internal dashboard for long-term measurement
traffic prediction and alarming;
* Fixed a broken dependency in the Altair library used on our Jupyter
Notebook instance;
* Fixed the Jupyter Notebook automatic runner (jupycron) to generate HTML
reports on hourly, daily and weekly intervals;
* Implemented tooling to measure the performance of the probe engine using
valgring / cachegrind (https://github.com/ooni/probe-cli/pull/1137);
* Continued working towards configuring Vector to centralize log management
(setting up SSL certificates);
* Configured the main backend host (backend-FSN) to send logs to the
monitoring host;
* Created initial internal dashboards for log monitoring.

*## Test lists updates*

In preparation for elections which could potentially trigger new censorship
events, we collaborated with our partners and other community members on
updating relevant test lists.

This involved updates to the test lists for the following countries:
* Turkey: https://github.com/citizenlab/test-lists/pull/1290,
https://github.com/citizenlab/test-lists/pull/1291
* Thailand: https://github.com/citizenlab/test-lists/pull/1289,
https://github.com/citizenlab/test-lists/pull/1280,
https://github.com/citizenlab/test-lists/pull/1282
* Timor Leste: https://github.com/citizenlab/test-lists/pull/1296

We thank community members for contributing to these test lists!

*## OONI profile for Mozilla 2023 Data Futures Lab Cohort*

OONI is thrilled to be part of Mozilla’s 2023 Data Futures Lab Cohort (
https://foundation.mozilla.org/en/blog/mozilla-welcomes-2023-data-futures-lab-cohort/)!
As part of this, we’re working towards releasing an improved version of
OONI Run that addresses key community feedback, supporting decentralized
censorship measurement campaigns around the world.

In May 2023, Mozilla published a blog post which provides a profile of OONI
(as a Mozilla Data Futures Lab awardee). This blog post includes quotes
from an interview with OONI’s Arturo, and can be accessed here:
https://foundation.mozilla.org/en/blog/crowdsourcing-data-as-a-tool-to-fight-internet-censorship/

*## ClickHouse blog post*

ClickHouse published a blog post describing how we use their database
management system for OONI data:
https://clickhouse.com/blog/ooni-analyzes-internet-censorship-data-with-clickhouse

We migrated to ClickHouse a few years ago, and that has enabled us to
analyze and publish OONI measurements in real-time (seconds)!

*## Blocking of social media in Pakistan*

In response to the blocking of social media platforms in Pakistan, we
shared relevant OONI data with the #KeepItOn advocacy mailing list and on
social media platforms (
https://twitter.com/OpenObservatory/status/1656417835365588994,
https://twitter.com/OpenObservatory/status/1658489171902885897).

Specifically, starting from around 15:00 UTC on 9th May 2023, OONI data
collected from Pakistan showed that several ISPs started to restrict access
to Facebook, YouTube, and Twitter. You can find relevant OONI data (along
with a chart) here:
https://explorer.ooni.org/chart/mat?test_name=web_connectivity&axis_x=measurement_start_day&since=2023-05-09&until=2023-05-11&time_grain=hour&probe_cc=PK&axis_y=domain&domain=www.youtube.com%2Cwww.facebook.com%2Ctwitter.com%2Cwww.instagram.com

The blocking of YouTube is further suggested by Google's Transparency
Report, which shows a drop in YouTube traffic from Pakistan:
https://transparencyreport.google.com/traffic/overview?hl=en&fraction_traffic=start:1682812800000;end:1683763199999;product:21;region:PK&lu=fraction_traffic

Starting from around 05:00 UTC on 10th May 2023, OONI data collected from
Pakistan showed that several ISPs started to restrict access to Instagram
as well:
https://explorer.ooni.org/chart/mat?probe_cc=PK&since=2023-04-12&until=2023-05-12&time_grain=day&axis_x=measurement_start_day&test_name=web_connectivity&domain=www.instagram.com

In most measurements (for the restricted social media platforms), OONI data
shows signs of TLS-level interference (as TLS handshakes time out), while
some measurements show DNS-based blocking as well (returning an NXDOMAIN
error). Notably, some providers implement both censorship techniques.

*## Blocking of social media in Guinea*

In response to the blocking of social media platforms in Guinea, we shared
relevant OONI data (
https://twitter.com/OpenObservatory/status/1660591036316499968).

Specifically, OONI data from Guinea suggests that some ISPs started
blocking access to the following social media platforms on 17th May 2023:
* Facebook Messenger:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023-05-23&time_grain=day&axis_x=measurement_start_day&test_name=facebook_messenger
* Facebook:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023-05-23&time_grain=day&axis_x=measurement_start_day&test_name=web_connectivity&domain=www.facebook.com
* WhatsApp:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023-05-23&time_grain=day&axis_x=measurement_start_day&test_name=whatsapp
* Telegram:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023-05-23&time_grain=day&axis_x=measurement_start_day&test_name=telegram
* Instagram:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023-05-23&time_grain=day&axis_x=measurement_start_day&test_name=web_connectivity&domain=www.instagram.com
* YouTube:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023-05-23&time_grain=day&axis_x=measurement_start_day&test_name=web_connectivity&domain=www.youtube.com
* Snapchat:
https://explorer.ooni.org/chart/mat?probe_cc=GN&since=2023-04-23&until=2023-05-23&time_grain=day&axis_x=measurement_start_day&test_name=web_connectivity&domain=www.snapchat.com
(we
only see blocking on 1 network from 19th May onwards)

For all of the above services, OONI data provides signals suggesting that
access to these platforms started to be blocked on 17th May 2023. However,
the block does not appear to be implemented on all networks in the country.

OONI data suggests that ISPs in Guinea implemented the blocks by means of
TLS interference. Specifically, OONI data shows that the TLS handshakes
result in a timeout error, which is why these social media services appear
to be inaccessible on tested networks.

*## Community use of OONI data*

*### Access Now statement in response to social media blocks in Guinea*

Access Now published a statement (citing OONI data) urging authorities in
Guinea to unblock social media:
https://www.accessnow.org/press-release/stop-shutting-down-the-internet-guinea/

*### ISOC Pulse post on social media blocks in Guinea*

As part of the ISOC Pulse shutdowns project, ISOC included OONI data and
analysis in their post about the social media blocks in Guinea:
https://pulse.internetsociety.org/shutdowns/social-media-blockage-observed-in-guinea



*## Community activities### OONI workshop for civil society groups in
Liberia*

On 4th May 2023, OONI’s Maria facilitated an online OONI workshop for civil
society groups in Liberia. This workshop was organized by Access Now and
Liberian civil society groups to prepare for Liberia’s 2023 elections.

*### OONI workshop in Tbilisi*

On 9th May 2023, OONI’s Elizaveta facilitated an in-person OONI workshop as
part of a training in Tbilisi (organized by Internews) for content makers
from Turkmenistan.

*### OONI workshop for journalists in Kazakhstan*

On 17th May 2023, OONI’s Elizaveta facilitated an online OONI workshop for
journalists in Kazakhstan as part of the Greater Internet Freedom (GIF)
Convening of Objective Partners in Almaty.

*### OONI Community Meeting*

On 30th May 2023, we hosted the monthly OONI Community Meeting on our Slack
channel (https://slack.ooni.org/), during which we discussed the following
topics:

1) Updates from the OONI team.

2) Addressing false positives in Signal measurements displayed on the MAT (
https://github.com/ooni/backend/issues/679).

3) Using machine learning as part of OONI data analysis (
https://docs.google.com/presentation/d/1rw7a02lpTj4CcguAz_nbqzNzkACKdFvMzTLrairAQ70/edit?usp=sharing
).

*## Measurement coverage*

In May 2023, 59,563,089 OONI Probe measurements were collected from 2,990
networks in 159 countries around the world.

This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/

~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20230712/4cec8b42/attachment-0001.htm>