Hi friends,
As a non-profit, donations from people like you support OONI's work in
building free tools for exposing Internet censorship worldwide.
Today we are excited to share 2 new pages:
** Donations:* https://ooni.org/donate
** Supporters:* https://ooni.org/about/supporters
If you donate ≥ €42, you’ll get an OONI t-shirt! :)
We have also introduced new supporter tiers and benefits for those
interested in supporting OONI's mission.
If you value our work, please consider donating today.
Thanks so much for your support!
Warm regards,
Maria (from the OONI team).
Hello,
Today we are thrilled to announce the *launch of the new OONI Censorship
Findings platform*!
Access the platform here: https://explorer.ooni.org/findings
The Censorship Findings platform includes *20 reports on internet
censorship* around the world based on OONI data. In response to emergent
censorship events, we will publish reports on the Censorship Findings
platform on an ongoing basis.
These short reports include key censorship findings and charts based on
OONI data, which we hope you will find useful. Our goal is to support
research and advocacy efforts aimed at monitoring and responding to
internet censorship around the world.
We thank the global OONI community for contributing measurements, making
the creation of this platform possible.
Learn more through our blog post:
https://ooni.org/post/2023-launch-ooni-censorship-findings-platform/
And please help spread the word:
https://twitter.com/OpenObservatory/status/1730274379433382022
Thanks,
OONI team.
Hello friends,
Last month, in collaboration with ISOC, M-Lab, Censored Planet, and IODA,
OONI co-hosted an *internet measurement hackathon* alongside the Internet
Measurement Conference (IMC) 2023.
We published a *blog post sharing the cool projects* that came out of the
hackathon: https://ooni.org/post/imc-hackathon-results-2023/
The hackathon participants worked on using OONI data to investigate IPv6
connectivity, data triangulation to infer censorship or shutdown events,
and on correlating alarms across multiple datasets. We were very impressed
with the results produced by the participants!
During the hackathon, OONI's Arturo created "*The Sound of Internet
Shutdowns*": a sonification project which transforms some of the features
of the IODA dataset into synthesized sounds:
https://shutdown-sound.vercel.app/
Here is an example of *how an internet shutdown in Iraq based on IODA data
sounds* like: https://www.youtube.com/watch?v=Z1TFc1AyqI4
Warm thanks to all the participants and co-organizers for making the
hackathon a great event!
Cheers,
Maria.
Hello,
This is a kind reminder that OONI will host a monthly community meeting next Tuesday.
=> Where? OONI Slack channel: https://slack.ooni.org/ <https://slack.ooni.org/> (bridged with IRC: ircs://irc.oftc.net:6697/#ooni <ircs://irc.oftc.net:6697/#ooni>)
=> When? Tuesday, 28th November 2023 at 14:00 UTC (for 1 hour)
Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep <https://pad.riseup.net/p/ooni-community-meeting-keep>
This is a meeting that we host each last Tuesday of the month, and it is an opportunity to:
* Share your feedback on OONI tools & methodologies
* Ask questions about OONI tools and OONI data
* Discuss internet censorship issues, or share the issues you are facing in your country
* Share updates about your work or projects
This time we will be also discussing revamping the community meetings format, so if you have any ideas or suggestions, please come and share!
We hope you can join us!
All the best,
~ OONI team
Hello,
This email shares OONI's monthly report for October 2023.
*# OONI Monthly Report: October 2023*
Throughout October 2023, the OONI team worked on the following sprints:
* Sprint 101 (1st - 8th October 2023)
* Sprint 102 (9th-22nd October 2023)
* Sprint 103 (24th-31st October 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Launch of News Media Scan app*
In collaboration with Deutsche Welle (DW), we developed an OONI Probe-based
app (“News Media Scan”) designed to measure the blocking of news media
websites. Similarly to OONI Probe, News Media Scan app test results are
published by OONI as open data in real-time (https://ooni.org/data/)
On 5th October 2023, Deutsche Welle and OONI launched the News Media Scan
app for Android, which is available in the Google Play Store:
https://play.google.com/store/apps/details?id=com.dw.ooniprobe&pli=1
Deutsche Welle published a press statement, announcing the launch:
https://corporate.dw.com/en/news-media-scan-by-dw-deutsche-welle-and-ooni-l…
The app also received press coverage by turi2:
https://www.turi2.de/aktuell/dw-und-ooni-bringen-app-zur-erkennung-von-inte…
We are excited about this launch, as we expect that it will help boost news
media censorship measurement worldwide, supporting efforts to defend press
freedom around the world.
*## New partnership*
We are excited to have established a new partnership with Digitally Right (
https://digitallyright.org/) a digital rights organization in Bangladesh.
We aim to collaborate on the study of internet censorship in Bangladesh.
We also published a page to feature our new research collaboration
(established in September 2023) with the GLITCH Research Interest Group at
the Oxford Internet Institute, a department of the University of Oxford.
This page is available here: https://ooni.org/partners/glitch/
*## Published OONI Community Interview with Siti Nurliza*
On 16th October 2023, we published an interview with Siti Nurliza: a
talented data analyst and technologist with Sinar Project, one of our most
dedicated and long-term partners who have led OONI censorship measurement
efforts in Southeast Asia for more than 6 years!
Siti’s interview (which we published on the OONI YouTube channel) can be
viewed here: https://www.youtube.com/watch?v=DOTjOR7zA2Y
This interview is part of our “OONI Community Interviews” series, through
which we highlight the important work of OONI community members.
We also published a blog post to share the interview and information about
Sinar Project and their work:
https://ooni.org/post/2023-interview-with-siti-nurliza/
*## OONI Probe user guides available in Vietnamese*
As of October 2023, the OONI Probe user guides are available in Vietnamese!
The Vietnamese user guides can be found below:
* OONI Probe Mobile: https://ooni.org/vi/support/ooni-probe-mobile
* OONI Probe Desktop: https://ooni.org/vi/support/ooni-probe-desktop
We thank Nathan Tran for the translation!
*## OONI Explorer available in Arabic, Burmese and Portuguese*
In October 2023, we released OONI Explorer in Arabic, Burmese, and
Portuguese.
* Arabic: https://explorer.ooni.org/ar
* Burmese: https://explorer.ooni.org/my
* Portuguese: https://explorer.ooni.org/pt-BR
We thank the Localization Lab community for these translations!
*## OONI Outreach Kit available in Russian*
We are excited to share that the OONI Outreach Kit is now available in
Russian!
The Russian version of the OONI Outreach Kit can be found here:
https://ooni.org/ru/support/ooni-outreach-kit/
*## OONI Probe Mobile*
In October 2023, we released OONI Probe Mobile 3.8.4 for both Android (
https://github.com/ooni/probe-android/releases/tag/v3.8.4) and iOS (
https://github.com/ooni/probe-ios/releases/tag/v3.8.4) which includes a
series of improvements.
Notably, OONI Probe 3.8.4 includes support for Vietnamese and Burmese.
*## OONI Run*
As part of our work on creating the next generation version of OONI Run
(“OONI Run v2”), we continued to iterate on the user interface designs for
the web and mobile applications.
We started updating the OONI Run UI on OONI Probe Mobile from v1 to v2, (
https://github.com/ooni/probe-android/pull/626) This included updating the
test overview view, implementing new components (
https://github.com/ooni/probe-android/pull/629) updating the dashboard
view (https://github.com/ooni/probe-android/pull/631) and updating the
choose websites view (https://github.com/ooni/probe-android/pull/630)
*## OONI Probe CLI*
In October 2023, we released OONI Probe CLI 3.19.0:
https://github.com/ooni/probe-cli/releases/tag/v3.19.0
Below are some highlights from this release:
1) We improved support for measuring throttling by integrating code that
takes periodic download speed snapshots while fetching websites in Web
Connectivity LTE (the currently-experimental next-generation Web
Connectivity implementation). This data collection process will further
enrich the measurements we collect and allow spotting throttling across the
whole web page download, as opposed to limiting the analysis to spotting
throttling during the TLS handshake and the beginning of the download
(which is what 3.18 was capable of doing).
2) We introduced the concept of OONI Probe Bridges. A bridge is an IP
address that we can use to communicate with the OONI backend and for which
we are confident that we can use any Server Name Indication during the TLS
handshake without errors. We believe this functionality will improve the
reliability under some censorship conditions.
3) We improved measurement scrubbing to remove any string that looks like
an IP address or a TCP/UDP endpoint from the HTTP response bodies and
headers. We are still scrubbing known IP addresses (e.g., the one used by
OONI Probe) from the whole measurement. By adding this extra scrubbing
step, we further increase OONI Probe’s safety.
4) We started using the netemx integration testing framework in the
codebase. This framework combines GVisor (https://gvisor.dev/) an emulated
internet topology, and deep packet inspection, to allow us to write complex
and realistic censorship scenarios. We migrated many of the most important
nettests to use this framework for testing. Going forward, this capability
will allow us to replace the implementation of experiments knowing that key
properties related to detecting censorship continue to hold.
5) We added support to the OONI Probe Engine for fetching OONI Run v2
descriptors, which is a precondition for implementing OONI Run v2 in mobile
applications.
6) Taking advantage of `netemx`, we migrated all the tests run by `go test
-short ./...` to use netemx rather than the host network. This means that,
from this release onwards, one does not need to have access to an
uncensored network to develop OONI Probe. In fact, all the key unit and
integration tests only rely on GVisor and on the emulated network topology
described above, meaning that no traffic is actually sent to or received
from actual network interface cards.
7) We fixed the bootstrap of OONI Probe in a case where the timeouts
configuration conflicted with the bootstrap being successful. While version
3.19 only includes a hotfix, we plan on refactoring this part of OONI Probe
and fixing some fundamental issues we identified (
https://github.com/ooni/probe/issues/2545)
8) We fixed a data quality issue in Web Connectivity v0.4 where Android’s
getaddrinfo resolver conflicted with the analysis logic in the presence of
NXDOMAIN-based censorship. Android has a getaddrinfo implementation that
always returns EAI_NODATA when there is any error. This happens because
getaddrinfo is actually a proxy for a DNS lookup service and there is no
support for passing errors along. The OONI Probe engine did already
identify this specific error condition as android_dns_cache_no_data. What
changed in version 3.19 is that we say that there is a measurement anomaly
if this error appears and the test helper, instead, says that the domain
for the website we’re measuring successfully resolves to IP addresses.
*## Expanding OONI’s testing model to support richer testing input*
After months of research on richer input, we were able to draft a plan that
describes exactly how we are going to implement it and for which
experiments. You can read the whole plan here:
https://github.com/ooni/ooni.org/issues/1295
Below are some highlights:
1) We will implement richer input by enriching the existing
`/api/v1/check-in` response (i.e., the response from the check-in API used
to deliver measurement targets to OONI Probe).
2) The check-in API client in the OONI Probe Engine will cache specific
bits of the check-in response using the disk or memory. Some information
will be cached on disk with a pretty long expiry time. Other information is
cached for a short timeframe or kept in memory.
3) We will modify how we run experiments by moving around a pure-data
structure that contains the experiment name, the experiment options, and
the experiment inputs. (The whole point of richer input is indeed to move
options around and honor them when running experiments.)
4) We will need to refactor how ooniprobe, the miniooni research client,
and the mobile apps execute experiments by replacing code that create
instances of “Experiment” with code that moves around this pure data
structure and defers the actual instantiation and execution of the
experiment deep inside of the OONI Engine.
5) The ooniprobe, miniooni, and mobile apps codebase will prepare pure-data
structures and submit them to the engine for execution. In the common case
of running experiments like we currently do, ooniprobe, miniooni, and the
mobile apps will not fill any option. Instead, the engine would do that
based on the cached check-in content.
In October 2023, we also started writing code to implement this plan, but
the bulk of the work is going to happen in subsequent weeks and months.
Obviously, the plan is tentative and possibly poised to change as we learn
more and flesh out the implementation.
*## Creating a throttling measurement methodology*
As mentioned above when discussing OONI Probe CLI 3.19.0 release notes,
there is better support for detecting throttling only in Web Connectivity
LTE. As you may remember from previous reports, Web Connectivity LTE is
currently experimental and only enabled for a small subset of our users.
This situation begs the question of how to bring these extra throttling
improvements to all experiments. For example, it would be nice if the OONI
Probe Telegram test measures throttling when testing web.telegram.org.
To make this possible, we need to switch the underlying engine used by
other experiments from the old `legacy/netx` to the new `mesurexlite`
engine. The reason why we need to do this change is that the `measurexlite`
engine uses a different methodology for collecting OONI measurements called
tracing. This methodology is more flexible than the one used by the
`legacy/netx` engine, called wrapping, and enables us to collect more data
– including data relevant for throttling – in a more flexible way.
Currently, Web Connectivity LTE is the only experiment using
`measurexlite`. But switching the engine requires rewriting large swaths of
code. Thus, we spent time polishing the `netemx` testing framework and
rewriting unit and integration testing code to use it. This work occurred
in previous reporting periods and was released in version 3.19. Hence, now
we start being well positioned to initiate the migration to the
`measurexlite` engine.
However, `measurexlite` is a low-level engine and writing code for it
directly has proven to be difficult. For writing Web Connectivity LTE, in
fact, we used a code generator. However, we were not completely satisfied
with this approach and we introduced `dslx`, which allows composing
measurement primitives in an easy way and which is based on `measurexlite`.
Therefore, the plan is now to rewrite existing experiments using `dslx`.
To make this possible, in October 2023 we applied fixes that allow writing
more compact and expressive code, based on our several-months-long
experience in using `dslx`. (To make sure `dslx` was up to the task, we
used it extensively when experimenting with richer input; this means that
we already know about some pain points.)
More specifically, below are the changes we applied to `dslx`:
1. https://github.com/ooni/probe/issues/2580
2. https://github.com/ooni/probe/issues/2582
3. https://github.com/ooni/probe/issues/2611
4. https://github.com/ooni/probe/issues/2612
5. https://github.com/ooni/probe/issues/2613
6. https://github.com/ooni/probe/issues/2614
7. https://github.com/ooni/probe/issues/2615
8. https://github.com/ooni/probe/issues/2617
9. https://github.com/ooni/probe/issues/2618
With these changes implemented, we’re now well positioned to move forward
with measuring throttling.
*## Creating a Social Media Censorship Alert System*
We made progress on the research side of the Social Media Censorship Alert
System by polishing a design document and updating the relevant pull
request (https://github.com/ooni/backend/pull/651) based on feedback. We
also started to reach out to members of the research community to collect
input on how we might improve upon the design of the alert system.
*## Creating a Censorship Incident Reporting Platform*
We continued working towards launching the Censorship Incident Reporting
Platform. Our focus was on testing, making improvements and UI tweaks (
https://github.com/ooni/explorer/pull/878) to ensure that everything works
smoothly. We also updated our component library as needed (
https://github.com/ooni/design-system/) On the backend side, we added
support for displaying the author's email address in the admin UI of the
incident reporting platform (https://github.com/ooni/backend/pull/732)
We wrote most of the censorship reports for the platform, and we started
publishing them as part of our internal testing process. As part of this
testing and publication process, we documented feedback for improvements (
https://github.com/ooni/explorer/issues/879) All of the reports (along
with the platform itself) will be accessible from the OONI Explorer menu
navigation bar when the platform is publicly launched.
*## OONI backend*
In order to improve the effectiveness in circumventing the blocking of the
traffic between probes and the API, we deployed a second ooni bridge using
a different provider than Hetzner: Greenhost.
We fixed a previously identified bug in the probe registration API entry
point affecting the authentication JWT. The bug did not impact the
live/production API because it uses Orchestrate for this (
https://github.com/ooni/backend/issues/731,
https://github.com/ooni/backend/pull/727)
Based on a request from a user, we investigated the generation of test
lists in Georgia and created a monitoring script and a dashboard. This is
an initial stepping stone to better understand how frequently the URLs in
test lists are "reordered" by the prioritization algorithm.
We detected a client scraping the aggregation API. The increase in load did
not impact the performance of the API to the point of disrupting the
service, however it increased the CPU load enough to be clearly noticeable
on performance dashboards. We improved an existing Jupyter notebook to
extract the heaviest queries.
We also investigated using traffic quotas in ClickHouse but encountered a
bug in the current version of the database that required postponing the
implementation. After noticing high database load due to the measurement
uploader tool and the database backup tool we investigated how to lower the
query priorities to decrease the impact.
We performed initial end-to-end tests for a custom STUN server using probe
VPSs as vantage points. The STUN server is meant to be used for
circumvention and potentially to measure latency between the probe and the
server.
We continued carrying out security updates in the OS on our servers (
https://github.com/ooni/backend/issues/740) We updated the monitoring host
to Debian Bookworm. As part of this Grafana and ClickHouse have been
upgraded to newer and more performant versions. We also worked on improving
the existing documentation for the backend infrastructure.
*## Automating censorship detection and characterization based on OONI
measurements*
We continued to make progress on automating censorship detection and
characterization based on OONI measurements. Specifically, we worked on
improving the performance of the observation generation for older buckets,
which are currently affected by a bug that makes them not be compressed.
As part of this work, we implemented a different parallelization strategy
which allows us to reprocess a bucket of measurements in a day in around 5
minutes (whereas previously it was in the range of 30-40 minutes).
We also added support for storing statistics in the database about how long
it took to process a given batch of measurements, allowing us to more
accurately assess how we are doing on the performance front. This work is
documented here: https://github.com/ooni/data/pull/37
*## Scoping upcoming research reports*
Throughout October 2023, we worked towards scoping some of our upcoming
research reports. This involved analyzing OONI data to identify interesting
censorship cases, writing internal documents with relevant research
questions, plans, and timelines, and coordinating with relevant research
partners.
*## Created surveys for upcoming OONI training in the DRC*
In preparation for a week-long OONI training that we will be facilitating
in Kinshasa (DRC) in November 2023, we created two surveys for the
participants. The first survey will be shared with the participants at the
start of the training, while the second survey will be shared at the end.
The goal of these surveys is to evaluate how effective the training will
be, and to collect relevant participant feedback. We shared these surveys
with the training organizers for French translation.
*## Created the agenda for the OONI Team Meeting 2023*
Between 8th-10th November 2023, we will host our annual OONI Team Meeting
in Nairobi, Kenya. This 3-day meeting will provide our team the opportunity
to discuss roadmaps, strategic priorities, and projects in person.
In preparation for our annual OONI Team Meeting, we solicited team feedback
for the agenda. Based on team feedback and session proposals, we worked on
creating and finalizing the detailed agenda for our 3-day team meeting.
We also coordinated on other relevant logistics pertaining to travel to
Nairobi for our upcoming meeting.
*## Test Lists updates*
In October 2023, we reviewed test list updates contributed by community
members.
These include:
* Extensive update of the Kazakhstan test list by our partner, Internet
Freedom Kazakhstan: https://github.com/citizenlab/test-lists/pull/1459
* Update of the Global test list by a community member:
https://github.com/citizenlab/test-lists/pull/1452
*## Rapid response*
*## Azerbaijan unblocked access to TikTok*
On 31st October 2023, Azerbaijan unblocked access to TikTok! Azerbaijan had
previously been blocking access to TikTok since 19th September 2023 (during
the military offensive in Nagorno-Karabakh). We shared OONI data and
reported on the unblocking of TikTok in Azerbaijan on X:
https://twitter.com/OpenObservatory/status/1719742224940859674
*## Community use of OONI data### iMAP 2023 Internet Censorship Reports*
On 13th October 2023, the iMAP project (https://imap.sinarproject.org/)
published 10 new research reports on internet censorship in the following
Asian countries: Malaysia, Myanmar, Thailand, Vietnam, Cambodia, Hong Kong
(China), India, Indonesia, Timor-Leste, and the Philippines. These reports
are based on the analysis of OONI data collected from these 10 countries.
The iMAP 2023 Internet Censorship Reports are available here:
https://imap.sinarproject.org/reports/2023
Specifically, they include the following reports:
1) Cambodia: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-cambodia-2023-internet-cens…
2) Hong Kong (China): 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-hong-kong-2023-internet-cen…
3) India: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-indonesia-2023-internet-cen…
4) Indonesia: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-indonesia-2023-internet-cen…
5) Malaysia: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-malaysia-2023-internet-cens…
6) Myanmar: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-myanmar-2023-internet-censo…
7) Philippines: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-philippines-2023-internet-c…
8) Thailand: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-thailand-2023-internet-cens…
9) Timor-Leste: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-timor-leste-2023-internet-c…
10) Vietnam: 2023 Internet Censorship Report:
https://imap.sinarproject.org/reports/2023/imap-vietnam-2023-internet-censo…
The launch and presentation of these reports was live-streamed here:
https://www.youtube.com/watch?v=YkFQxzvJrMs
*### Freedom on the Net 2023 reports*
Freedom House published their annual Freedom on the Net report (
https://freedomhouse.org/report/freedom-net/2023/repressive-power-artificia…)
which includes numerous country reports (
https://freedomhouse.org/countries/freedom-net/scores)
OONI data is cited in the following 38 (out of 70 in total)
country-specific Freedom on the Net 2023 country reports:
* Azerbaijan: https://freedomhouse.org/country/azerbaijan/freedom-net/2023
* Belarus: https://freedomhouse.org/country/belarus/freedom-net/2023
* Brazil: https://freedomhouse.org/country/brazil/freedom-net/2023
* Cambodia: https://freedomhouse.org/country/cambodia/freedom-net/2023
* Colombia: https://freedomhouse.org/country/colombia/freedom-net/2023
* Cuba: https://freedomhouse.org/country/cuba/freedom-net/2023
* Egypt: https://freedomhouse.org/country/egypt/freedom-net/2023
* Ethiopia: https://freedomhouse.org/country/ethiopia/freedom-net/2023
* India: https://freedomhouse.org/country/india/freedom-net/2023
* Indonesia: https://freedomhouse.org/country/indonesia/freedom-net/2023
* Iran: https://freedomhouse.org/country/iran/freedom-net/2023
* Iraq: https://freedomhouse.org/country/iraq/freedom-net/2023
* Italy: https://freedomhouse.org/country/italy/freedom-net/2023
* Jordan: https://freedomhouse.org/country/jordan/freedom-net/2023
* Kazakhstan: https://freedomhouse.org/country/kazakhstan/freedom-net/2023
* Kenya: https://freedomhouse.org/country/kenya/freedom-net/2023
* Lebanon: https://freedomhouse.org/country/lebanon/freedom-net/2023
* Libya: https://freedomhouse.org/country/libya/freedom-net/2023
* Malaysia: https://freedomhouse.org/country/malaysia/freedom-net/2023
* Myanmar: https://freedomhouse.org/country/myanmar/freedom-net/2023
* Nigeria: https://freedomhouse.org/country/nigeria/freedom-net/2023
* Pakistan: https://freedomhouse.org/country/pakistan/freedom-net/2023
* Philippines: https://freedomhouse.org/country/philippines/freedom-net/2023
* Russia: https://freedomhouse.org/country/russia/freedom-net/2023
* Rwanda: https://freedomhouse.org/country/rwanda/freedom-net/2023
* Saudi Arabia:
https://freedomhouse.org/country/saudi-arabia/freedom-net/2023
* Sri Lanka: https://freedomhouse.org/country/sri-lanka/freedom-net/2023
* Sudan: https://freedomhouse.org/country/sudan/freedom-net/2023
* Thailand: https://freedomhouse.org/country/thailand/freedom-net/2023
* Tunis: https://freedomhouse.org/country/tunis/freedom-net/2023
* Turkey: https://freedomhouse.org/country/turkey/freedom-net/2023
* Uganda: https://freedomhouse.org/country/uganda/freedom-net/2023
* UAE:
https://freedomhouse.org/country/united-arab-emirates/freedom-net/2023
* UK: https://freedomhouse.org/country/united-kingdom/freedom-net/2023
* Uzbekistan: https://freedomhouse.org/country/uzbekistan/freedom-net/2023
* Venezuela: https://freedomhouse.org/country/venezuela/freedom-net/2023
* Zambia: https://freedomhouse.org/country/zambia/freedom-net/2023
* Zimbabwe: https://freedomhouse.org/country/zimbabwe/freedom-net/2023
*## Community activities### OONI training in Thailand for participants from
South Asia*
Between 2nd-4th October 2023, OONI’s Maria co-facilitated a 3-day training
programme in Thailand in collaboration with IODA’s Amanda. This was an
OPTIMA training programme for human rights defenders from South Asia,
designed to share skills and knowledge around measuring internet shutdowns.
As part of this training programme, Maria facilitated the following OONI
workshops:
* Introduction to OONI
* Using OONI Probe and OONI Run for measuring internet censorship
* Updating test lists to improve website censorship testing
* Using OONI Explorer to investigate internet censorship based on real-time
open data
The above workshops included a combination of presentations, live demos,
and hands-on exercises (through which participants made use of the tools
based on real-world scenarios).
In addition to the above, Maria collaborated with Amanda on presenting a
case study (“Internet censorship in Myanmar”) that combines both OONI and
IODA data for investigating internet shutdowns (involving both outages and
blocks). Maria and Amanda also collaborated on facilitating a day-long
workshop through which participants worked on group projects aimed at
investigating internet censorship based on both OONI and IODA data. At the
end of this workshop, participants presented their projects.
*### OONI training in Tbilisi for media managers from Turkey*
On 5th October 2023, OONI’s Elizaveta facilitated a workshop on OONI tools
as part of the training organized by Internews for Turkish media managers.
As part of this workshop, managers learned how to use OONI Run to
coordinate measurement campaigns, how to collect evidence on the blocking
of media tools, and how to use OONI Explorer to investigate the blocking of
news media websites in Turkey.
*### Hackathon at Internet Measurement Conference (IMC) 2023*
On 23rd October 2023, OONI collaborated with ISOC, Censored Planet, and
M-Lab on co-hosting an internet measurement hackathon right before the
Internet Measurement Conference (IMC) 2023. Information about the hackathon
is available here: https://ooni.org/post/2023-imc-hackathon/
As part of the hackathon 3 teams worked on projects which made use of OONI
data. The teams which made use of OONI data worked on the following:
* Using OONI data to investigate IPv6 connectivity, where they used OONI
data to investigate differences and evolution in IPv6 connectivity
worldwide and disaggregated by network type.
* Data triangulation, where they extracted signals for internet shutdowns
by comparing multiple different datasets, including OONI.
* ARIMA based anomaly detection, where they developed a statistical anomaly
detection model to identify anomalous signals in various datasets,
including OONI.
All in all, it was great to see how much was accomplished in such a short
period of time and we hope that these researchers will have learned more
about OONI data and continue to make use of it in the future.
In addition to hosting the hackathon, OONI’s Arturo also attended IMC 2023 (
https://conferences.sigcomm.org/imc/2023/)
*### OONI Community Meeting*
On 31st October 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we discussed the
following topics:
1) Updates from OONI and the OONI community
2) Community feedback for the naming of OONI’s upcoming new “Censorship
Incident Reporting Platform”
3) Community feedback related to re-designing the OONI website
4) Community feedback on OONI’s educational resources: Which are most
useful, and which are missing/needed?
5) IODA updates on internet outages in Gaza
*## Measurement coverage*
In October 2023, 63,123,792 OONI Probe measurements were collected from
3,112 networks in 171 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hello,
This is a kind reminder that OONI will host a monthly community meeting next Tuesday.
=> Where? OONI Slack channel: https://slack.ooni.org/ <https://slack.ooni.org/> (bridged with IRC:
ircs://irc.oftc.net:6697/#ooni <ircs://irc.oftc.net:6697/#ooni>)
=> When? Tuesday, 31st October 2023 at 14:00 UTC (for 1 hour)
Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep <https://pad.riseup.net/p/ooni-community-meeting-keep>
This is a meeting that we host each last Tuesday of the month, and it is an opportunity to:
* Share your feedback on OONI tools & methodologies
* Ask questions about OONI tools and OONI data
* Discuss internet censorship issues, or share the issues you are facing in your country
* Share updates about your work or projects
We hope you can join us!
All the best,
~ OONI team
Hello,
This email shares OONI's monthly report for September 2023.
*# OONI Monthly Report: September 2023*
Throughout September 2023, the OONI team worked on the following sprints:
* Sprint 99 (1st - 10th September 2023)
* Sprint 100 (11th - 24th September 2023)
* Sprint 101 (25th - 30th September 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## New research collaboration*
In September 2023, we formed a new research collaboration with the GLITCH
Research Interest Group (https://glitch.oii.ox.ac.uk/) at the Oxford
Internet Institute, a department of the University of Oxford.
This involves collaboration on the GLITCH Dashboard (on the
interdisciplinary study of information controls) and the exchange of
knowledge and expertise on Internet censorship.
*## Published report on Grindr blocking in Jordan*
In collaboration with My Kali Magazine, we published a new report on the
blocking of Grindr in Jordan ("Grindr blocked in Jordan: Shrinking LGBTQ
spaces").
Read the report here: https://ooni.org/post/2023-jordan-blocks-grindr/
Our report shares OONI data on the ongoing blocking of Grindr in Jordan. As
of 8th August 2023, OONI data shows that access to Grindr (both the website
and app) is blocked by means of TLS interference. According to Grindr, the
block impacts their public health strategy as their Grindr for Equality
program has played an important role in addressing the HIV epidemic in
Jordan.
*## OONI Probe Mobile*
Throughout September 2023, we worked on improvements to the OONI Probe
mobile app. Specifically, on OONI Probe iOS, we worked on bug fixes related
to upload buttons (https://github.com/ooni/probe/issues/2487) and
displaying the “upload measurement” toast (
https://github.com/ooni/probe/issues/2511) On OONI Probe Android, we
worked on supporting the optional grouping of results (
https://github.com/ooni/probe-android/pull/613) and addressing a dashboard
glitch (https://github.com/ooni/probe-android/pull/612) We also worked on
addressing the inconsistent handling of empty OONI Run links on Android and
iOS (https://github.com/ooni/probe/issues/2514)
In preparation for the launch (October 2023), we worked on final
improvements to the “News Media Scan” Android app that we have been
developing in collaboration with Deutsche Welle (DW).
*## Published OONI Probe user guides in Vietnamese*
Thanks to Nathan Tran, the OONI Probe user guides are now available in
Vietnamese!
The Vietnamese user guides are available below:
* OONI Probe Mobile: https://ooni.org/vi/support/ooni-probe-mobile/
* OONI Probe Desktop: https://ooni.org/vi/support/ooni-probe-desktop/
*## Translation of OONI documentation*
Thanks to Nathan Tran, the OONI Frequently Asked Questions (FAQ)
documentation is now available in Vietnamese:
https://ooni.org/vi/support/faq/
Thanks to our Senegalese partner, Computech Institute (
https://ooni.org/partners/computech/) the OONI Risks documentation is now
available in French: https://ooni.org/fr/about/risks/
*## OONI Run*
As part of our work on creating the next generation version of OONI Run
(“OONI Run v.2”), we continued working on adding UI support to OONI Probe
Android for displaying OONI Run v.2 link descriptors based on the API (
https://github.com/ooni/probe-android/pull/582) We also added core support
for OONI Run v.2 to OONI Probe iOS (
https://github.com/ooni/probe-ios/pull/533) As part of this, we added
support for fetch descriptors, save/update descriptors, and for a run
descriptor with existing test suites for running tests (
https://github.com/ooni/probe/issues/2471)
*## OONI Probe CLI*
We continued working towards facilitating A/B comparison of Web
Connectivity v0.4 and Web Connectivity v0.5. To this end, we adapted all
the v0.4 QA tests, which previously used the Jafar tool, to use the new
netem QA framework. We are now well positioned to compare v0.4 and v0.5 in
complex censorship scenarios and make sure there are no regressions (
https://github.com/ooni/probe/issues/2525,
https://github.com/ooni/probe/issues/1803) The changes to OONI Probe CLI
as part of this work can be found here:
https://github.com/ooni/probe-cli/compare/f0f9b5f7...2051a2d3
We redesigned how OONI Probe communicates with the OONI backend and the Web
Connectivity Test Helpers (THs) to increase reliability and performance.
Specifically, we created a specific TLS and HTTPS network stack for
performing these operations that relies on the concept of “beacons”. A
beacon is a known-in-advance IP address that allows communicating with the
backend or the THs and with which OONI Probe can use arbitrary SNI values.
Refactoring OONI Probe CLI to add support for beacons touched upon several
areas of the implementation. Roughly speaking, we can divide this work into
four major activities:
1) Creating a custom TLS dialer and HTTP/HTTPS stack for communicating with
backends and test helpers to avoid mixing measuring-specific and
operational concerns;
2) Recognizing that the availability of a custom stack for these
interactions opens up nearly for free the possibility of using HTTP/HTTPS
proxies for OONI Probe;
3) Adapting the custom TLS dialer and HTTP/HTTPS stack to have a more
complex notion of what it should connect to, which is more complex than
just knowing domain names and the resulting IP addresses, and includes
several “tactics” where each tacting may incorporate knowledge from DNS
lookups, prior knowledge about well-known IP addresses, alternative SNIs to
use, and knowledge about what OONI Probe knows to be working well;
4) Recognising that some properties of the new TLS dialer may, in the
future, cause issues with our probe-IP-address scrubbing policy, and
proactively applying changes to be sure that we are not going to include
any IPv4 or IPv6 addresses and endpoints into the HTTP body and HTTP header
value of submitted OONI measurements.
A more detailed description of this work is available here:
https://github.com/ooni/probe/issues/2531#issuecomment-1743087781
*## Creating a Censorship Incident Reporting Platform*
As part of our ongoing work on the (new) Censorship Incident Reporting
Platform, we refactored the code (https://github.com/ooni/explorer/pull/862)
and moved the new components to our component library (
https://github.com/ooni/design-system/pull/163) We tested the latest
version of the platform to identify further opportunities for improvements.
We also continued to make progress on identifying censorship events (along
with relevant OONI data) to be published as reports on the platform once
launched.
*## OONI backend*
We began adding support for the compression of measurements (
https://facebook.github.io/zstd/) uploaded in the API. This will
significantly improve the measurement submission speed and reduce the
impact of the probe on slow networks. This is especially useful for OONI
Probe users running tests in environments with poor network performance.
We cleaned up a significant amount of disk space on the backend host by
removing query logs. We also created a Jupyter notebook to better
investigate high query load on the database. The txexitmap tool has been
extended to integrate with OONI’s infrastructure (support ClickHouse,
generate metrics and alerts ,etc).
We started updating the backend hosts, CI and development containers to
Debian Bookworm. This is expected to provide security and performance
benefits and newer libraries and tools.
*### Enabling circumvention*
Following the blocking of OONI in China in July 2023 (
https://ooni.org/post/2023-china-blocks-ooni/) we have been working
towards boosting circumvention support and making OONI Probe more resilient
to blocking. As part of this process, we have been evaluating many options
(each with different trade-offs).
We prepared an internal design document describing a probe bootstrap
process using STUN. This will increase the reliability of the probes on
highly censored networks and provide a method for circumvention.
We started deploying the forwarding of "OONI bridges" that provide
circumvention in the event that OONI services are blocked by means of
SNI-based or DNS-based filtering, simplifying the process of probe
bootstrap and measurement submission. They also provide better reliability
for the test helpers by doing active load balancing. We tested load
balancing in Nginx and Haproxy and selected the latter.
*## New ECH experiment contributed to OONI by CIS India*
In September 2023, Divyank Katira from CIS India (
https://ooni.org/partners/cis-india/) published a blog post describing the
new Encrypted Client Hello (ECH) blocking experiment that they contributed
to OONI last year:
https://cis-india.org/internet-governance/blog/detecting-encrypted-client-h…
We worked towards integrating this experiment as part of the experimental
suite of the OONI Probe apps: https://github.com/ooni/probe-cli/pull/1217
*## OONI Community Interviews*
We have an “OONI Community Interviews” series on our YouTube channel (
https://www.youtube.com/watch?v=Pam2UQoZ1qM&list=PL1sH9kYR-16nlPlFT-RDBs8O0…)
where we publish video interviews with OONI community members to highlight
their important work, and the ways through which they use OONI tools and
data.
In September 2023, we continued working on creating several new video
interviews with OONI community members. This involved drafting relevant
interview questions, filming the interviews, and working with a
videographer to edit the footage.
*## Preparing for upcoming OONI trainings in the DRC and Thailand*
In preparation for a week-long OONI training for human rights defenders and
journalists in the DRC, we coordinated with the training organizers on
creating a detailed training agenda that meets the needs of the
participants. We provided live demos of the OONI tools that will be taught
as part of the training, in order to enable live translation during the
upcoming training programme. We also prepared and shared relevant OONI
resources and reading materials for the participants.
Similarly, we also prepared a detailed training agenda, related workshop
slides and hands-on exercises for our upcoming OONI training programme in
Thailand.
*## Planning the OONI Team Meeting 2023*
In September 2023, we continued working on required logistical planning for
the 2023 OONI Team Meeting and we continued to solicit team feedback for
the sessions of the meeting.
*## Planning the OONI Partner Gathering 2024*
In September 2023, we continued to work on fundraising efforts required for
hosting the next OONI Partner Gathering in May 2024.
*## Rapid response efforts### Blocking of TikTok and Google Play Store in
Azerbaijan*
On 19th September 2023 (amid the military offensive in Nagorno-Karabakh),
TikTok users in Azerbaijan reported that they were unable to use the
platform. On the same day, we reported on Twitter that the user reports
were corroborated by OONI data collected from Azerbaijan, which showed that
access to TikTok was blocked on at least 2 networks in the country (
https://twitter.com/OpenObservatory/status/1704160713256628716)
We followed-up to share that in addition to TikTok, OONI data also showed
the blocking of the Google Play Store in Azerbaijan amid the conflict (
https://twitter.com/OpenObservatory/status/1704475063867642269) We shared
all relevant OONI data with advocates from the #KeepItOn campaign (
https://www.accessnow.org/campaign/keepiton/)
*## Community activities### Global Gathering 2023*
Between 15th-17th September 2023, OONI’s Maria and Arturo traveled to
Portugal to attend Team CommUNITY’s Global Gathering event (
https://wiki.digitalrights.community/index.php?title=Global_Gathering)
At the Global Gathering, we participated in the 3-day Summit (during which
we collected community feedback on our upcoming Censorship Alert System)
and we hosted an OONI booth for outreach purposes.
*### Advocacy Assembly Internet Shutdown Mentored Training Program*
On 21st September 2023, OONI’s Elizaveta hosted a session for the
participants of the Advocacy Assembly Internet Shutdown Mentored Training
Program (https://advocacyassembly.org/en/news/236) As part of this
session, she addressed all the OONI-related questions of the training
programme participants.
*### FIFAfrica 2023*
Between 26th September to 2nd October 2023, OONI’s Norbel and Arturo
traveled to Tanzania to participate in the FIFAfrica 2023 conference (one
of the largest annual digital rights conferences in Africa). Information
about the conference is available here: https://internetfreedom.africa/
On 26th September 2023, we facilitated a session (“Measuring Internet
censorship with OONI tools and data”) as part of the Internet Shutdowns
Academy FIFAfrica 2023 workshop (
https://twitter.com/uproar_fyi/status/1706972250157539551) On the same
day, we also facilitated an OONI workshop as part of the Tor Training
Academy (https://twitter.com/zaituni_njovu/status/1706742476596236331)
As part of our participation at the main FIFAfrica 2023 conference, we
facilitated the following 2 sessions:
* Using OONI Run to coordinate the testing of website censorship (28th
September 2023):
https://whova.com/embedded/session/rWwmoMGALzfjDBkAPdWzA1K4LxMjt38D6BmWiph4…
* Measure All The Things! Using Open Data to Investigate Internet
Shutdowns, A Primer on Network Interference Research (29th September 2023):
https://whova.com/embedded/session/rWwmoMGALzfjDBkAPdWzA1K4LxMjt38D6BmWiph4…
*### OONI Community Meeting*
On 26th September 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we discussed the
following topics:
1) Updates from the OONI team
2) Discussion on recommended reading materials in the field of network
measurement and how to gain deeper knowledge on the impacts of internet
censorship
*## Measurement coverage*
In September 2023, 61,542,643 OONI Probe measurements were collected from
3,012 networks in 175 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
