ooni-talk
Threads by month
- ----- 2025 -----
- June
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
January 2023
- 2 participants
- 6 discussions
Hi everyone!
This is a reminder that the monthly OONI Community Meeting is taking place tomorrow.
Where: https://slack.ooni.org/ <https://slack.ooni.org/> (OONI Slack channel)
When: Tuesday, 31st January 2023 at 14:00 UTC (for 1 hour)
Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep <https://pad.riseup.net/p/ooni-community-meeting-keep>
Thanks,
Elizaveta
1
0
Hello,
Happy new year and hope you had a great holiday season!
The OONI team warmly welcomes you to join us next Tuesday for our monthly community meeting.
=> Where? OONI Slack channel: https://slack.ooni.org/ <https://slack.ooni.org/> (bridged with IRC:
ircs://irc.oftc.net:6697/#ooni)
=> When? Tuesday, 31st January 2023 at 14:00 UTC (for 1 hour)
Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep &…
[View More]lt;https://pad.riseup.net/p/ooni-community-meeting-keep>
The monthly OONI community meetings aim to:
* Collect community feedback on OONI tools & methodologies
* Address questions in relation to the use of OONI tools and OONI data
* Foster discussions on internet censorship issues
* Receive updates from the community
We hope you can join us!
All the best,
~ OONI team
[View Less]
1
0
Hello friends,
The OONI team is excited to share that today we published a *new screencast
for OONI Run*: https://www.youtube.com/watch?v=OGRN7ve6cIA
OONI Run (https://run.ooni.io/) is a web platform that you can use to
generate mobile deep links and widget code to coordinate OONI Probe website
testing.
Specifically, you can use OONI Run to create links or widget code for:
- Testing websites (of your choice) for censorship;
- Coordinating website censorship testing with other OONI Probe …
[View More]users.
Please note that OONI Run links only work with the OONI Probe mobile app (
https://ooni.org/install/mobile)
We hope you find the screencast helpful for using the OONI Run platform and
coordinating censorship testing around the world!
Thanks,
Maria.
[View Less]
1
0
Hello,
Below I share OONI's status report for December 2022.
*# OONI Monthly Report: December 2022*
Throughout December 2022, the OONI team worked on the following sprints:
* Sprint 79 (1st-4th December 2022)
* Sprint 80 (5th-18th December 2022)
* Sprint 81 (19th-31st December 2022)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Year in Review: OONI in 2022*
We published our annual blog post, "…
[View More]Year in Review: OONI in 2022", which
shares:
* OONI highlights from 2022
* Highlights from the OONI community (including many of our partners)
* Our plans for 2023
You can read this post here: https://ooni.org/post/ooni-in-2022/
*## OONI’s 10th Anniversary*
On 5th December 2022, OONI celebrated its 10th anniversary! To mark this
milestone, we organized and hosted 2 live-streamed events, published a
timeline animation, community video, and blog post. Further details are
shared below.
*### Live-streamed events*
In celebration of OONI’s 10th anniversary, we organized and hosted 2
live-streamed events (https://ooni.org/post/10th-ooniversary-events/) on
our YouTube channel:
* 10th Ooniversary: OONI Highlights (5th December 2022)
* 10th Ooniversary: OONI Community (6th December 2022)
As part of the “OONI Highlights” live-streamed event, Arturo (OONI’s
founder) shared OONI’s history and highlights from the past 10 years, as
well as our plans for the future. This presentation can be viewed here:
https://www.youtube.com/watch?v=f4D4gq9TBMg
As part of the “OONI Community” live-streamed event, we invited 5 community
members who shared how they’ve used OONI tools and data as part of their
research and advocacy efforts (
https://ooni.org/post/10th-ooniversary-events/#10th-ooniversary-ooni-commun…)
Their presentations can be viewed here:
https://www.youtube.com/watch?v=Co_f8KPBIZM
Both events included presentations followed by discussion addressing
questions shared by participants through the live chat and shared pads.
On 5th December 2022, we also hosted an online Distance Disco (
https://distancedisco.nl/dd/ooni-s-10th-birthday-party) to dance with
community members in celebration of OONI’s 10th anniversary.
*### New animation of OONI highlights*
On 5th December 2022, we published an animation that provides a timeline of
OONI highlights from the past 10 years. This animation was produced in
collaboration with Robotina (https://www.robotina.it/)
The animation is available here: https://www.youtube.com/watch?v=AjpfF2kQ9jA
*### New OONI Community Video*
In celebration of OONI’s 10th Anniversary, we published a new video with
OONI community members.
You can watch this video here: https://www.youtube.com/watch?v=zrclQ2QZjVo
The video features 11 community members discussing how OONI has been useful
to their work, and what they would like to see OONI do in the future.
You can learn about the featured community members through our blog post:
https://ooni.org/post/ooni-community-video/
*### Highlights: 10 Years of OONI*
To mark OONI’s 10-year anniversary, we published a blog post where we
document the key OONI highlights from the past 10 years, as well as some of
our future plans.
This blog post is available here:
https://ooni.org/post/highlights-10-years-of-ooni/
*## OONI Probe Mobile*
We released OONI Probe Android 3.7.3 (
https://github.com/ooni/probe-android/releases/tag/v3.7.3) and OONI Probe
iOS 3.73 (https://apps.apple.com/us/app/id1199566366) with bug fixes and
improvements. We worked towards ensuring that OONI Probe Mobile uses the
check-in API instead of the test list API (
https://github.com/ooni/probe/issues/2390) we fixed broken Android tests (
https://github.com/ooni/probe/issues/2386) and we worked towards ensuring
that the testing of OONI Run links does not stop if a device is locked (
https://github.com/ooni/probe/issues/2361)
*## Published a new OONI Run user guide*
To enable community members to use the OONI Run (https://run.ooni.io/)
platform for custom website testing (and for coordinating censorship
measurement campaigns around the world), we created and published a new
user guide for OONI Run.
Our OONI Run user guide can be accessed here:
https://ooni.org/support/ooni-run/
Through the OONI Run user guide, you can learn how to:
* Create OONI Run links for testing websites (of your choice) for
censorship;
* Use OONI Run links with the OONI Probe mobile app;
* Share OONI Run links with other OONI Probe mobile app users;
* Generate OONI Run widget code.
Upon reading this guide, we hope you will feel empowered to share your OONI
Run knowledge and skills with others! We also hope this guide helps with
coordinating OONI Probe website testing around the world.
*## OONI Probe CLI*
We released OONI Probe CLI 3.16.6 (
https://github.com/ooni/probe-cli/releases/tag/v3.16.6) and OONI Probe
3.16.7 (https://github.com/ooni/probe-cli/releases/tag/v3.16.7) These two
releases include improvements to the experimental Web Connectivity v0.5
nettest, as well as the following important fixes:
* In OONI Probe CLI 3.16.6, we included the reliability fix (developed in
November) that makes OONI Probe try all the available test helpers, rather
than just the first one in the list;
* In OONI Probe CLI 3.16.7, we backported fixes for the WhatsApp and
Telegram experiments that stop measuring unencrypted web endpoints to avoid
possible false positives (an issue discussed in detail here:
https://github.com/ooni/ooni.org/issues/1317)
We continued the process of merging ooniprobe (the official CLI client) and
miniooni (the experimental CLI client) by introducing a common database
abstraction that could be used by both in the following pull request:
https://github.com/ooni/probe-cli/pull/988
*## OONI Probe Desktop*
We released OONI Probe Desktop 3.8.3 to include fixes from OONI Probe CLI
3.16.7 https://github.com/ooni/probe-desktop/releases/tag/v3.8.3
*## Expanding OONI’s testing model to support richer testing input*
To improve the quality of OONI measurements (and reduce the risk of false
positives), we stopped measuring unencrypted web endpoints for WhatsApp and
Telegram (https://github.com/ooni/ooni.org/issues/1317) We continued our
refactoring work to support providing richer testing input (
https://github.com/ooni/ooni.org/issues/1291,
https://github.com/ooni/ooni.org/issues/1292,
https://github.com/ooni/probe/issues/2362)
During December 2022, we focused on cleaning up “probeservices” (the Go
package that communicates with the OONI API) to set the stage for
implementing richer input. We removed support for deprecated APIs (
https://github.com/ooni/probe/issues/2380) we started to sync up the data
format with the backend (https://github.com/ooni/probe-cli/pull/996) and
we added support in the backend and in ooniprobe for receiving compressed
responses (https://github.com/ooni/probe/issues/2379) The latter change,
in particular, is required because adding more testing options to each URL
significantly increases the size of the response message. We then annotated
code to communicate with the backend with the request and response message
types (https://github.com/ooni/probe-cli/pull/1010,
https://github.com/ooni/probe-cli/pull/1011) This change allows us to
generate a swagger describing ooniprobe’s view of the API, which we can
compare to the one generated by the server, to be sure the two
implementations are (and stay) in sync.
Regarding richer input, we started sketching out a design document (
https://github.com/ooni/probe/issues/2381) and a prototype (
https://github.com/ooni/probe-cli/pull/1005) implementing this design
document. The general idea is to give experiments freedom to call the
specific backend API they require for getting their inputs, which removes
the need to maintain middleware code that delivers inputs to the
experiments. With more input diversity, such a middleware code becomes
increasingly the bottleneck to innovation, so we want to experiment with
removing it and adding additional functions that experiments can call to
implement common functionality instead.
We also added support to the Web Connectivity Test Helper (TH) for
measuring HTTP/3 hosts (https://github.com/ooni/probe-cli/pull/985)
Support for HTTP/3 in the TH is crucial to start measuring websites using
HTTP/3 in the Web Connectivity experiment.
Reflecting on our own experience writing experiments in the step-by-step
style (
https://github.com/ooni/probe-cli/blob/master/docs/design/dd-003-step-by-st…)
as well as by learning from users’ contributions (
https://github.com/ooni/probe-cli/pull/989#discussion_r1032544878) we
determined that the step-by-step API provides the basic building blocks for
writing all the experiments we need, but there is also a need to have more
user-friendly and intuitive wrappers on top of it. To this end, we started
experimenting with a simple domain specific language for expressing and
composing low-level operations that should simplify the writing of
experiments (https://github.com/bassosimone/oonidsl/)
*## OONI Explorer*
We started working towards integrating relevant third-party data into OONI
Explorer. Specifically, we worked on third-party data visualizations on
country and network pages using the internet outage data from IODA and
internet traffic data from Cloudflare. We also continued conducting UX
research to improve the OONI Explorer measurement pages (
https://6q8h6s64.optimalworkshop.com/optimalsort/avnrhnv5/sort)
We added support for choosing the time granularity on the Measurement
Aggregation Toolkit (MAT), beyond day granularity (which has been the only
option so far). Users can now also display the data per hour, week and
month (https://github.com/ooni/explorer/pull/818)
*## Published an OONI Explorer demo*
To enable community members to use OONI Explorer (https://explorer.ooni.org/)
for investigating internet censorship based on OONI data, we published an
OONI Explorer demo on our YouTube channel.
The OONI Explorer demo is available here:
https://www.youtube.com/watch?v=6Rce-xshLac
*## OONI backend*
In response to a change in the testing of WhatsApp that made our tests
fail, we implemented a temporary workaround that restores the functionality
of the test (https://github.com/ooni/pipeline/pull/403)
We added support for HTTPS to the deb.ooni.org Debian package repository
after it was requested by the community to circumvent blocking of the HTTP
version. We also moved the service to the new backend host.
We enabled gzip support in Nginx to compress responses from the check-in
API and other JSON content and created a simple dashboard to monitor the
improvement. This makes the responses from the Measurement Aggregation
Toolkit (MAT) quicker for the browser to download.
*## Automating censorship detection and characterization based on OONI
measurements*
As part of our ongoing effort to automate and improve our characterization
of censorship events through the analysis of OONI measurements, we made
significant progress on the ooni/data tool (https://github.com/ooni/data)
Following internal technical discussions, as well as conversations with
members of the OONI community, we implemented a new iteration of the data
model used for representing the results of experiments (
https://github.com/ooni/data/pull/19)
Moreover, we started the process of evaluating the new analysis engine
against the analysis that currently exists and is mostly performed as part
of the measurement workflow (https://github.com/ooni/data/issues/21) Some
very preliminary findings suggest that it’s quite an important improvement
both in terms of accuracy and recall.
We also experimented with producing plots that breakdown the means through
which blocking is implemented (https://github.com/ooni/data/issues/22)
Smaller incremental improvements include:
Handling the deduplication of already processed data to support the easier
reprocessing of data (
https://github.com/ooni/data/pull/19/commits/eea26f7138b14fc4595d55b432d287…)
Expanding cloud provider detection to reduce DNS false positives (
https://github.com/ooni/data/pull/19/commits/01dc691d695dfc103d604846ad7568…)
Refactoring of the DNS anomaly analysis subsystem (
https://github.com/ooni/data/pull/20)
We also started implementing the extraction of the blocking type in the
fastpath pipeline.
*## Creating a Social Media Censorship Alert System*
We continued to work on the development of the Social Media Blocking Event
detector. We started running a prototypal version, created a dashboard and
a jupyter notebook. We ran the prototype for weeks to review the initial
results and improve the algorithm over time.
*## Published Test Lists Editor screencast*
To enable community members to contribute websites for censorship testing,
we created and published a new screencast for our Test Lists Editor (
https://test-lists.ooni.org/)
The Test Lists Editor screencast can be viewed here:
https://www.youtube.com/watch?v=6i2OVHUQEpE
Through our Test Lists Editor, you can review and contribute websites for
censorship testing by OONI Probe users around the world.
*## TikTok blocked in Jordan*
On 16th December 2022, Jordan blocked access to TikTok amid fuel protests.
We rapidly responded by sharing relevant OONI data, charts, and information
about the block on Twitter:
https://twitter.com/OpenObservatory/status/1603845292343889924
*## Test list updates*
Following the blocking of TikTok in Jordan, we added TikTok endpoints to
the test list for Jordan (https://github.com/citizenlab/test-lists/pull/1201)
Thanks to our URL prioritization system, these newly added URLs were
immediately prioritized for testing in Jordan as soon as the pull request
was merged (enabling the immediate collection of relevant measurements and
supporting rapid response efforts).
Throughout December 2022, we also reviewed and merged many other updates to
test lists contributed by community members:
https://github.com/citizenlab/test-lists/pulls?q=is%3Apr+is%3Aclosed
*## Creating a new OONI Outreach Kit*
Based on review and feedback, we continued to make improvements to the
materials and resources of the upcoming OONI Outreach Kit. Specifically, we
made extensive edits and improvements to the set of OONI workshop slides
included in the OONI Outreach Kit.
*## Updated OONI Partner pages*
On our website, we have an “OONI Partners” section which features our
partners, their important work, and the specific projects and research
reports that we’ve collaborated on (https://ooni.org/partners)
We updated many OONI Partner pages to include our latest collaborations:
https://github.com/ooni/ooni.org/pull/1328
*## Cloudflare blog post *
Cloudflare published a blog post describing how civil society organizations
use Cloudflare Radar (https://radar.cloudflare.com/) to track internet
shutdowns around the world:
https://blog.cloudflare.com/partnering-with-civil-society-to-track-shutdown…
This blog post highlights OONI, where we share how Cloudflare Radar has
been useful to our work.
*## Community use of OONI data### iMAP 2022: Research reports on internet
censorship in 8 Asian countries*
On 21st December 2022, our long-term partner, Sinar Project, published 8
new research reports on internet censorship in Southeast Asia and Hong Kong
(China) in collaboration with their Internet Monitoring and Action Project
(iMAP) partners (https://imap.sinarproject.org/) Based on the analysis of
OONI data, these research reports investigate internet censorship in
Malaysia, Thailand, Myanmar, Indonesia, Cambodia, Vietnam, Hong Kong, and
the Philippines.
Their research reports can be found here:
https://imap.sinarproject.org/reports/2022
Our iMAP partners launched and presented these 8 research reports through
the following live-streamed presentation:
https://www.youtube.com/watch?v=u9JsCqXebPI
To highlight and help disseminate their important work, we published a blog
post which briefly discusses and links to their 8 research reports:
https://ooni.org/post/2022-imap-8-research-reports-southeast-asia/
*## Community activities### Internet Governance Forum (IGF) 2022*
On 1st December 2022, OONI’s Maria presented OONI on the panel “Help! The
Kill switch is taking away my limited agency” at the Internet Governance
Forum (IGF) 2022 (
https://intgovforum.org/en/content/igf-2022-town-hall-54-help-the-kill-swit…)
*### OONI workshop for journalists in Kyrgyzstan*
On 20th December 2022, OONI’s Elizaveta facilitated an online OONI workshop
for journalists in Kyrgyzstan as part of a training organized by Internews.
During this workshop, Elizaveta explained how journalists can use OONI
Probe and OONI data to investigate internet censorship.
*## Userbase*
In December 2022, 57,009,977 OONI Probe measurements were collected from
2,815 AS networks in 161 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
[View Less]
1
0
Hello,
Below I share OONI's status report for November 2022.
*# OONI Monthly Report: November 2022*
Throughout November 2022, the OONI team worked on the following sprints:
* Sprint 77 (1st-6th November 2022)
* Sprint 78 (7th-20th November 2022)
* Sprint 79 (21st-30th November 2022)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Technical multi-stakeholder research report on Internet shutdowns …
[View More]in
Iran*
In collaboration with IODA, M-Lab, Cloudflare, Kentik, Censored Planet,
ISOC, and Article19, OONI published a new research report: "Technical
multi-stakeholder report on Internet shutdowns: The case of Iran amid
autumn 2022 protests".
Read the report here:
https://ooni.org/post/2022-iran-technical-multistakeholder-report/
This report combines multiple measurement datasets in an attempt to
investigate the internet shutdown events in Iran over the past months amid
protests. We consider this a particularly important research report, as
it’s likely the first ever report that combines so many internet
measurement datasets in investigating a case study.
Specifically, we provide data and analysis on the following in Iran:
* Mobile network outages
* Regional outages
* HTTP/3 and QUIC traffic drop
* IPv6 disruption on Irancell
* Increased blocking of encrypted DNS
* Impact on internet speed tests
* Blocking of WhatsApp, Instagram, Skype, Linkedin, and Viber
* Blocking of Google Play Store and Apple App Store
* Blocking of browser extension repositories
ISOC also shared their analysis on examining the economic impact of the
shutdowns.
We provide a summary of the key findings here:
https://ooni.org/post/2022-iran-technical-multistakeholder-report/#summary-…
ISOC also published a blog post about the report:
https://pulse.internetsociety.org/blog/technical-multistakeholder-report-on…
*## OONI Team Meeting*
Between 2nd-4th November 2022, we organized and hosted a 3-day OONI Team
Meeting in Rome, Italy.
This was the first in-person OONI Team Meeting since the start of the
COVID-19 pandemic. Unfortunately not everyone was able to join us in person
in Rome, and so we facilitated a hybrid meeting, with some participants in
person, and others joining us remotely online. In addition to OONI’s 9 team
members, the meeting also included the 2 OTF Information Controls Research
Fellows (Ain Ghazal and Gurshabad Grover) that we’re currently hosting.
During this 3-day meeting, we improved our roadmaps and discussed strategic
priorities and implementation details for many new systems we’ll be
building over the next few years.
Specifically, the 3-day OONI Team Meeting included the following sessions:
* Time for reflection
* Community needs and priorities
* Expanding OONI’s testing model to support richer testing input
* Incrementally integrating Websteps into Web Connectivity LTE
* Measuring the blocking of VPNs
* Research on Internet censorship in South Asia and analysis of “failed”
measurements
* Creating a methodology for measuring throttling: Next steps
* Automating the detection and characterization of censorship
* Creating a Social Media Censorship Alert System
* Presenting thematic censorship findings on OONI Explorer
* Creating a Censorship Incident Reporting Platform
* Aligning the anomaly calculations between the pipeline, Explorer, and
probe
* Handling of HTTP response bodies in measurements
* Triage and prioritization of issues
* Roadmapping sessions
* Fundraising priorities
During the OONI Team Meeting, we also had an OONI-themed cake to celebrate
OONI’s 10th Anniversary together (
https://ooni.org/post/ooni-in-2022/images/01.png) After the 3-day OONI
Team Meeting, we had a “team bonding day” together in Rome.
*## Organizing OONI’s 10th Anniversary*
In preparation for OONI’s 10th Anniversary (on 5th December 2022), we
worked towards organizing live-streamed events and creating multimedia
content to mark the important milestone.
In November 2022, we published a blog post which invites the internet
freedom community to attend 2 live-streamed events for OONI’s 10th
Anniversary: https://ooni.org/post/10th-ooniversary-events/
Throughout November 2022, we interviewed 11 OONI community members and
worked with a videographer on creating a video that highlights how OONI has
been useful to the internet freedom community. We also wrote a script for
an animation that provides a timeline of key OONI highlights from the past
10 years, and we worked with animators on producing an animation. Both the
community video and animation were scheduled for publication around OONI’s
10th Anniversary in early December 2022.
*## OONI Probe Mobile*
In November 2022, we released OONI Probe Android 3.7.2 (
https://github.com/ooni/probe-android/releases/tag/v3.7.2) with bug fixes
and improvements. We worked towards ensuring that flutter implementations
are updated properly on all platforms and we started making progress on
unsupported platforms (https://github.com/ooni/probe-cli/pull/880) We also
worked on triaging issues reported in Sentry.
*## OONI Run*
We edited the copy on the OONI Run platform:
https://github.com/ooni/run/pull/118
*## OONI Probe CLI*
We released OONI Probe CLI 3.16.4 (
https://github.com/ooni/probe-cli/releases/tag/v3.16.4) and v3.16.5 (
https://github.com/ooni/probe-cli/releases/tag/v3.16.5) to fix issues with
the Signal nettest caused by Signal changing the certification authority it
uses to authenticate its endpoints.
Additionally, we started merging ooniprobe (the official command line
client) and miniooni (the research command line client) by abstracting
common functionality into shared Go packages to be used by both tools. This
work is documented through the following pull requests:
https://github.com/ooni/probe-cli/pull/979 and
https://github.com/ooni/probe-cli/pull/982.
Notably, we added support to ooniprobe for a `--proxy` command line
argument, thus facilitating the use of circumvention tools (such as tor):
https://github.com/ooni/probe-cli/pull/981
We also made the code that communicates with the Web Connectivity Test
Helper more reliable by trying all the available Test Helpers, rather than
just using the first one returned by the backend API. This work is
available through the following pull request:
https://github.com/ooni/probe-cli/pull/980
*## OONI Probe Desktop*
We released OONI Probe Desktop 3.8.2 which includes Signal nettest fixes
from OONI Probe CLI:
https://github.com/ooni/probe-desktop/releases/tag/v3.8.2
*## Published an OONI Probe Desktop screencast*
To enable community members worldwide to use OONI Probe, we created an OONI
Probe Desktop screencast which we published on our YouTube channel.
Our OONI Probe Desktop screencast is available here:
https://www.youtube.com/watch?v=tLDVpyHFsW0
*## Russian translation of the OONI Probe Mobile and Desktop User Guides*
We translated the OONI Probe user guides to Russian, and published the
translated versions:
* OONI Probe Mobile: https://ooni.org/ru/support/ooni-probe-mobile
* OONI Probe Desktop: https://ooni.org/ru/support/ooni-probe-desktop
*## Expanding OONI’s testing model to support richer testing input*
We started refactoring work to support providing richer input to OONI Probe
tests (https://github.com/ooni/ooni.org/issues/1291) This work stems from
the observation that OONI experiments require input that is more complex
than a URL in several cases. For example, to correctly test HTTP/3 websites
we would need to know whether a website supports HTTP/3. We started off by
refactoring the signature of OONI experiment’s “Run” function to simplify
the changing of arguments passed to each experiment (
https://github.com/ooni/probe-cli/pull/983) without having to refactor ~30
Go packages each time.
*## OONI Explorer*
We continued to work on the user feedback reporting mechanism (
https://github.com/ooni/explorer/pull/790) focusing on usability aspects
with the help of a UI/UX designer. In order to improve the current OONI
Explorer interface and plan new features, we started conducting UX research.
We added support for RTL languages and localized date formatting across
OONI Explorer as part of localization work (
https://github.com/ooni/explorer/pull/805) We also fixed the back
navigation on pages that include filter params in the URL (
https://github.com/ooni/explorer/pull/815) and we added support for
filtering by anomalies (https://github.com/ooni/explorer/pull/816) on the
Measurement Aggregation Toolkit (MAT).
*## OONI backend*
We started reviewing the security report from Cure53 (who conducted an
independent security audit of our software) and we deployed different
mitigations based on the feedback we received. We implemented Bearer Token
in the API as an improvement over the previous security cookie (
https://github.com/ooni/api/pull/313) We also implemented a mitigation to
a minor security issue affecting parameter validation in the API
documentation path "/apidocs".
We deployed a fix to hide Clickhouse error traces from the API output for
improved security. We also carried out some tests to compress ooniprobe
binaries using UPX. We updated the fastpath pipeline (
https://github.com/ooni/pipeline/pull/401) to support the updated version
of the Signal test.
*## OONI Test Lists Editor*
Following the backend changes to authentication, the cookie based
authentication was replaced by a Bearer Token one in the Test Lists Editor:
https://github.com/ooni/test-lists-ui/pull/68
*## Automating censorship detection and characterization based on OONI
measurements*
We created a new data analysis tool (https://github.com/ooni/data) for
exposing anomaly details of website measurements and characterizing website
blocking. Some of the building blocks for designing this tool were
previously applied in practice as part of our investigation into emergent
website blocks in Russia (
https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/) where we
were able to characterize the blocking methods across ISPs (
https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/#blocked-websi…)
The high level architecture (
https://github.com/ooni/data#architecture-overview) of this new data
analysis tool is that of transforming the raw network measurements coming
from OONI Probes into normalized and post-processed “Observations” (
https://github.com/ooni/data#observation-generation) These Observations
are “time stamped” statements about some network condition that was
observed by a particular vantage point. These observations are then in turn
post-processed to generate what we call an “Experiment result” (
https://github.com/ooni/data#experimentresult-generation) which attributes
to a set of observations from an individual test run a series of outcomes
with an associated level of confidence.
In order to reach a design that would be flexible enough to be expanded,
but also performant enough to run in a reasonable amount of time, we
implemented several prototype iterations of this design (
https://github.com/ooni/data/pull/15) Throughout the process we carried
out benchmarks of key components to identify any performance bottlenecks,
working to resolve them (https://github.com/ooni/data/issues/4) The last
iteration (https://github.com/ooni/data/pull/17) based on some cursory
benchmarks (https://github.com/ooni/data/pull/17#issue-1452100512) should
be able to generate observations from raw OONI measurements in less than a
week and re-generate all experiment results in about a day, running on a
single machine.
Through this data analysis tool (which anyone can run on their own computer
pointing it at the raw public OONI dataset:
https://github.com/ooni/data#developer-setup) it’s possible to gain a much
deeper understanding about the root cause of a blocking event, and to carry
out more advanced research that might not be captured by the existing
analysis.
The new design is an important shift in how we view results from OONI
measurements and we are confident that it will allow us to characterize
blocks in a much richer way. It also opens the door for empowering
researchers to carry out more advanced censorship investigations using our
datasets.
*## Creating a Social Media Censorship Alert System*
To enable human rights defenders to rapidly respond to social media blocks,
we aim to create a Social Media Censorship Alert System which allows access
to real-time OONI data on the blocking of major social media platforms
around the world.
In November 2022, we started the initial research process for designing
this new platform, which involved experimenting with implementations for
social media blocking event detection. Specifically, we experimented with
different designs to provide lightweight event detection without impacting
the backend infrastructure.
As part of this initial experimentation, we applied analysis logic for
issuing an “alert” when anomalies are present for selected social media
targets (the initial experiment involved Twitter and Facebook). We set the
thresholds for issuing “alerts” based on moving averages, as the volume and
frequency of relevant measurements varies significantly from ASN to ASN,
and country to country over time.
Based on our experimentation, we created Jupyter Notebook charts to
investigate ways to classify groups of measurements as blocking or
unblocking events.
*## Social media blocked in Turkey*
On 13th November 2022, following the bomb explosion in Istanbul, some ISPs
in Turkey blocked access to Twitter, Instagram, Facebook, and YouTube.
We responded by sharing real-time OONI data collected from Turkey (along
with a MAT chart), documenting the blocks:
https://twitter.com/OpenObservatory/status/1591871076677701635
Our tweet was cited in articles published by:
* Global Voices:
https://globalvoices.org/2022/11/13/turkey-throttles-internet-access-follow…
* Coda Story:
https://www.codastory.com/newsletters/apple-beijing-turkey-istanbul-blast/#…
*## Test list updates*
In collaboration with community members, we updated the test list for
Kazakhstan (https://github.com/citizenlab/test-lists/pull/1182) We also
reviewed and merged updates to the test list for Malaysia (
https://github.com/citizenlab/test-lists/pull/1175) contributed by our
partner, Sinar Project.
*## Community use of OONI tools and data### OONI censorship measurement
campaign during Malaysia’s 2022 general elections*
Leading up to and during Malaysia’s 2022 elections, our partner, Sinar
Project, coordinated an OONI Probe measurement campaign (using OONI Run
widgets for custom testing) to monitor potential censorship events.
Information about the censorship measurement campaign, along with OONI Run
widgets for relevant testing (including election monitoring and political
party websites, among others), was published on their website:
https://sinarproject.org/digital-rights/measuring-and-detecting-network-int…
Sinar Project also shared relevant OONI Probe testing instructions through
this Twitter thread:
https://twitter.com/sinarproject/status/1592879196610920448
*### OONI censorship measurement testing during Kazakhstan’s 2022 elections*
Leading up to Kazakhstan’s 2022 elections, we collaborated with local
groups on creating a custom list of websites that are most relevant to test
in Kazakhstan for censorship.
To enable the testing of such sites during the elections, we embedded an
OONI Run widget (https://github.com/ooni/ooni.org/pull/1304) into our
website: https://ooni.org/get-involved/run (the “Kazakhstan (38 URLs)”
widget)
We shared relevant OONI Probe testing instructions on Twitter:
https://twitter.com/OpenObservatory/status/1593988313932828675
We also wrote documentation providing relevant OONI Probe testing
instructions for community members in Kazakhstan.
*### Global Voices blog post on the blocking of a website that digitizes
books in Mexico*
Global Voices published a blog post documenting the blocking of a website
run by a Mexican collective that digitizes books as a political position.
This post cites OONI data on the block.
Their blog post is available here:
https://globalvoices.org/2022/11/30/mexican-collective-pirating-books-to-ma…
*## Community activities### OONI workshop for civil society in Kazakhstan*
On 3rd November 2022, OONI’s Elizaveta facilitated an online OONI workshop
for civil society groups in Kazakhstan.
Information about the workshop is available here:
https://www.facebook.com/medianetkaz/posts/pfbid02nhnfrTotir9yaeZA3EJtG64oi…
*### OONI presentations at IETF 115*
On 8th November 2022, OONI’s Simone presented our censorship measurements
in Iran as part of the Internet Architecture Board (IAB) open meeting
during IETF 115 (https://www.ietf.org/live/ietf115-iab-open/) His IAB Open
presentation can be viewed here:
https://www.youtube.com/watch?v=7_G6XVjlun8&t=1730s
On 9th November 2022, OONI’s Simone presented our strategies for measuring
the blocking and throttling of encrypted protocols (including HTTPS, DNS
over HTTPS, and HTTP/3) as part of the Privacy Enhancements and Assessments
Research Group (PEARG) meeting during IETF 115 (
https://datatracker.ietf.org/meeting/115/session/pearg) His PEARG
presentation can be viewed here:
https://www.youtube.com/watch?v=NO3pw3_hees&t=3592s
*### OONI workshop for journalists in Central Asia*
On 10th November 2022, OONI’s Maria facilitated an online OONI workshop for
journalists in Central Asia.
*### OONI presentation at State of the Onion 2022*
On 16th November 2022, OONI’s Maria presented OONI highlights from 2022, as
well as upcoming OONI projects for 2023 as part of the Tor Project’s annual
State of the Onion event (
https://blog.torproject.org/state-of-the-onion-2022/)
The live-streamed event can be viewed here:
https://www.youtube.com/watch?v=O-7k0PjnBbk
*### OONI presentation at DataFest 2022*
On 17th November 2022, OONI’s Elizaveta presented OONI at DataFest 2022 in
Tbilisi (https://www.datafest.ge/schedule) As part of this presentation,
Elizaveta explained how journalists can use OONI’s Measurement Aggregation
Toolkit (MAT) to investigate internet censorship around the world.
*### OONI datathon at DataFest 2022*
On 19th November 2022, we facilitated an OONI datathon as part of DataFest
2022 in Tbilisi ((https://www.datafest.ge/datathons) During the OONI
datathon, participants worked on analyzing OONI data collected from Russia,
Kazakhstan and Uzbekistan to investigate local internet censorship.
*### OONI presentation at Internet Without Borders conference*
On 18th November 2022, OONI’s Arturo presented censorship measurement
findings from Russia as part of the Internet Without Borders conference in
Paris (
https://web.archive.org/web/20221101023014/https://internetborders.net/conf…)
*### OONI hackathon at Internet Without Borders conference*
On 19th November 2022, we facilitated an OONI hackathon as part of the
Internet Without Borders conference in Paris (
https://internetborders.net/ooni-challenge/) During the hackathon, a
participant wrote new SMTP(s), IMAP(s) (
https://github.com/ooni/probe-cli/pull/989) and bittorrent experiments (
https://github.com/ooni/probe-cli/pull/986) for OONI Probe!
*### OONI workshops in Kyrgyzstan*
Between 24th–26th November 2022, OONI’s Elizaveta traveled to Kyrgyzstan to
facilitate 5 in-person OONI workshops for local journalists, students and
civil society organizations. As part of these workshops, Elizaveta
explained how they can use OONI Probe to collect internet measurements and
how they can use OONI data to investigate internet censorship.
*### OONI workshop in Kazakhstan by Access Now*
On 18th November 2022, Access Now organized an offline event for elections
observers in Kazakhstan. As part of this event, Anastasia Zhyrmont from
Access Now facilitated an OONI workshop and explained how election
observers can contribute to collecting internet measurements during the
elections.
*### OONI Community Meeting*
On 29th November 2022, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we discussed the
following topics:
1) Strategies for monitoring potential social media blocks during
Indonesia's 2024 elections. Discussion of how blocks are implemented in
Indonesia and improving the data quality of OONI measurements.
2) Requesting feedback on the type of materials that community members
would find most useful and would like to see in the upcoming OONI Outreach
Kit.
*## Userbase*
In November 2022, 61,907,164 OONI Probe measurements were collected from
2,851 AS networks in 166 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
[View Less]
1
0
Hello,
Happy New Year!
We'd like to share updates from the OONI team over the last 3 months. Below
I share our October 2022 report, and I'll follow-up to share our November
and December 2022 reports.
*# OONI Monthly Report: October 2022*
Throughout October 2022, the OONI team worked on the following sprints:
* Sprint 75 (1st-9th October 2022)
* Sprint 76 (10th-23rd October 2022)
* Sprint 77 (24th-31st October 2022)
Our work can be tracked through the various OONI GitHub repositories:
…
[View More]https://github.com/ooni
Highlights are shared in this report below.
*## New OONI Team member*
In October 2022, Kathrin Elmenhorst joined the OONI team! Kathrin will work
on improving the data quality of OONI measurements and adding support to
OONI Probe for new experiments.
*## Launch of Advocacy Assembly OONI training course*
On 30th September 2022, Small Media’s Advocacy Assembly launched our online
OONI training course (
https://advocacyassembly.org/en/courses/63/#/chapter/1/lesson/1)
Following the launch, we published a blog post to share the course with our
community:
https://ooni.org/post/2022-ooni-training-course-advocacy-assembly/
This 90-minute free, online training course provides a deep-dive into
measuring internet censorship with OONI Probe, and using OONI Explorer to
access and interpret real-time OONI data collected from around the world.
Designed for human rights defenders, activists, journalists, and
researchers, the course includes a mix of videos, screencasts, slides,
quizzes and hands-on exercises. It also features case study videos from
OONI community members, and the course is available in English, Arabic,
Spanish, and Farsi.
*## OONI Probe Desktop*
In October 2022, we released OONI Probe Desktop 3.8.1 (
https://github.com/ooni/probe-desktop/releases/tag/v3.8.1) which was
supported by OONI Probe CLI 3.16.4, and which fixed an issue affecting the
OONI Probe Signal test. Specifically, Signal changed their root CA,
resulting in our Signal test leading to false positives globally. We
documented this issue and the fix we implemented in the following ticket:
https://github.com/ooni/probe/issues/2344
*## Farsi translation of OONI Probe Mobile and Desktop User Guides*
Thanks to our community’s localization efforts, we published a Farsi
translation of:
* OONI Probe Mobile User Guide:
https://ooni.org/fa/support/ooni-probe-mobile
* OONI Probe Desktop User Guide:
https://ooni.org/fa/support/ooni-probe-desktop
*## OONI Run*
As part of our work to release an improved version of OONI Run (“OONI Run
v2”), we have written a specification describing the new functionality. In
October 2022, we updated the specification to include more details about
the descriptor format (
https://github.com/ooni/spec/pull/249#issuecomment-1286974194)
*## Expanding censorship measurement methodologies*
We started moving traffic generated by OONI Probe to a remote host at the
TCP/IP level to flexibly implement integration testing in a censored
environment. The overall idea has been documented in the following ticket:
https://github.com/ooni/probe/issues/2340
The main pull request (https://github.com/ooni/probe-cli/pull/969) includes
a fully working prototype. We also merged pull requests that re-introduce
the TProxy functionality (https://github.com/ooni/probe-cli/pull/975) and
other functionalities (https://github.com/ooni/probe-cli/pull/976)
*## OONI Explorer*
In October 2022, we continued working on OONI Explorer localization
efforts, which involved moving hardcoded text to translation files and
uploading them to Transifex in order for translators to fill in missing
strings (https://github.com/ooni/explorer/pull/805) We also fixed the text
alignment for languages that require RTL support (
https://github.com/ooni/ooni.org/issues/1264,
https://github.com/ooni/ooni.org/pull/1269)
We continued working on OONI Explorer user accounts and on the user
feedback reporting mechanism, which will enable users to login and
contribute feedback about each measurement (
https://github.com/ooni/explorer/pull/790)
We also worked on the following OONI Explorer improvements:
* Build process optimisations that result in smaller Docker image (
https://github.com/ooni/explorer/pull/807)
* Replaced Babel with SWC for faster code transpilation (
https://github.com/ooni/explorer/pull/804)
* Refactored the MAT filters to virtualize rows which improves the
performance and memory usage when we’re showing a big list of rows (
https://github.com/ooni/explorer/pull/809)
* Removed unused environment variables to reduce ambiguity (
https://github.com/ooni/explorer/pull/807)
* Refactored the OONI Explorer Search page to move the API request from
server side to client and improve rendering speed (
https://github.com/ooni/explorer/pull/799)
Moreover, we contracted a designer to help us improve the user interface of
OONI Explorer pages over the next months.
*## OONI Test Lists Editor*
Due to the API changes on the backend, code was refactored to only use one
endpoint instead of three to retrieve data. These changes contribute to
simpler code and improved performance of the Test Lists Editor:
https://github.com/ooni/test-lists-ui/pull/67/
*## OONI backend*
In October 2022, we worked on the following backend activities:
* Made backend adjustments to annotate OONI Probe Signal measurements as
“failed” as a temporary solution to an issue (
https://github.com/ooni/pipeline/pull/400) and updated the measurements in
the database;
* Looking into legacy-probe entry points and worked towards disabling the
onion service;
* Disabled the MySQL and PostgreSQL interfaces (
https://github.com/ClickHouse/ClickHouse/issues/40994)
* Test list API: Added measurement summary statistics in the test list
UI/API to enable review (https://github.com/ooni/api/pull/309) and added
statistics from measurement feedback;
* Backend changes to handle unsupported countries in test lists (
https://github.com/ooni/api/pull/310)
* Wrote an internal design document to look into moving away from
orchestrate for login/registration and documented the next steps;
* Added a probe login metric and implemented other fixes (
https://github.com/ooni/api/pull/311)
* Fixed the API aggregation time range (https://github.com/ooni/api/pull/309
).
*# Advancing OONI data analysis*
We made progress on our new data analysis tool (https://github.com/ooni/data)
through which we aim to automate the detection and characterization of more
forms of internet censorship. Specifically, we added support for generating
blocking events based on OONI Probe Signal measurements (
https://github.com/ooni/data/pull/10) we introduced a shared data
directory for storing assets needed for performing analysis tasks (eg.
GeoIP databases and blockpage fingerprints) (
https://github.com/ooni/data/pull/8) and we improved testing and the CLI (
https://github.com/ooni/data/pull/9)
*## Published new OONI Code of Conduct*
In October 2022, we published a new version of the OONI Code of Conduct
(CoC): https://ooni.org/get-involved/code-of-conduct/
To help ensure a safe and inclusive environment for the global OONI
community, we created a new Incident Response Committee with the goal of
improving OONI’s Code of Conduct (CoC) and addressing any CoC violations
when they occur. In collaboration with the Committee and based on community
feedback collected from several meetings, we re-wrote and published a new
version of OONI’s Code of Conduct (CoC).
*## Published a survey to collect community feedback on OONI’s strategic
priorities*
Leading up to OONI’s 10th Anniversary (5th December 2022), we wanted to
collect community feedback to help inform the development of our strategic
priorities for the future.
As the OONI community has always been at the heart of our work, we consider
it essential that community needs continue to inform OONI’s future goals
and priorities. We therefore circulated a survey to collect community
feedback that can help shape OONI’s strategic priorities for the future.
On 20th October 2022, we published a blog post which links to the survey:
https://ooni.org/post/2022-survey-ooni-strategic-priorities/
We thank all community members for their valuable feedback!
*## Test list updates*
In October 2022, we discussed with Netalitica how we can improve our
policies for updating test lists, with the goal of improving the quality of
measurements and simplifying the workflow of researchers. Specifically, we
discussed cases that involve updating URLs to HTTPS, how to handle parked
domains or domains that otherwise no longer exist, and about involving
researchers in the process of determining which URLs to configure for
automated prioritized testing.
We reviewed and merged multiple pull requests that involve updates to the
Thai test list, contributed by our partner, Thai Netizen Network (
https://ooni.org/partners/thai-netizen-network/) These pull requests are
the following: https://github.com/citizenlab/test-lists/pull/1152,
https://github.com/citizenlab/test-lists/pull/1169,
https://github.com/citizenlab/test-lists/pull/1168,
https://github.com/citizenlab/test-lists/pull/1167,
https://github.com/citizenlab/test-lists/pull/1166,
https://github.com/citizenlab/test-lists/pull/1165.
We also reviewed the pull requests contributed (via our Test Lists Editor)
by community members for the test lists of France (
https://github.com/citizenlab/test-lists/pull/1125) the Global test list (
https://github.com/citizenlab/test-lists/pull/1162,
https://github.com/citizenlab/test-lists/pull/1163) and several other test
lists (https://github.com/citizenlab/test-lists/pull/1159)
*## OONI citations### Freedom on the Net Report 2022*
OONI data (and OONI research reports) are cited in the following Freedom on
the Net 2022 (
https://freedomhouse.org/report/freedom-net/2022/countering-authoritarian-o…)
country reports:
* Iran: https://freedomhouse.org/country/iran/freedom-net/2022
* Myanmar: https://freedomhouse.org/country/myanmar/freedom-net/2022
* Malaysia: https://freedomhouse.org/country/malaysia/freedom-net/2022
* Cuba: https://freedomhouse.org/country/cuba/freedom-net/2022
* Azerbaijan: https://freedomhouse.org/country/azerbaijan/freedom-net/2022
* Colombia: https://freedomhouse.org/country/colombia/freedom-net/2022
* Ethiopia: https://freedomhouse.org/country/ethiopia/freedom-net/2022
* Italy: https://freedomhouse.org/country/italy/freedom-net/2022
* Jordan: https://freedomhouse.org/country/jordan/freedom-net/2022
* Kenya: https://freedomhouse.org/country/kenya/freedom-net/2022
* Saudi Arabia:
https://freedomhouse.org/country/saudi-arabia/freedom-net/2022
* Tunisia: https://freedomhouse.org/country/tunisia/freedom-net/2022
* UAE:
https://freedomhouse.org/country/united-arab-emirates/freedom-net/2022
* Uzbekistan: https://freedomhouse.org/country/uzbekistan/freedom-net/2022
* Zambia: https://freedomhouse.org/country/zambia/freedom-net/2022
*### Ranking Digital Rights Research Lab*
Ranking Digital Rights (RDR) launched a new Research Lab which discusses
how researchers can make use of OONI tools and data (among other
resources).
The RDR Research Lab is available here:
https://rankingdigitalrights.org/research-lab/preparing-your-research/
*### Team CommUNITY’s publication*
Team CommUNITY published a blog post (“Internet Freedom Tools, Resources &
Actions to Support Iran’s Feminist Uprising”) which cites our research
report on new blocks that emerged in Iran amid protests (
https://ooni.org/post/2022-iran-blocks-social-media-mahsa-amini-protests/)
Team CommUNITY’s blog post is available here:
https://www.digitalrights.community/blog/tools-resources-actions-to-support…
*## Community activities### OTF Shutdown Getdown Event*
Between 12th-14th October 2022, OONI’s Maria and Arturo traveled to Oxford
to attend the OTF Shutdown Getdown event. This 3-day event brought internet
measurement projects together to share skills and knowledge around
measuring internet censorship and shutdowns.
As part of our participation at this event, we:
* Presented OONI as part of a lightning talk;
* Facilitated a workshop to collect community feedback on OONI’s
Measurement Aggregation Toolkit (MAT);
* Facilitated a workshop to brainstorm on methods for measuring throttling;
* Co-facilitated a workshop with M-Lab which aimed to introduce
participants to the various measurement datasets available for
investigating different aspects of internet shutdowns.
*### OONI Community Meeting*
On 25th October 2022, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/) during which we discussed the
following topics:
1) Feature request: Adding support to the MAT for filtering measurements
based on multiple domains
2) Upcoming 10th OONI Anniversary: Community survey and feedback for
related activities
3) Test lists updates and re-thinking the test lists
*### RIPE85*
On 27th October 2022, OONI’s Arturo presented our research on measuring the
blocking of encrypted DNS services (and how this experiment is now part of
OONI Probe) at RIPE85: https://ripe85.ripe.net/
*## Userbase*
In October 2022, 66,272,107 OONI Probe measurements were collected from
2,797 AS networks in 169 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
[View Less]
1
0