There is no Anti-Censorship meeting today, most people is having time off. The
next meeting will be August 4.
Have a nice time off :)
--
meskio | https://meskio.net/
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
My contact info: https://meskio.net/crypto.txt
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Nos vamos a Croatan.
Hello! Here's our monthly meeting minutes. Our meeting schedule is
rather affected by those pesky northern-hemisphere-summer-holidays, we
should go back to your regular programming in September.
# Roll call: who's there and emergencies
anarcat, gaba, kez, lavamind, no emergencies.
# Dashboard review
We didn't have time to do a full quarterly review (of Q2), and people
are heading out to vacations anyways so there isn't much we can do
about late things. But we reviewed the dashboards to make sure nothing
drops to the floor with the vacations. We started with the per user
dashboards:
* https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&…
* https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&…
* https://gitlab.torproject.org/groups/tpo/-/boards?scope=all&utf8=%E2%9C%93&…
... as we usually do during our weekly checkins ("what are you working
on this week, do you need help"). Then moved on to the more general
dashboards:
* https://gitlab.torproject.org/tpo/tpa/team/-/boards/117
* https://gitlab.torproject.org/groups/tpo/web/-/boards
* https://gitlab.torproject.org/groups/tpo/tpa/-/boards
Normally we should be making new OKRs for Q3/Q4 at this time, but it
doesn't look like we have the cycles to build that system right now,
and it doesn't look like anyone else is doing so either in other
teams. We are aware of the problem and will work on figuring out how
to do roadmapping later.
Anarcat nevertheless did a quick review of the roadmap and found that
the bullseye upgrade might be a priority. He opened [issue
tpo/tpa/team#40837][] to make sure the 13 machines remaining to
upgrade are properly covered by Debian LTS while we finish the
upgrades.
[issue tpo/tpa/team#40837]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/40837
The other big pending change is the email services improvements, but
that has been deferred to TPA-RFC-31, the outsourcing of email
services, which is still being drafted.
# TPA-RFC-33: monitoring requirements adoption
Anarcat had already read aloud the requirements in the last meeting,
so he spared us from this exercise. Instead we reviewed the [changes
proposed by lavamind][] which mostly seem good. Kez still has to look
at the proposal, and their input would be crucial as someone less
familiar with our legacy stuff: new pair of eyes will be useful!
[changes proposed by lavamind]: https://gitlab.torproject.org/tpo/tpa/wiki-replica/-/merge_requests/34
Otherwise the requirements seem to be mostly agreed on, and anarcat
will move ahead with a proposal for the monitoring system that will
try to address those.
# Vacations and next meeting
As anarcat and lavamind both have vacations during the month of
August, there's no overlap when we can do a 3-way meeting, apart at
the very end of the month, a week before what will be the september
meeting. So we cancel the meeting for august, next meeting is in
september.
Regarding holidays, it should be noted that only one person of the
team is out at a time, unless someone is out sick. And that can
happen, but we can usually withstand a temporary staff outage. So
we'll have two people around all August, just at reduced capacity.
For triage of the week rotation, rotation will be changed to keep
anarcat on rotation an extra week this week, so that things even out
during the vacations (two weeks each):
* week 31 (this week): anarcat
* week 32 (next week): kez, anarcat on vacation
* week 33: lavamind, anarcat on vacation
* week 34: anarcat, lavamind on vacation
* week 35: kez, lavamind on vacation
* week 36 (september): lavamind, everyone back
# Metrics of the month
* hosts in Puppet: 96, LDAP: 96, Prometheus exporters: 164
* number of Apache servers monitored: 30, hits per second: 298
* number of self-hosted nameservers: 6, mail servers: 9
* pending upgrades: 0, reboots: 0
* average load: 2.16, memory available: 4.72 TiB/5.86 TiB, running processes: 883
* disk free/total: 29.47 TiB/91.36 TiB
* bytes sent: 420.66 MB/s, received: 298.98 MB/s
* planned bullseye upgrades completion date: 2022-09-27
* [GitLab tickets][]: 184 tickets including...
* open: 0
* icebox: 151
* backlog: 20
* next: 9
* doing: 2
* needs review: 1
* needs information: 1
* (closed: 2807)
[Gitlab tickets]: https://gitlab.torproject.org/tpo/tpa/team/-/boards
Upgrade prediction graph lives at https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/upgrades/bullseye/
# Date of the month
September 27! We moved back the estimated Debian bullseye completion
date by almost three weeks, from 2022-10-14, to 2022-09-27. This is
bound to slow down however, with the vacations coming up, and all the
remaining server needing an upgrade being the "hard" ones. Still, we
can dream, can we?
--
Antoine Beaupré
torproject.org system administration
Hello Everyone,
We're going to be cancel today's Tor Browser Meeting. The next one will be August 1, 2022
(2022-08-01) at 1100 UTC. Future meetings will most likely be at a later time to be decided.
best,
-Richard
Over the last few months we have being working on reimplementing gettor as part
of rdsys[0]. As a last step before moving it into production we are testing that
everything works fine.
You can help us testing it and telling us if everything works as expected or
something could be improved by sending emails to it at:
gettor-test(a)riseup.net
The links provided should work, but they are not located in the official gettor
places.
You can find the documentation on how to use gettor at:
* https://gettor.torproject.org or
* https://support.torproject.org/censorship/gettor-2/
* or sending an email with the word help to gettor-test(a)riseup.net
Thank you.
[0] https://gitlab.torproject.org/tpo/anti-censorship/rdsys/
--
meskio | https://meskio.net/
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
My contact info: https://meskio.net/crypto.txt
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Nos vamos a Croatan.
Hello,
OONI's June 2022 status report is shared below.
*# OONI Monthly Report: June 2022*
Throughout June 2022, the OONI team worked on the following sprints:
* Sprint 66 (1st - 5th June 2022)
* Sprint 67 (6th -19th June 2022)
* Sprint 68 (20th - 30th June 2022)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## OONI’s submission for the OHCHR report on internet shutdowns and human
rights*
In response to the UN High Commissioner for Human Rights’s call for
submissions in support of the OHCHR report on internet shutdowns and human
rights to the 50th session of the Human Rights Council in June 2022, the
OONI team provided a submission with relevant information on the occurrence
of mandated disruptions of access to social media and messaging platforms
over the past 5 years based on empirical OONI network measurement data.
Read our submission here:
https://ooni.org/documents/2022-ooni-submission-ohchr-report-internet-shutd…
We share OONI data on social media blocks during elections and protests
over the last 5 years.
*## OONI Probe Mobile*
We made a private beta release of OONI Probe Android 3.7.0, which we tested
extensively and shared with community members for testing.
Notably, the private beta of OONI Probe Android 3.7.0 includes:
* Support for running the Experimental group card automatically
* New Vanilla Tor test
* In-app support for changing languages
* Support for sharing a link to the OONI Probe app
* Measurement engine synced with OONI Probe CLI 3.15.2
Based on testing and community feedback, we worked on various bug fixes and
improvements to the OONI Probe mobile app in preparation for the public
release (3.7.0).
Specifically, we worked on:
* Fixing a bug to ensure that the correct ASN is displayed (when a failed
test is re-run) when a different network is selected (
https://github.com/ooni/probe/issues/851);
* Removing deprecated methods from Android (
https://github.com/ooni/probe/issues/1708);
* Fixing a bug affecting the view of data for uploaded measurements (
https://github.com/ooni/probe/issues/2140);
* Fixing an Android UX bug affecting the screen lock and back button
behavior (https://github.com/ooni/probe/issues/2156);
* Investigating a bug that makes the Android app slow when loading and
showing measurements (https://github.com/ooni/probe/issues/2150);
* Fixing a bug that caused the Android app to crash after a clean install
with default settings (https://github.com/ooni/probe/issues/1911);
* Ensuring that automated runs on OONI Probe Mobile are appended to the
software name (https://github.com/ooni/probe/issues/2167).
*## OONI Run*
We started drafting a plan for incrementally releasing a new version of
OONI Run (https://run.ooni.io/). During the final week of the month, we
discussed 3 different design documents for supporting a version of OONI Run
that doesn’t require a backend. In parallel, we started drafting a
specification for the new version of OONI Run (
https://github.com/ooni/spec/pull/249).
*## OONI Probe Web*
The OONI Probe Web app (https://ooni.github.io/probe-web/) includes a
website measurement experiment called “browser_web”. In June 2022, we wrote
a test specification for this experiment (
https://github.com/ooni/spec/pull/244/files). As part of this, we
identified some of the limitations to this testing methodology and we
documented some of the precautions that should be taken when analyzing the
data.
We also added support for:
* Detecting an incompatible browser in the web application (
https://github.com/ooni/probe-web/commit/f6a75f9940fe3e5961dabd0d21cc8596af…
);
* Skipping the testing of a URL on HTTP (
https://github.com/ooni/probe-web/commit/86367a614224cff53713262d366ceca614…)
because HTTP targets cannot be measured when the probe web application is
deployed on a HTTPS site due to the requests being “mixed content” (see:
https://github.com/ooni/spec/blob/master/nettests/ts-036-browser_web.md#cav…
).
*## Expanding censorship measurement methodologies*
While reflecting on the difficulties of merging Websteps into OONI Probe
and, more generally, on the toil of maintaining OONI Probe, we came up with
a better plan for organizing new experiments that should significantly
reduce the maintenance overhead.
This plan, called step-by-step, has been discussed internally, as well as
with our ICFP fellows (Ain Ghazal and Gurshabad Grover) and GSoC student
(Mehul Gulati). After several rounds of review, we published a design
document (https://github.com/ooni/probe-cli/pull/814), which was
accompanied by many proof of concepts.
*## Measuring DoT/DoH Blocking Using OONI Probe: A Preliminary Study*
Last year, we wrote a research paper ("Measuring DoT/DoH Blocking Using
OONI Probe: A Preliminary Study") which was presented at the Network and
Distributed System Security Symposium (NDSS'21) as part of the DNS Privacy
Workshop. You can read this paper here:
https://www.ndss-symposium.org/wp-content/uploads/dnspriv21-02-paper.pdf
This study involves an investigation of encrypted DNS (DoT & DoH) blocking
in Kazakhstan, Iran, and China with a new OONI Probe experiment (DNSCheck).
In June 2022, we published a blog post with a summary of findings (and
video presentation): https://ooni.org/post/2022-doh-dot-paper-dnsprivacy21/
We also shared a summary of findings through a Twitter thread:
https://twitter.com/OpenObservatory/status/1537764247442268161
*## A Quick Look at QUIC Censorship*
In previous months, OONI had the great pleasure to serve as the host
organization for Kathrin Elmenhorst's Open Technology Fund (OTF)
Information Controls Fellowship. As part of her fellowship with us, Kathrin
investigated QUIC blocking & QUIC's potential for censorship resistance.
Her findings reveal QUIC censorship in China, India, Russia and Uganda, and
she shares approaches for QUIC censorship evasion.
To help share her findings with the community, we cross-posted her blog
post (originally published on the OTF website):
https://ooni.org/post/2022-quick-look-quic-censorship/
*## OONI Explorer*
We implemented new network-centric views inside of OONI Explorer (
https://github.com/ooni/explorer/pull/750). These views enable users to
view OONI measurements from a network perspective.
We also made some small improvements, such as adding the input field on the
OONI Explorer Search tool and Measurement Aggregation Toolkit (MAT) pages (
https://github.com/ooni/explorer/pull/759), as well as other bug fixing (
https://github.com/ooni/explorer/pull/761).
*## Building a web platform for updating test lists*
We continued to polish our new web platform for updating test lists (
https://test-lists.ooni.org/) in preparation for a public launch.
Specifically, we removed the nickname field from the login page (
https://github.com/ooni/test-lists-ui/pull/41) and we made several other
bug fixes (https://github.com/ooni/test-lists-ui/pull/42,
https://github.com/ooni/test-lists-ui/pull/43).
*## OONI backend### Migration to the Clickhouse database*
We continued to work on the ongoing migration to the new OLAP database. As
part of this work, we migrated the test helper rotation script to the new
backend host.
We also worked on cleaning up (i.e. removing the legacy code), adding
changes (to support the new database), and moving our test list updater to
the new database/backend host on Clickhouse (
https://github.com/ooni/pipeline/pull/387). We made a configuration change
on the OONI API to migrate traffic to the new test helper (
https://github.com/ooni/api/pull/294).
*### URL prioritization*
In response to community feedback (
https://github.com/ooni/backend/issues/582), we adjusted the URL
prioritization system to prioritize the testing of URLs on an ASN-level (as
opposed to a countrywide-level). We tested the new ASN-based URL
prioritization system successfully (https://github.com/ooni/api/pull/293).
We also added support for filtering the URL prioritization query by week (
https://github.com/ooni/api/pull/296).
*## Test list updates*
To improve the quality of global website testing, we worked on cleaning the
Global test list (https://github.com/citizenlab/test-lists/pull/1006). This
involved reviewing URLs and associated category codes, updating URLs,
removing non-operational/irrelevant URLs, and updating category codes.
We also contributed the following updates to the Citizen Lab test lists:
* Updated the test lists for Azerbaijan and Russia (
https://github.com/citizenlab/test-lists/pull/1000,
https://github.com/citizenlab/test-lists/pull/1007);
* Added social media sites to the Global test list (
https://github.com/citizenlab/test-lists/pull/1008);
* Other updates (https://github.com/citizenlab/test-lists/pull/1004,
https://github.com/citizenlab/test-lists/pull/1005).
We also reviewed and merged many test list pull requests opened by
community members (
https://github.com/citizenlab/test-lists/pulls?q=is%3Apr+is%3Aclosed).
*## Google Summer of Code (GSoC) 2022*
During the summer of 2022, we have the opportunity to host 2 Google Summer
of Code (GSoC) students: Germa Vinsmoke and Mehul Gulati. They started
working with us in June 2022.
In particular, our GSoC 2022 students will work on the following projects:
* OONI Probe Network Experiments:
https://summerofcode.withgoogle.com/programs/2022/projects/GAEEQk5w
* OONI Explorer & Design System Improvements:
https://summerofcode.withgoogle.com/programs/2022/projects/EMOQ4Zai
*## Interviewed Community Coordinator candidates*
Following the publication of a job opening for an OONI Community
Coordinator (
https://ooni.org/post/2022-job-opening-ooni-community-coordinator/), we
reviewed all applications carefully, created a shortlist of candidates for
the first round of interviews, prepared interview questions, and followed
up with all applicants.
In June 2022, we completed the first round of interviews for the OONI
Community Coordinator position. Based on these interviews, we created a
shortlist for the second (and final) round of interviews (for early July
2022) and coordinated with candidates.
*## Improved OONI’s Code of Conduct*
Based on some initial community feedback, we edited OONI’s Code of Conduct
to improve upon it (https://github.com/ooni/ooni.org/pull/1194). We further
discussed and collected community feedback for the improvement of OONI’s
Code of Conduct during our June 2022 OONI Community Meeting (
https://pad.riseup.net/p/improving-ooni-coc-keep).
*## Created OONI video recording for the Internet Without Borders event in
Yerevan*
Between May 2022 to June 2022, eQualitie, Roskomsvoboda, and other Russian
civil society organizations organized and hosted a series of conferences
(called “Internet without Borders”) for the Russian diaspora in 5 cities
(Tbilisi, Yerevan, Vilnius, Berlin, Warsaw), with the goal of fighting
internet censorship. Information about these conferences is available here:
https://internetborders.net/
We created a 15-minute video recording, which included a brief presentation
of OONI’s tools, methods, and dataset. This video recording was shown to
participants at the Internet Without Borders event in Yerevan, Armenia, on
3rd June 2022 (https://internetborders.net/project/yerevan/).
*## Notable community use of OONI Probe and OONI data### Access Now paper
on taxonomy of internet shutdowns*
Access Now published a paper providing a taxonomy of internet shutdowns
(citing OONI data):
https://www.accessnow.org/cms/assets/uploads/2022/06/A-taxonomy-of-internet…
*## Community activities*
*### OONI training session for civil society groups from Zimbabwe, Zambia,
and South Africa*
On 1st June 2022, OONI’s Maria facilitated an OONI training session for
civil society groups from Zimbabwe, Zambia, and South Africa. This was
organized by our partner, Digital Society of Africa (
https://ooni.org/partners/digital-society-of-africa/).
*### RightsCon 2022*
As part of our participation at RightsCon 2022 (https://www.rightscon.org/),
we co-facilitated/spoke at the following sessions:
* Workshop on "Building capacity to document and circumvent internet
shutdowns" co-hosted with SFLC.in and Top10VPN (6th June 2022)
* Community lab on "Monitoring internet censorship in South and Southeast
Asia" co-hosted with Sinar Project (7th June 2022)
* Panel on "Internet splintering due to server-side blocking and embargo
sanction", hosted by Censored Planet (8th June 2022)
* Social hour on "A light in the dark: learning about internet shutdowns
and the #KeepItOn campaign" co-hosted with organizations from the #KeepItOn
campaign (9th June 2022)
We also participated in two Private Meetings (where we presented OONI)
hosted at RightsCon by community organizations.
*### OONI presentation and hackathon at Internet without Borders event in
Vilnius*
We traveled to Vilnius, Lithuania, where we attended the Internet Without
Borders conference (for the Russian diaspora) in person (
https://internetborders.net/project/vilnius/).
As part of our participation, we:
* Presented OONI (10th June 2022);
* Facilitated a workshop on using OONI’s Measurement Aggregation Toolkit
(MAT) for investigating internet censorship in Russia (10th June 2022);
* Facilitated a day-long OONI hackathon (11th June 2022).
As part of the OONI hackathon, participants primarily worked on analyzing
OONI data from Russia and investigated the correlation between the spike of
failed and anomalous measurements.
*### OONI presentation and hackathon at Internet without Borders event in
Berlin*
We traveled to Berlin, Germany, where we attended the Internet Without
Borders conference (for the Russian diaspora) in person (
https://internetborders.net/project/berlin/).
As part of our participation, we:
* Presented OONI (17th June 2022);
* Facilitated a day-long OONI hackathon (18th June 2022).
*### DW Global Media Forum 2022*
OONI’s Arturo traveled to Bonn, Germany, to attend Deutsche Welle’s (DW)
Global Media Forum 2022 (https://gmf-event.com/) in person, where he
participated as a speaker on a panel discussion on documenting internet
shutdowns.
*### OONI Community Meeting*
On 28th June 2022, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:
1) Request for community feedback on improving OONI's Code of Conduct (
https://github.com/ooni/ooni.org/blob/master/CODE_OF_CONDUCT.md#ooni-code-o…
,
https://github.com/ooni/gatherings/blob/master/partner-gatherings/2017-07-t…
)
2) Enabling researchers to dynamically adjust custom test lists used by
community networks of volunteers (needed in countries like Iran and
Venezuela)
*## Userbase*
In June 2022, 38,147,681 OONI Probe measurements were collected from 2,833
AS networks in 170 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hello,
OONI's May 2022 status report is shared below.
*# OONI Monthly Report: May 2022*
Throughout May 2022, the OONI team worked on the following sprints:
* Sprint 64 (1st - 8th May 2022)
* Sprint 65 (9th - 22nd May 2022)
* Sprint 66 (23rd - 31st May 2022)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Job opening for OONI Community Coordinator*
In May 2022, we published a job opening for hiring a Community Coordinator
for OONI: https://ooni.org/post/2022-job-opening-ooni-community-coordinator/
In an effort to help ensure that our job opening reached interested
candidates around the world, we worked on quite extensive outreach efforts.
In particular, we posted the job opening on:
* All OONI social media channels (Twitter, Mastodon, Facebook, Instagram,
Linkedin);
* All OONI mailing lists;
* Community mailing lists (such as those of the #KeepItOn campaign, Global
Voices Advocacy, and NetRights);
* OONI Slack channel (and other relevant Slack channels);
* Digital Rights Job Board (https://www.digitalrights.community/job-board);
* Hasgeek Job Board (https://hasjob.co/openobservatory.org/12jm2);
* FOSS Job Board (
https://www.fossjobs.net/job/10936/ooni-community-coordinator-at-open-obser…
).
We also reached out to our partners and many other community members,
requesting that they help share this opportunity with their networks.
Following the publication of this job opening, we worked on tracking and
reviewing incoming applications, and following-up with applicants.
*## OONI Probe Mobile*
Throughout May 2022, we worked on various improvements to the OONI Probe
mobile app in preparation for the next release.
Specifically, we worked on:
* Including the Vanilla Tor experiment as part of the Experimental card of
OONI Probe (https://github.com/ooni/probe/issues/2103);
* Ensuring that the Vanilla Tor and Tor Snowflake experiments only run as
part of automated testing (https://github.com/ooni/probe/issues/2102);
* Providing a working sample for Flutter on OONI Probe apps with probe-cli
integration (https://github.com/aanorbel/probe-shared);
* Enabling users to see all logs (https://github.com/ooni/probe/issues/1777,
https://github.com/ooni/probe-android/pull/516);
* Fixing an iOS bug affecting the icons displayed when all tests are run (
https://github.com/ooni/probe/issues/2055).
*## OONI Run*
We implemented a fix related to the copy-pasting of URLs into the OONI Run
web interface (https://github.com/ooni/run/pull/105). Based on community
feedback, we explored how we can add support to OONI Run for saving custom
test lists (https://github.com/ooni/probe/issues/867), and we had team
meetings to discuss possible implementation approaches. We also wrote an
internal design document on implementation approaches for addressing the
most pressing OONI Run community needs (as identified through a previous
OONI Run usability study and ongoing community interactions).
*## OONI Probe Desktop*
We made some improvement to the description of the Experimental card in
OONI Probe Desktop (https://github.com/ooni/probe-desktop/pull/294) and we
worked towards ensuring that the autorun directory is deleted when a hard
reset is performed (https://github.com/ooni/probe-desktop/pull/295).
*## OONI Probe Web*
We added support to OONI Probe Web for using the check-in API to retrieve
information about the probe IP address country code and ASN (
https://github.com/ooni/probe-web/commit/bc680323df0c7404b14651acea11a281c7…).
We also worked on improving the UI of OONI Probe Web.
We circulated an internal testing beta of OONI Probe Web (
https://ooni.github.io/probe-web/) to collect community feedback that can
help inform further development and improvements.
*## Expanding censorship measurement methodologies### Website measurement*
After some initial experimentation in April 2022, we sketched out a plan
for integrating some methodologies from the “Understanding the Practices of
Global Censorship through
Accurate, End-to-End Measurements” (
https://shhaos.github.io/papers/sigmetrics22.pdf) study into OONI Probe (
https://github.com/ooni/probe/issues/2086).
We started implementing the plan for merging the Websteps prototype (
https://github.com/bassosimone/websteps-illustrated/) into OONI Probe. To
this end, we merged several improvements for netxlite, the ooniprobe
package that provides basic networking functionality (
https://github.com/ooni/probe/issues/2096).
We also drafted and implemented a plan for observing duplicate DNS replies (
https://github.com/ooni/probe/issues/2099). This plan differs from the
Websteps implementation because community feedback (in particular from
Jigsaw’s Vinicius Fortuna and Sarah Laplante) convinced us that the
implementation of this functionality that we had sketched out in the
Websteps prototype could be further simplified and streamlined.
*### New experiments*
We implemented several new exploratory experiments:
* dnsping:
https://github.com/ooni/spec/blob/master/nettests/ts-035-dnsping.md
* simplequicping:
https://github.com/ooni/spec/blob/master/nettests/ts-034-simplequicping.md
* tcpping:
https://github.com/ooni/spec/blob/master/nettests/ts-032-tcpping.md
* tlsping:
https://github.com/ooni/spec/blob/master/nettests/ts-033-tlsping.md
These are “exploratory” in the sense that we expect users to manually run
them from the command line when they are actively researching censorship.
We introduced these four new experiments based on community feedback and
requests.
*### Vanilla Tor experiment*
We implemented the Vanilla Tor experiment (
https://github.com/ooni/probe-cli/pull/718), which used to be part of the
legacy version of OONI Probe. We reintroduced this experiment following
discussions with Tor developers, who mentioned that it would be quite
useful to them.
*### Data quality improvements*
We investigated and fixed a rather surprising and complex data quality
issue. It turns out that the Android implementation of `getaddrinfo`
behaves differently from any other existing `getaddrinfo` implementation
and only returns one of two return codes: 0 in case of success, and
`EAI_NODATA` in case any other possible error occurs, including cases where
a domain name does not exists, cases where the server is temporarily
unavailable, cases where the server is misconfigured, and cases of
censorship.
In practical terms, this means that the result of `getaddrinfo` on Android
is unreliable and we need to improve our heuristics on the backend and our
algorithms on the probe to work around this. The pull request where we
fixed this bug refers to our investigation, as well as to our follow up
plans to further improve DNS measurements on Android (
https://github.com/ooni/probe-cli/pull/764). The reference issue describing
this problem (https://github.com/ooni/probe/issues/2029) is still open
because we have not included the fix into the 3.15.x release cycle (which
was being finalized while we were still investigating the root cause of
this surprising bug).
*## OONI Explorer*
We fixed a couple of OONI Explorer bugs. Specifically, we added support for
searching URLs with a longer TLD (https://github.com/ooni/explorer/pull/757)
and we updated the copyright year in the footer (
https://github.com/ooni/explorer/pull/756).
*## Building a web platform for updating test lists*
Based on community feedback on the private beta version of our new test
list web platform (https://test-lists.ooni.org/), we added backend support (
https://github.com/ooni/api/pull/286) to enable users to make changes to
open pull requests (once a pull request has been opened through a test list
web platform submission).
In anticipation of a potentially increased volume of test list pull
requests following the upcoming public launch of the test list web
platform, we (along with the Citizen Lab) invited Censored Planet and
Netalitica team members to get added to the Citizen Lab test list
repository as reviewers of pull requests.
*## OONI backend*
*### Migration to the Clickhouse database*
We continued to work on the ongoing migration to the new OLAP database. In
May 2022, we migrated 12 API entry points to the new backend host.
*### Reprocessing legacy measurements*
We continued to work on the reprocessing of measurements collected before
2021 to improve data quality and ensure all measurements are reachable from
OONI Explorer.
*### OONI API*
We reviewed and deployed a change to support the detection of the ASN and
country code of probes based on their IP address on the API side. This will
allow the removal of bulky GeoIP database files from the probes. We also
improved the account_metadata API call to display login status on web UIs (
https://github.com/ooni/api/pull/292).
*### Other operations *
We added blackbox exporter-based testing and support for the alarming of
various API entry points. We finished moving the test helper rotation
script to the new backend host (https://github.com/ooni/pipeline/pull/386).
*## Test list updates*
In May 2022, we contributed a few updates to the Citizen Lab test lists. In
particular, we updated the US test list (
https://github.com/citizenlab/test-lists/pull/984) and we removed dead URLs
from the Global test list (https://github.com/citizenlab/test-lists/pull/995).
*## Notable community use of OONI Probe and OONI data### Sinar Project
report*
Malaysia’s Sinar Project published a report on the blocking of a NFT
fundraising campaign website (of graphics designer and political activist,
Fahmi Reza):
https://sinarproject.org/digital-rights/measuring-and-detecting-network-int…
Their report makes use of OONI Probe and OONI data.
*## Community activities### OONI workshop for civil society groups in Latin
America*
On 19th May 2022, OONI’s Maria facilitated a 2-hour OONI workshop for civil
society groups in Latin America. This workshop (organized by Derechos
Digitales) was facilitated in collaboration with Venezuela Inteligente.
*### OONI presentation at Tactical Tech*
On 24th May 2022, OONI’s Maria presented OONI at Tactical Tech (
https://tacticaltech.org/) in Berlin. This presentation also involved a
live demo of using the Measurement Aggregation Toolkit (MAT) to investigate
various forms of internet censorship around the world.
*### OONI presentation for journalists*
On 25th May 2022, OONI’s Maria presented OONI’s tools, methods, research,
and dataset to a group of journalists from Russia and Belarus.
*### OONI Community Meeting*
On 31st May 2022, we hosted the monthly OONI Community Meeting on our Slack
channel (https://slack.ooni.org/), during which we discussed the following
topics:
1) Job Opening: OONI Community Coordinator (
https://ooni.org/post/2022-job-opening-ooni-community-coordinator/)
2) Detecting regional censorship, which is increasingly becoming an urgent
issue (https://github.com/ooni/probe/issues/1581#issuecomment-1125221383)
In particular, we discussed the following questions with the community:
* Should there be an option to also include the region or should we include
the region by default?
* Should we invest some legal research time to figure out if we could use
MaxMind?
* Should we try to have location permission in the app and use that? Should
we have two apps, one with location and the other without?
* Should we allow users to manually annotate/correct the geolocation
results if they are inaccurate?
3) Copyright law for enforcing internet censorship
*## Userbase*
In May 2022, 41,882,898 OONI Probe measurements were collected from 2,905
AS networks in 171 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
Hello,
These days we're primarily supported by the DRL, which is why we do
quarterly reporting. That said, we'd like to share monthly updates from the
OONI team with the community, hence our April 2022 report shared below.
I'll also follow-up to share our May and June 2022 reports.
*# OONI Monthly Report: April 2022*
Throughout April 2022, the OONI team worked on the following sprints:
* Sprint 62 (1st - 10 April 2022)
* Sprint 63 (11th - 24th April 2022)
* Sprint 64 (25th April - 30th April 2022)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Launched the OONI Measurement Aggregation Toolkit (MAT)*
We publicly launched OONI’s new Measurement Aggregation Toolkit (MAT):
https://explorer.ooni.org/chart/mat
The MAT is a tool that enables the public to create their own custom charts
based on aggregate views of real-time OONI data collected from around the
world. This tool was built for researchers, journalists, and human rights
defenders interested in examining internet censorship around the world.
When examining cases of internet censorship, it's important to look at many
measurements at once ("in aggregate") in order to identify censorship
trends and patterns. The MAT incorporates our data analysis techniques,
enabling the public to answer such questions without any data analysis
skills, and with the click of a button!
To provide further details about using the MAT and interpreting MAT charts,
we wrote and published a blog post: https://ooni.org/post/2022-ooni-mat/
We hope the internet freedom community finds the MAT useful for
investigating and responding to internet censorship events around the world.
*### Preparing for the MAT launch *
In preparation for the public launch of OONI’s Measurement Aggregation
Toolkit (MAT), we:
* Improved the interaction between the table and the chart in the
multi-axis charts (https://github.com/ooni/explorer/pull/724);
* Added support for exporting data in CSV format and added links to CSVs
under the charts (https://github.com/ooni/explorer/pull/741);
* Wrote (and published) “Help” text for the MAT (
https://github.com/ooni/explorer/issues/727);
* Worked on several other bug fixes and improvements (
https://github.com/ooni/explorer/pull/731,
https://github.com/ooni/explorer/pull/732,
https://github.com/ooni/explorer/pull/737,
https://github.com/ooni/explorer/pull/740,
https://github.com/ooni/explorer/pull/741).
*## OONI MAT shows social media blocks in Sri Lanka*
OONI’s new Measurement Aggregation Toolkit (MAT) was useful for enabling
human rights defenders to corroborate the social media blocks that were
reported in Sri Lanka between 2nd to 3rd April 2022.
The following OONI MAT charts (which we shared with the #KeepItOn
community) present aggregate views of OONI data, displaying the temporary
blocking of social media services in Sri Lanka on 2nd and 3rd April 2022:
* Twitter:
https://explorer.ooni.org/experimental/mat?probe_cc=LK&test_name=web_connec…
* Facebook:
https://explorer.ooni.org/experimental/mat?probe_cc=LK&test_name=web_connec…
* Instagram:
https://explorer.ooni.org/experimental/mat?probe_cc=LK&test_name=web_connec…
* Facebook Messenger:
https://explorer.ooni.org/experimental/mat?probe_cc=LK&test_name=facebook_m…
* Telegram:
https://explorer.ooni.org/experimental/mat?probe_cc=LK&test_name=telegram&s…
* WhatsApp:
https://explorer.ooni.org/experimental/mat?probe_cc=LK&test_name=whatsapp&s…
By clicking on each of these charts, the public can access the raw OONI
measurement data, showing details of how the blocks were implemented and
how they vary across ISPs in the country. The links to the above charts are
automatically updated with real-time OONI data.
*## Circumvention Tool Reachability Dashboard*
We made some improvements to our new Circumvention Tool Reachability
Dashboard, which presents OONI findings from the testing of censorship
circumvention tools around the world (
https://explorer.ooni.org/chart/circumvention). In particular, we added
copy that highlights a limitation to the dashboard, while also encouraging
users to refer to Tor Metrics and the Psiphon Data Engine (
https://github.com/ooni/explorer/issues/726) to gain a more comprehensive
understanding of whether circumvention tools work in a country.
*## OONI Explorer network-centric views*
In April 2022, we worked on implementing network-centric views for OONI
Explorer (https://github.com/ooni/explorer/issues/744). The implementation
(and subsequent review, feedback, and improvements) is available through
the following pull request: https://github.com/ooni/explorer/pull/750
*## OONI Probe Mobile*
We released OONI Probe iOS 3.6.0:
https://github.com/ooni/probe-ios/releases/tag/v3.6.0
This release includes the following improvements:
* New Tor Snowflake experiment
* New DNS Check experiment
* RiseupVPN test temporarily disabled
* Links to OONI blog posts and research reports
* User-initiated tests start off minimized
* Improved progress bar and test runtime counter
* Measurement engine synced with OONI Probe CLI 3.14
* Bug fixes and improvements
We followed-up with an OONI Probe iOS 3.6.1 release (
https://github.com/ooni/probe-ios/releases/tag/v3.6.1) with bug fixes and
improvements.
Notably, we set up a Huawei AppGallery account, where we released the OONI
Probe Android app (https://appgallery.huawei.com/app/C105911849).
We also worked on:
* Adding support for sharing a link to the OONI Probe app (
https://github.com/ooni/probe/issues/1997);
* Making the OONI Probe VPN check compatible with Netguard firewall (
https://github.com/ooni/probe/issues/2039);
* Writing an internal design document for improving the architecture of
OONI Probe apps for shared codebases;
* Adding support for paginating test results (
https://github.com/ooni/probe/issues/1430);
* Fixing a bug involving the re-run of failed tests (
https://github.com/ooni/probe/issues/2048);
* Fixing a bug affecting a link on the test details page (
https://github.com/ooni/probe/issues/2075);
* Other improvements (https://github.com/ooni/probe/issues/1252,
https://github.com/ooni/probe/issues/910).
*## Expanding censorship measurement methodologies*
Our goal is to ensure that our new experiments (particularly Websteps, our
new experiment for measuring website blocking) are effective in some of the
most censored environments and meet community needs. Following several
rounds of community feedback, we developed our latest prototype (“Websteps
winter edition”) in March 2022.
In April 2022, we started discussing the “Websteps winter edition” (
https://github.com/bassosimone/websteps-illustrated/) prototype with
community members, we collected their feedback, and we drafted plans for
incorporating changes developed in this prototype inside of ooniprobe.
We also read the “Understanding the Practices of Global Censorship through
Accurate, End-to-End Measurements” paper (
https://shhaos.github.io/papers/sigmetrics22.pdf) and we started
experimenting with its methodologies to understand how to integrate them
into ooniprobe (https://github.com/bassosimone/websteps-illustrated/pull/2).
*## Building a web platform for updating test lists*
We continued to make progress towards launching our new web platform for
updating test lists (https://test-lists.ooni.org/).
Specifically, we made improvements related to:
* Validating the login form (https://github.com/ooni/test-lists-ui/pull/28
);
* Setting the default source field (
https://github.com/ooni/test-lists-ui/pull/21);
* Upgrading to the latest versions of next.js (
https://github.com/ooni/test-lists-ui/pull/31).
We also discussed some approaches that would allow us to support multiple
edits after the changes have been submitted, and we documented how this can
be implemented in the backend (https://github.com/ooni/backend/issues/588).
*## OONI backend### Migration to the Clickhouse database*
We continued to work on various tasks related to the ongoing migration to
an OLAP database that delivers much better performance and enables new
functionalities like the MAT.
We migrated multiple API calls to the host running the new database and we
improved monitoring. This is a long-running process.
*### Reprocessing legacy measurements*
OONI measurements collected before 2021 have been reprocessed to improve
data quality and to ensure all measurements are now reachable from OONI
Explorer. This is also a long-running process.
*### URL prioritization*
The "reactive" URL prioritization system is designed to ensure acceptable
and fair coverage of URLs over time. But based on community feedback (
https://github.com/ooni/backend/issues/582), we realized that the existing
implementation could allow a manually-run probe to over-measure a given URL
and prevent other probes from measuring this URL at all on a national
level. We therefore investigated how we can change the implementation of
our URL prioritization algorithm to ensure that per-ASN testing coverage
does not impact countrywide testing coverage of a URL. Based on our
research, we wrote an internal design document for improving our URL
prioritization system.
*### Other operations*
We added support for STUN reachability tests in the API search function. We
improved NTP dashboards and monitoring, fixed IPv6 and ICMP filtering on
backend hosts, and we moved the test-helper rotation tool to use Debian 11
and fixed a bug. We also looked into the price/performance ratio on new
hosts.
*## OONI data analysis tool*
To enable the analysis of raw OONI measurements, we built a tool
(“oonidata”) which supports downloading raw OONI data in batch (
https://github.com/ooni/backend/issues/581). In April 2022, we published a
version of oonidata on pypi (https://pypi.org/project/oonidata/).
*## Preparing a security audit*
We aim to receive a security audit for several OONI software components:
OONI API accounts, OONI Probe Web, OONI Probe engine. To this end, we
coordinated with several security companies to discuss audit requirements
and receive quotes.
*## Filmed OONI training course for Advocacy Assembly project*
In previous months, we completed the creation of all materials for the
online OONI training course for Small Media’s Advocacy Assembly project (
https://www.advocacyassembly.org/).
In April 2022, OONI’s Maria traveled to London to record 8 direct-to-camera
videos at a studio (with Small Media staff) for the online OONI training
course.
In particular, this involved the filming of the following videos:
* Introduction to course
* What is internet censorship?
* The problem of internet censorship
* What is OONI Probe?
* Interpreting OONI data
* What is OONI Explorer?
* Looking at OONI data in aggregate
* Conclusion
Small Media aims to launch the online OONI training course over the next
months.
*## ISOC Pulse project on Internet shutdowns*
OONI is a data partner for the Internet Society (ISOC) Pulse project on
Internet shutdowns (https://pulse.internetsociety.org/partners). We aim to
contribute OONI data (along with relevant charts and information) for all
the “content blocking” events listed on the ISOC Pulse shutdowns timeline (
https://pulse.internetsociety.org/shutdowns).
We created OONI reports (including OONI data, charts, and relevant
interpretation of the findings) for the following 4 “content blocking”
entries listed in the ISOC Pulse shutdown timeline:
* April 2022 - Sri Lanka (
https://pulse.internetsociety.org/shutdowns/social-media-disruption-in-sri-…)
* February 2022 - Russia (
https://pulse.internetsociety.org/shutdowns/incremental-service-limitation-…)
* January 2022 - Burkina Faso (
https://pulse.internetsociety.org/shutdowns/second-shutdown-in-a-month-in-b…)
* August 2021 - Zambia (
https://pulse.internetsociety.org/shutdowns/social-media-limitations-in-zam…)
We shared our reports with ISOC to be listed under the “Data and Analysis”
section of relevant ISOC Pulse shutdown entries.
*## Report for Azerbaijan Internet Watch*
In support of our partner, Azerbaijan Internet Watch (
https://ooni.org/partners/azerbaijan-internet-watch/), we analyzed OONI
measurements collected from Azerbaijan between January 2022 to March 2022.
Based on our analysis, we produced charts and wrote a report interpreting
the findings, which we shared with Azerbaijan Internet Watch.
*## Test list updates*
In April 2022, we contributed a few updates to the Citizen Lab test lists (
https://github.com/citizenlab/test-lists/pull/956,
https://github.com/citizenlab/test-lists/pull/962,
https://github.com/citizenlab/test-lists/pull/957), and we reviewed and
merged several test list pull requests opened by community members (
https://github.com/citizenlab/test-lists/issues?page=3&q=is%3Aclosed).
*## Job opening for bookkeeper*
In collaboration with the Hermes Center (OONI’s current fiscal sponsor), we
published a job opening for hiring a new bookkeeper:
https://www.hermescenter.org/it/cerchiamo-un-una-consulente-contabile-e-amm…
Throughout April 2022, we worked on managing and reviewing applications,
following up with candidates, and interviewing shortlisted candidates for
the position.
*## OONI featured in Dataplane.org newsletter*
OONI was among the measurement platforms (along with Censored Planet, RIPE
Atlas, and NLNOG Ring) discussed in Dataplane.org’s April 2022 newsletter:
https://dataplane.substack.com/p/sensor-strategy-monitoring-platforms
*## Notable community use of OONI Probe and OONI data### Access Now’s 2021
Annual KeepItOn Report*
Access Now published their 2021 Annual #KeepItOn campaign Report (“The
Return Of Digital
Authoritarianism: Internet shutdowns in 2021”):
https://www.accessnow.org/cms/assets/uploads/2022/04/2021-KeepItOn-Report-1…
This report makes use of OONI data, citing OONI research reports (
https://ooni.org/reports) and OONI data from OONI Explorer (
https://explorer.ooni.org/).
*### VEsinFiltro’s 2021 Annual Report*
In April 2022, our Venezuelan partner, VEsinFiltro, published their 2021
Annual Report on digital rights and internet censorship in Venezuela:
https://vesinfiltro.com/noticias/2021_annual_report/
In the Methodology section of their report, VEsinFiltro describe that they
measured internet blocks in Venezuela through the use of OONI Probe and
OONI data.
*## Community activities### Mozilla Twitter Space on network outages*
On 5th April 2022, OONI’s Maria participated as a speaker on Mozilla’s
Twitter Space discussion on network outages:
https://twitter.com/mozilla/status/1511091238774190085
As part of her participation, Maria explained OONI’s methods and discussed
how OONI had previously analyzed Mozilla telemetry (in collaboration with
IODA) to investigate internet shutdowns (
https://ooni.org/post/2021-investigating-internet-shutdowns-mozilla-telemet…).
The recording was available here: https://twitter.com/i/spaces/1MnxnkayOXEKO
*### OONI data analysis lecture for Georgia Tech students*
On 12th April 2022, OONI’s Arturo was invited by IODA (
https://ioda.inetintel.cc.gatech.edu/) to give a lecture to Georgia Tech
computer science students on analyzing OONI data to investigate internet
censorship. In preparation for this lecture, Arturo prepared a Jupyter
notebook with exercises for the students.
*### OONI Community Meeting*
On 19th April 2022, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:
1) New launch: OONI Measurement Aggregation Toolkit (MAT)
2) New test list web platform (https://test-lists.ooni.org/): Community
feedback and next steps
3) Tweaking OONI’s URL prioritization algorithm to ensure counts per ASN (
https://github.com/ooni/backend/issues/582)
4) Inconsistent measurement counts returned by API endpoints (
https://github.com/ooni/backend/issues/583)
*## Userbase*
In April 2022, 41,200,730 OONI Probe measurements were collected from 3,372
AS networks in 162 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
TAILS REPORT FOR JUNE 2022 [1]
Here are a few highlights about what we did in May, among many other
things:
*
Our helpdesk email [2] now has auto-reply (#17935 [3]).
*
Improved the UX of _Tor Connection_: people that were "exploring" the
interface going back and forth used to find the wrong boxes to be
automatically checked #18546 [4]
*
We partecipated in Tor hackweek. It was very fun and very useful!
*
We implemented QR code scanning #18219 [5]. This seems to work, but it
still isn't ready for inclusion.
*
We added support for Snowflake #5494 [6]. Again, this is a PoC: it's
nice to see it working, but don't expect it to be released so soon.
RELEASES
Tails 5.1 was released on June 4 [7]. It fixed an important security
vulnerability [8], but it also brought many improvements, among which:
*
Improvements for users who leave east of UTC: _Tor Connection_ now
automatically fixes the computer clock if you choose to connect to Tor
automatically; also, the time displayed in the top navigation uses the
time zone selected when fixing the clock in _Tor connection_.
*
_Unsafe browser_ got a new homepage, which makes it easier to understand
how to sign in to the local network using a captive portal.
After that, Tails 5.1.1 was released on June 23 [9] as an emergency
release.
METRICS
Tails has been started more than 772283 times this month. This makes
25742 boots a day on average.
Links:
------
[1] https://tails.boum.org/news/report_2022_06/
[2] https://tails.boum.org/support/index.en.html
[3] https://gitlab.tails.boum.org/tails/tails/-/issues/17935
[4] https://gitlab.tails.boum.org/tails/tails/-/issues/18546
[5] https://gitlab.tails.boum.org/tails/tails/-/issues/18219
[6] https://gitlab.tails.boum.org/tails/tails/-/issues/5494
[7] https://tails.boum.org/news/version_5.1/index.en.html
[8] https://tails.boum.org/security/prototype_pollution/index.en.html
[9] https://tails.boum.org/news/version_5.1.1/index.en.html