- ooni-talk - lists.torproject.org

OONI Community Meeting: Tomorrow, 25th October 2022
by Elizaveta Yachmeneva 24 Oct '22

24 Oct '22

Hello, The OONI team warmly welcomes you to join us next Tuesday for our monthly community meeting. => Where? OONI Slack channel: https://slack.ooni.org/ <https://slack.ooni.org/> (bridged with IRC: ircs://irc.oftc.net:6697/#ooni) => When? Tuesday, 25th October 2022 at 14:00 UTC (for 1 hour) Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep <https://pad.riseup.net/p/ooni-community-meeting-keep> The monthly OONI community meetings aim to: * Collect community feedback on OONI tools & methodologies * Address questions in relation to the use of OONI tools and OONI data * Foster discussions on internet censorship issues * Receive updates from the community We hope you can join us! All the best, ~ OONI team

1 0

OONI Monthly Report: September 2022
by Maria Xynou 19 Oct '22

19 Oct '22

Hello, OONI's September 2022 status report is shared below. *# OONI Monthly Report: September 2022* Throughout September 2022, the OONI team worked on the following sprints: * Sprint 73 (1st-11th September 2022) * Sprint 74 (12th-25th September 2022) * Sprint 75 (26th-30th September 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## Published report on new censorship events in Iran* On 25th September 2022, we published a new research report documenting the latest censorship events in Iran amid the ongoing protests. Read the report here: https://ooni.org/post/2022-iran-blocks-social-media-mahsa-amini-protests/ In this report, we share OONI network measurement findings from Iran on the blocking of WhatsApp, Instagram, Linkedin, Skype, Google Play Store, Apple App Store, and encrypted DNS (DNS over HTTPS). We also share IODA and Cloudflare Radar data on mobile network outages. On 27th September 2022, we published an updated version of the report (“Network outages” section) including contributions from our partner, IODA: https://ooni.org/post/2022-iran-blocks-social-media-mahsa-amini-protests/#n… This report received press coverage from a data journalism piece by Tagesspiegel (which also makes use of raw OONI data): https://interaktiv.tagesspiegel.de/lab/internet-sperren-im-iran-blackouts-g… *## Published report on TikTok block in Azerbaijan and Armenia* On 16th September 2022, in collaboration with Azerbaijan Internet Watch, we published a new research report which shares OONI data on the blocking of TikTok in Armenia and Azerbaijan during the border clashes. Read our report here: https://ooni.org/post/2022-azerbaijan-and-armenia-blocks-tiktok/ Overall, OONI data shows that Azerbaijan blocked TikTok by means of TLS level interference, while Armenia implemented both DNS level blocks and TLS level interference. *## Report for Azerbaijan Internet Watch* In support of our partner, Azerbaijan Internet Watch ( https://ooni.org/partners/azerbaijan-internet-watch/) we analyzed OONI measurements collected from Azerbaijan between May 2022 to September 2022. Based on this analysis, we produced a report with the key findings, which Azerbaijan Internet Watch published on their website: https://www.az-netwatch.org/wp-content/uploads/2022/10/2022-09-OONI-Report-… *## OONI Probe Mobile* During September 2022, we made progress on implementing a functionality that enables OONI Probe Mobile users to view all logs ( https://github.com/ooni/probe-android/pull/516, https://github.com/ooni/probe-ios/pull/499) This change stems from user requests. While OONI Probe already implements some form of logging, the app is not able to collect logs generated during the engine bootstrap. Crucially, these logs are key to debug and understand cases in which OONI services are blocked. The new design, instead, allows us to observe the engine bootstrap logs. Additionally, we added support for sharing the logs with OONI developers in an easier way, using the operating systems’ share functionality. We also made progress in implementing new Experimental settings under the Test Options settings of the OONI Probe apps ( https://github.com/ooni/probe-android/pull/543, https://github.com/ooni/probe-ios/pull/501) This will enable OONI Probe users to enable or disable experimental tests ( https://github.com/ooni/probe/issues/2266) As these settings would apply to both manual and automated runs, we added relevant copy for clarification (https://github.com/ooni/probe/issues/2268) This work addresses community feedback that the OONI Probe Experimental card includes long-running tests which users may want to have disabled as part of automated testing. We are currently performing QA of these new mobile functionalities and we are confident we will be able to include them in the next OONI Probe mobile releases. As part of our work on using Flutter for a shared codebase between the OONI Probe mobile and desktop apps, we wrote an internal guide to document the migration steps for all OONI Probe apps from Native Platforms to Flutter Application using OONI Test Descriptors. Through this document, we aim to capture all components of the system that need to be changed in the process in order to attain complete migration of Android, iOS and Desktop applications to the respective Flutter versions. *## OONI Run* We added support to miniooni for running an OONI Run v2 descriptor saved on the local disk, as opposed to fetching it from a remote URL: https://github.com/ooni/probe-cli/pull/966 *## OONI Probe Desktop* We released OONI Probe Desktop 3.8.0: https://github.com/ooni/probe-desktop/releases/tag/v3.8.0 This release uses the latest version of OONI Probe CLI and improvements to the Experimental card. *## OONI Probe localization* Thanks to the Localization Lab community, new OONI Probe strings were translated into the following languages: Farsi, Russian, Chinese, Turkish, Spanish, French, and Greek (https://explore.transifex.com/otf/ooniprobe/) *## Updated localization guidelines* We worked towards updating the OONI Probe localization guidelines, and writing new localization guidelines for OONI Explorer. This work is documented through the following pull request: https://github.com/ooni/translations/pull/26 The new localization guidelines are available through the following links: * OONI Probe localization guidelines: https://github.com/ooni/translations/blob/master/Guidelines%20for%20OONI%20… * OONI Explorer localization guidelines: https://github.com/ooni/translations/blob/master/Guidelines%20for%20OONI%20… *## Published updated versions of the OONI Probe user guides* In September 2022, we published updated versions (with extensive edits, and many new screenshots and steps) of the OONI Probe user guides: * User Guide for OONI Probe Mobile: https://ooni.org/support/ooni-probe-mobile * User Guide for OONI Probe Desktop: https://ooni.org/support/ooni-probe-desktop The specific updates can be viewed through this pull request: https://github.com/ooni/ooni.org/pull/1202 Now both the OONI Probe Mobile and Desktop user guides cover all the latest features up until version 3.7.0. We subsequently further edited the OONI Probe Mobile ( https://github.com/ooni/ooni.org/pull/1239) and Desktop ( https://github.com/ooni/ooni.org/pull/1240) user guides to clarify settings related to automated testing. *## Localizing the OONI Probe user guides* To enable community members around the world to use OONI Probe, we coordinated with the Localization Lab (https://www.localizationlab.org/) on creating new Transifex projects for the translation of the OONI Probe user guides. Community members can now localize the OONI Probe user guides via the following Transifex projects: * OONI Probe Mobile User Guide: https://www.transifex.com/otf/ooni-probe-mobile-user-guide/ * OONI Probe Desktop User Guide: https://www.transifex.com/otf/ooni-probe-desktop-user-guide/ *## Updated the OONI Data Policy* We made a minor update to OONI’s Data Policy ( https://github.com/ooni/ooni.org/pull/1221) reiterating our use of Sentry in the "Third party services" section. The updated version of OONI’s Data Policy is available here: https://ooni.org/about/data-policy *## Expanding censorship measurement methodologies### uTLS-light* We wrote a prototype fork (“uTLS-light”) of Golang’s crypto/tls library that is able to parrot the ClientHello of Chrome browser. This prototype is specifically designed to make merging from upstream as simple and as straightforward as possible. The main drawback of this prototype is that, compared to similar efforts such as uTLS, it is not configurable, as users of uTLS-light always use the Chrome browser’s configuration (which is what we need as OONI to perform measurements). Our work on developing uTLS-light is available here: https://github.com/hellais/utls-light *### Improving Web Connectivity experiment* We continued testing Web Connectivity v0.5 with support from users in our community. Based on testing and community feedback, we made a series of improvements: * https://github.com/ooni/probe-cli/pull/933 * https://github.com/ooni/probe-cli/pull/935 * https://github.com/ooni/probe-cli/pull/943 * https://github.com/ooni/probe-cli/pull/948 * https://github.com/ooni/probe-cli/pull/951 * https://github.com/ooni/probe-cli/pull/952 * https://github.com/ooni/probe-cli/pull/953 * https://github.com/ooni/probe-cli/pull/957 * https://github.com/ooni/probe-cli/pull/958 * https://github.com/ooni/probe-cli/pull/959 * https://github.com/ooni/probe-cli/pull/960 * https://github.com/ooni/probe-cli/pull/961 * https://github.com/ooni/probe-cli/pull/965 The main focus of these changes was to improve the data quality and correctly flag edge cases to minimize false positives and detect cases where websites are down. In the meanwhile, we spent time testing the test helper improvements to support Web Connectivity v0.5. We performed A/B testing for a few days and then decided to switch all users to the new test helper version. We also improved our measurement scrubbing procedure, by extending the scrubbing to all the fields of a measurement, as opposed to limiting it to the test keys (https://github.com/ooni/probe-cli/pull/956) All this work culminated in the release of OONI Probe CLI v3.16.3: https://github.com/ooni/probe-cli/releases/tag/v3.16.3 *### Experiments developed by Google Summer of Code (GSoC) student* We merged the TLS middlebox experiment ( https://github.com/ooni/probe-cli/pull/817) and port filtering experiment ( https://github.com/ooni/probe-cli/pull/891) contributed by Mehul Gulati, as part of his Google Summer of Code (GSoC) with OONI. We wrapped up the GSoC with a comprehensive recap of the work that was done in the following ticket: https://github.com/ooni/probe/issues/2272 *## OONI Explorer* In September 2022, we completed most development work for the new domain-centric pages of OONI Explorer ( https://github.com/ooni/explorer/pull/762) We refactored the OONI Explorer Search tool to move measurement requests to the client side ( https://github.com/ooni/explorer/pull/799) We continued to make progress towards building the user feedback reporting system, and enabling localization on OONI Explorer. We also replaced Babel with SWC ( https://github.com/ooni/explorer/pull/804) fixed builds, and reviewed pull requests. *## OONI backend* In September 2022, we worked on the following main backend activities: * Improvements to API rate limiting (https://github.com/ooni/api/pull/307) * Added support for filtering by IP address in the aggregation API ( https://github.com/ooni/api/pull/308) * Added database query prioritization in the API ( https://github.com/ooni/api/pull/304) and fastpath ( https://github.com/ooni/pipeline/pull/397) * Improved upon the backend prototype for the user feedback reporting system (https://github.com/ooni/api/pull/302) * Added bot mitigations in the API using robots.txt; * Added API metrics and dashboards; * Investigated test failures; * Implemented an initial scoring table for OpenVPN and obfs4 ( https://github.com/ooni/pipeline/pull/399) *## Published user guide for Test Lists Editor* In September 2022, we published a user guide for our new Test Lists Editor ( https://test-lists.ooni.org/) Our user guide is available here: https://ooni.org/support/test-lists-editor This user guide provides step-by-step instructions (with screenshots) on how to use our new Test Lists Editor for contributing to the lists of websites tested by OONI Probe users around the world. *## Published test list guide for GitHub users* Previously, we used to include instructions for updating the Citizen Lab test lists (https://github.com/citizenlab/test-lists/tree/master/lists) via GitHub in our general test lists documentation. In September 2022, we created a separate guide for GitHub users with updated instructions. Our user guide for updating test lists via GitHub is available here: https://ooni.org/support/github-test-lists *## Published updated test list documentation* We updated our general test list documentation: https://ooni.org/get-involved/contribute-test-lists This documentation explains what test lists are, why and how to contribute to test lists (while linking to more detailed guides), and how to approach test list research. *## Test list updates* In September 2022, we contributed updates to the following test lists: * Global (https://github.com/citizenlab/test-lists/pull/1107, https://github.com/citizenlab/test-lists/pull/1128; https://github.com/citizenlab/test-lists/pull/1114) * Azerbaijan (https://github.com/citizenlab/test-lists/pull/1111) We reviewed test list updates provided by Netalitica researchers and, on their behalf, opened pull requests for the following test lists: * Hong Kong (https://github.com/citizenlab/test-lists/pull/1115) * Poland (https://github.com/citizenlab/test-lists/pull/1116) We opened a pull request with updates to the test list for Kyrgyzstan, provided by the School of Peacemaking ( https://github.com/citizenlab/test-lists/pull/1124) We also reviewed and merged multiple pull requests ( https://github.com/citizenlab/test-lists/pulls?q=is%3Apr+is%3Aclosed) submitted by community members (via both GitHub and our new Test Lists Editor). *## Creating a new OONI Outreach Kit* To better meet community needs (particularly around OONI community engagement), we worked towards creating multiple new materials and resources that will eventually get published as part of a new OONI Outreach Kit. These resources include OONI-related brochures, fact sheets, workshop slides, and screencasts. *## Launch of Advocacy Assembly OONI training course* On 30th September 2022, Small Media’s Advocacy Assembly launched an online OONI training course ( https://advocacyassembly.org/en/courses/63/#/chapter/1/lesson/1) We had previously collaborated with Advocacy Assembly on creating the content of the OONI training course, while they provided amazing support for everything related to editing, designing, and translating the course. Prior to the launch, Advocacy Assembly published a blog post which includes an interview with the OONI team about the course: https://advocacyassembly.org/en/news/228/ *## Improving OONI’s Code of Conduct* Based on feedback from community meetings and discussions, we re-wrote OONI’s Code of Conduct (in collaboration with OONI’s new Incident Response Committee). The updated version of OONI’s Code of Conduct is available here: https://deploy-preview-1245--ooni.netlify.app/get-involved/code-of-conduct/ We shared the updated version with community members during the September 2022 OONI Community Meeting, with the goal of soliciting further community review and feedback. Based on the latest community feedback, we plan to make further improvements to the Code of Conduct before it is published. *## Ford Foundation Retreat* Between 19th-22nd September 2022, OONI’s Maria attended the Ford Foundation Retreat in Atlanta, USA. As part of her participation at the retreat, Maria facilitated a session with the goal of collecting feedback on how internet measurement projects can better meet advocacy needs. *## Community activities### OONI hackathon at DEMHACK 2022* On 24th and 25th September 2022, OONI’s Elizaveta organized an OONI hackathon as part of Roskomsvoboda’s DEMHACK 2022 hackathon ( https://demhack.ru/) in Russia. Hackathon participants were invited to work on the following OONI hackathon challenges (https://demhack.ru/tasks) * Updating the lists of websites (“test lists”) measured for censorship; * Developing a tool for the automatic clean-up of test lists (e.g. removing expired domains); * Analyzing OONI data to investigate internet censorship. As part of the hackathon, OONI’s Maria provided a live demo on using OONI’s Measurement Aggregation Toolkit (https://explorer.ooni.org/chart/mat) encouraging participants to use the tool as a starting point for further analyzing OONI data. *### OONI workshops at FIFAfrica 2022* Between 26th-29th September 2022, OONI’s Arturo traveled to Lusaka, Zambia, to participate in the Forum on Internet Freedom in Africa (FIFAfrica) 2022 conference organized by CIPESA – one of Africa’s largest digital rights events (https://cipesa.org/fifafrica/) As part of his participation (https://cipesa.org/fifafrica/agenda/) Arturo: * Facilitated an OONI workshop (“Investigating Internet censorship in Africa through OONI tools and open data”); * Co-facilitated a workshop with Access Now (“Strengthening Advocacy to Fight Internet Shutdowns in Africa”). *### OONI Community Meeting* On 27th September 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/) during which we discussed the following topics: 1) Improving OONI’s Code of Conduct: Request for community feedback (based on the updated version) 2) Reconsidering test list policies around expired or parked domains *## Userbase* In September 2022, 60,831,677 OONI Probe measurements were collected from 2,765 AS networks in 170 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0

OONI Monthly Report: August 2022
by Maria Xynou 19 Oct '22

19 Oct '22

Hello, OONI's August 2022 status report is shared below. *# OONI Monthly Report: August 2022* Throughout August 2022, the OONI team worked on the following sprints: * Sprint 71 (1st-14th August 2022) * Sprint 72 (15th-28th August 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## New OONI Team member* In August 2022, Elizaveta Yachmeneva joined the OONI team as our new Community Coordinator! Elizaveta will support the global OONI community and help expand our community work. *## OONI Probe Mobile* In August 2022, we released OONI Probe Android 3.7.1 with bug fixes and improvements (https://github.com/ooni/probe-android/releases/tag/v3.7.1) We continued to work on a prototype of the OONI Probe app using Flutter, which will allow us to have a shared codebase between the OONI Probe mobile and desktop apps. Specifically, we experimented with connecting Flutter with the OONI Probe engine (https://github.com/ooni/probe/issues/2197) In particular, we experimented with three distinct approaches ( https://github.com/ooni/probe-cli/pull/849, https://github.com/ooni/probe-cli/pull/850, https://github.com/ooni/probe-cli/pull/851) to expose the OONI Probe engine to code written in Dart (which is the language used by Flutter). Once that work was done, we adapted our app prototypes to use the recent OONI Probe engine changes and generate PoC apps for mobile and desktop using Flutter ( https://github.com/ooni/probe-cli/pull/880) We also looked into Sentry issues (https://github.com/ooni/probe/issues/2187, https://github.com/ooni/probe/issues/2186) Google Play errors, and failing CI on OONI Probe Android. *## OONI Run* Following the implementation of OONI Run v2 for miniooni users (in July 2022), we wrote a design document which describes the miniooni OONI Run v2 functionality (https://github.com/ooni/probe-cli/pull/916) *## OONI Probe Desktop* In response to community feedback, we fixed the summary table for Tor measurements on OONI Probe Desktop ( https://github.com/ooni/probe/issues/2209) and we looked into fixing inconsistencies when displaying results ( https://github.com/ooni/probe/issues/2210) *## Worked on updating the OONI Probe user guides* Throughout August 2022, we worked on extensive updates to the OONI Probe mobile and desktop user guides to ensure that they include information and screenshots for features based on the latest releases. This work is documented through the following pull request: https://github.com/ooni/ooni.org/pull/1202 *## Test Lists Editor* Following the launch of OONI’s Test Lists Editor in July 2022 ( https://ooni.org/post/2022-test-lists-editor/) we continued to work on improvements to the platform. Specifically, we improved the UI of the “Add new URL” section (https://github.com/ooni/test-lists-ui/issues/61) we fixed an issue with repeated background requests to account metadata endpoint (https://github.com/ooni/test-lists-ui/issues/62) and we adapted the login request to work with the new API changes. *## Expanding censorship measurement methodologies* In previous months, we created Websteps ( https://github.com/bassosimone/websteps-illustrated/) which includes significant improvements to how we measure website blocking. But shipping Websteps would create a wide range of complexities for interdependent OONI software components and data structures. We therefore decided that rather than shipping Websteps, we will instead improve upon our Web Connectivity experiment based on Websteps. To this end, we drafted and implemented a plan ( https://github.com/ooni/probe/issues/2237) for improving upon our Web Connectivity experiment by incrementally integrating selected features and experiments from our Websteps prototype ( https://github.com/bassosimone/websteps-illustrated/) into Web Connectivity. During August 2022, we created a new version of Web Connectivity (“Web Connectivity v0.5”) based on the “step-by-step” style ( https://github.com/ooni/probe-cli/pull/814) which aims to incrementally integrate features from Websteps. The initial implementation of Web Connectivity v0.5 was introduced through the following pull request: https://github.com/ooni/probe-cli/pull/882 Subsequent changes focused on collecting late, duplicate DNS replies, which often provide a clear indication of censorship ( https://github.com/ooni/probe-cli/pull/878, https://github.com/ooni/probe-cli/pull/883) as well as on improving the Web Connectivity test helper to support v0.5 measurements ( https://github.com/ooni/probe-cli/compare/c3964e43b33d36b7cba371e9906d5ed0b…) One of these patches (https://github.com/ooni/probe-cli/pull/878) was contributed by Mehul Gulati, one of our Google Summer of Code (GSoC) students. We improved the code used by Web Connectivity v0.5 to simplify the task of splitting measurements into observations ( https://github.com/ooni/probe-cli/pull/879, https://github.com/ooni/probe-cli/pull/873) thus enabling efforts aimed at automating the detection and characterization of more forms of website censorship. We also improved DNS measurement code to collect CNAMEs from DNS responses (https://github.com/ooni/probe-cli/pull/875, https://github.com/ooni/probe-cli/pull/876, https://github.com/ooni/probe-cli/pull/877) To write this new version of Web Connectivity in a more automated way, we specified and drafted a code generation tool called “boilerplate” ( https://github.com/ooni/probe-cli/pull/853) This tool should simplify maintaining code used for performing specific measurements using the “step-by-step” style. As part of research on censorship events in Indonesia, we added support for running measurements in parallel (https://github.com/ooni/probe-cli/pull/848) which helps with running our new experiment (“DNS Scan”) for scanning open DNS resolvers to detect blocking. ( https://github.com/ooni/probe-cli/pull/847) Lastly, the work on allowing Dart clients to use the OONI Probe engine (described above) highlighted the need to cleanup OONI Probe CLI build scripts. To this end, we refactored the Makefile to defer most of its work to support scripts (https://github.com/ooni/probe/issues/2217, https://github.com/ooni/probe/issues/2218) *## OONI Explorer### Domain-centric pages* Throughout August 2022, we made progress on developing domain-centric pages for OONI Explorer (https://github.com/ooni/explorer/pull/762) We also implemented a new API endpoint for returning domain metadata that we can display in the new domain-centric pages ( https://github.com/ooni/api/pull/300) *### OONI accounts* We wrote an internal design document which outlines our plans for implementing authentication to OONI services for OONI accounts. As part of this, we documented the primary use cases (user stories) for these accounts and the implementation details. Generally, the OONI accounts are being created to support the following use cases: * Presenting website measurements to account users; * Crowdsourcing false positives and fingerprints for blocking detection; * Enabling OONI Probe users to view their own measurements; * Monitoring the success of measurement campaigns; * Creating and monitoring OONI Run links; * Enabling users to request higher testing priority for specific URLs; * URL submission and management. Based on our design document, we implemented a modal for the login system and worked on relevant code refactoring. We also added relevant backend support for authentication (https://github.com/ooni/api/pull/298) *### User feedback reporting system* We created a design document for the (upcoming) OONI Explorer user feedback reporting system, which will enable users to share feedback via OONI Explorer to report false positives, or share other information that is useful for evaluating measurements (such as information about blocked websites). Based on this, we implemented logic functionality in OONI Explorer as a base for our (upcoming) user feedback reporting system ( https://github.com/ooni/explorer/issues/774) We also implemented an initial prototype in the backend which enables registered users to submit, view, and edit feedback for measurements ( https://github.com/ooni/api/pull/302) *### Other OONI Explorer improvements* We continued to work towards adding localization support to OONI Explorer by updating Transifex strings, coordinating with the Localization Lab, and fixing relevant issues (https://github.com/ooni/explorer/pull/778) We also improved upon OONI Explorer by removing links to external pages on measurement pages (https://github.com/ooni/explorer/issues/783) bug fixing (https://github.com/ooni/explorer/pull/777) including failed measurements in the MAT tooltip (https://github.com/ooni/explorer/issues/779) and by upgrading dependencies. *## OONI backend* Notably, we switched all our infrastructure over to the new backend host and we completed the migration to the Clickhouse database ( https://github.com/ooni/pipeline/pull/380) We continued to test and make improvements to the new ASN-based URL prioritization system, we monitored the availability of our test helpers and improved internal dashboards, we fixed measurement bugs, we improved upon the API uploader, and we continued to improve upon JSONL tables ( https://github.com/ooni/api/pull/297/) We handled an incident that involved an attack on our infrastructure. Specifically, we added IP-address based filtering as a temporary mitigation, then more strict filtering on inputs on API calls ( https://github.com/ooni/api/pull/301) and also extended API caching to more entry points. We fixed a backend bug (https://github.com/ooni/pipeline/pull/396) that was impacting the correct annotation of measurements ( https://github.com/ooni/explorer/issues/764) We added DNS fingerprints to our database to automatically detect and confirm more cases of website blocking in Iran (https://github.com/ooni/pipeline/pull/390, https://github.com/ooni/backend/issues/543) We also created a diagram which shows the process through which measurements are processed and published (https://github.com/ooni/pipeline/pull/391) *## Advancing OONI data analysis capabilities* We have been working on a set of tools (https://github.com/ooni/data) that make it easier for us and third parties to analyze OONI data as part of research and to expand the capabilities of our data processing pipeline. In particular, we wrote parsers for several common data formats used in OONI measurements and evaluated the performance ( https://github.com/hellais/oonidata-bench) of several approaches to building python data class objects based on these definitions. One of the benefits of having a well-defined schema for all OONI probe tests is that people interested in analyzing OONI data can benefit from type hints and code completion for accessing all the necessary attributes of a particular test, without needing to refer to OONI specifications. Moreover, it takes care of handling some inconsistencies across various revisions of the data generated by older probes so that the access patterns are as smooth as possible. Lastly, the data is being validated to be conformant with the specification, so it’s easier to identify issues affecting data quality. Other aspects worked on include adding the ability to break up some common types of tests into distinct observations and writing them to either a Clickhouse database backend or a flat CSV file on disk. We also started outlining a system for taking these observations and generating verdicts related to censorship around the world. Since we are also using it heavily internally for doing research with OONI data, our goal is to eventually consolidate all the knowledge and patterns we develop into it, so that it can be beneficial to the wider public. We also created a new GitHub repository for the collection of blocking fingerprints (https://github.com/ooni/blocking-fingerprints) with the goal of enhancing our capabilities to automatically confirm more website blocks. *## Data requests* In response to community requests, we shared relevant OONI data (and interpretation) on censorship events. Specifically, we responded to Freedom on the Net’s data requests, providing relevant OONI data and information pertaining to 11 censorship cases. We also coordinated with community members in the Philippines on interpreting relevant OONI data, and we continued to coordinate with members of the #KeepItOn campaign on monitoring blocks. *## Creating a new OONI Outreach Kit* To better meet community needs (particularly around OONI community engagement), we started working towards creating new materials and resources that will eventually get published as part of a new OONI Outreach Kit. In August 2022, we worked on several OONI fact sheets and related brochures. *## OONI blog post on encrypted-DNS censorship published by RIPE Labs* On 4th August 2022, RIPE Labs published our blog post where we discuss our research on encrypted-DNS censorship. This blog post is available here: https://labs.ripe.net/author/bassosimone/measuring-encrypted-dns-censorship… *## Test list updates* In August 2022, we contributed updates to the following test lists: * Philippines (https://github.com/citizenlab/test-lists/pull/1055) * Kenya (https://github.com/citizenlab/test-lists/pull/1051) * Russia (https://github.com/citizenlab/test-lists/pull/1032) * Tajikistan (https://github.com/citizenlab/test-lists/pull/1034) * Georgia (https://github.com/citizenlab/test-lists/pull/1038) * Lebanon and Global (https://github.com/citizenlab/test-lists/pull/1059) *## Started planning the OONI Team Meeting for 2022* In August 2022, we started logistical preparations for organizing the first (since the start of the pandemic) in-person OONI Team Meeting, which will take place between 2nd-4th November 2022 in Rome, Italy. This meeting will bring all OONI team members and OTF research fellows hosted by OONI together for a 3-day event to plan strategic priorities and roadmaps. *## Created a new OONI Code of Conduct Incident Response Committee* In August 2022, we created a new Incident Response Committee with the goal of improving OONI’s Code of Conduct (CoC) and addressing any CoC violations when they occur. We are excited to share that our new CoC Incident Response Committee comprises of the following members: 1) Elizaveta Yachmeneva (OONI) 2) Amanda Meng (IODA, Georgia Tech) 3) Felicia Anthonio (Access Now) 4) Kelly Koh (Sinar Project) 5) Maria Xynou (OONI) 6) Gurshabad Grover (OTF Information Controls Fellow) 7) Federico Ceratto (OONI) Throughout August 2022, we organized several meetings to discuss the goals and responsibilities of the committee, how to improve upon OONI’s Code of Conduct (CoC), and how to handle CoC violation reports. We also worked towards re-writing OONI’s Code of Conduct based on initial community feedback collected during the June 2022 OONI Community Meeting, as well as feedback shared by Code of Conduct experts in our field. *## Community activities### OONI training session for election observers* On 4th August 2022, OONI’s Maria facilitated an OONI training session for election observers. This training introduced participants to OONI Probe and OONI data, particularly with the goal of monitoring emerging censorship events during elections around the world. *### OONI training session for researchers in Egypt* On 10th August 2022, OONI’s Maria facilitated a 3-hour OONI training session for researchers in Egypt. This workshop provided an introduction to internet censorship, as well as to OONI tools and data for measuring internet censorship. *## Userbase* In August 2022, 60,352,386 OONI Probe measurements were collected from 2,798 ASNs in 173 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0

OONI Monthly Report: July 2022
by Maria Xynou 19 Oct '22

19 Oct '22

Hello, These days we're primarily supported by the DRL, which is why we do quarterly reporting. That said, we'd like to share monthly updates from the OONI team with the community, hence our July 2022 report shared below. I'll also follow-up to share our August and September 2022 reports. *# OONI Monthly Report: July 2022* Throughout July 2022, the OONI team worked on the following sprints: * Sprint 69 (1st-17th July 2022) * Sprint 70 (18th-31st July 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## Launched new Test Lists Editor* In July 2022, OONI launched a new Test Lists Editor: https://test-lists.ooni.org/ The Test Lists Editor enables the public to review and contribute to the Citizen Lab test lists ( https://github.com/citizenlab/test-lists/tree/master/lists) without a GitHub account. Their contributions automatically end up as pull requests on the Citizen Lab test list repository, which means they'd still be peer-reviewed. We also published a blog post which provides a brief user guide and shares information about the Test Lists Editor: https://ooni.org/post/2022-test-lists-editor/ *## Updated the OONI Data Policy* In preparation for the launch of the new Test Lists Editor ( https://test-lists.ooni.org/) we updated the OONI Data Policy to include a new section ("OONI web services") that mentions that we do not store email addresses (used to send login links for OONI accounts), and which shares the providers (and their data policies) that we use for deploying web services (https://github.com/ooni/ooni.org/pull/1198) The updated OONI Data Policy is available here: https://ooni.org/about/data-policy *## OONI Probe Mobile* In July 2022, we released OONI Probe Mobile 3.7.0 for: * Android: https://github.com/ooni/probe-android/releases/tag/v3.7.0 * iOS: https://github.com/ooni/probe-ios/releases/tag/v3.7.0 This release includes: * New Vanila Tor experiment; * Automated testing of experimental tests; * In-app language selection; * Support for sharing a link to the OONI Probe app; * Measurement engine synced with OONI Probe CLI 3.15.2; * Bug fixes and improvements. While working towards the OONI Probe Mobile 3.7.0 release, we also: * Fixed a bug that prevented the VPN warning from appearing when running an OONI Run link (https://github.com/ooni/probe-ios/pull/496) * Worked towards improving the app performance when loading and showing measurements (https://github.com/ooni/probe/issues/2150) * Fixed a bug that affected storage usage ( https://github.com/ooni/probe/issues/2149) * Worked towards improving the pagination of test results ( https://github.com/ooni/probe/issues/1430) * Updated the measurement-log retention policy ( https://github.com/ooni/probe/issues/2162) * Worked on a prototype of the OONI Probe app using flutter, which will allow us to have a shared codebase between the OONI Probe mobile and desktop apps. *## OONI Run* In an attempt to meet community needs (particularly those of researchers performing custom testing), we designed and implemented a minimal version of the next generation version of OONI Run (“OONI Run v2”) that enables community members to run OONI Run links without backend support ( https://github.com/ooni/probe-cli/pull/844) We made this functionality available to miniooni users with the goal of enabling community members to start experimenting with this new OONI Run version ( https://github.com/ooni/probe/issues/2184) and share feedback which can support further development and design decisions. Based on community requests, we also introduced a command line flag which enables users to repeat a measurement every given number of seconds (until OONI Run v2 has support for repeating the measurement with a schedule): https://github.com/ooni/probe-cli/pull/819 As part of our plan for incrementally releasing a new version of OONI Run ( https://run.ooni.io/) we started updating the specification for the new version of OONI Run based on our experimental miniooni-based deployment ( https://github.com/ooni/spec/pull/249) *## Expanding censorship measurement methodologies* We worked on data quality improvements to OONI measurements. Specifically, we improved the way in which the Web Connectivity test helper works, by using a single, independent HTTP Client per test helper request, to avoid hitting the limit of maximum connections per HTTP Client ( https://github.com/ooni/probe/issues/2182) We supported our Google Summer of Code (GSoC) student in developing a TLS middlebox experiment (https://github.com/ooni/probe-cli/pull/817) and experimented around improving how we perform DNS resolution to enable the opportunistic measurement of DoH endpoints ( https://github.com/ooni/probe-cli/pull/846) We also reviewed the work of an OTF Information Controls Research Fellow that we’re currently hosting ( https://github.com/ooni/minivpn/pull/10, https://github.com/ooni/minivpn/pull/12) *## OONI Explorer* As part of our work on improving OONI Explorer, we worked on: * Adding support for localization (https://github.com/ooni/explorer/pull/705 ); * Implementing domain-centric pages ( https://github.com/ooni/explorer/pull/762) * Maintenance, package upgrades, and fixing security vulnerabilities; * Improving the SEO of the Measurement Aggregation Toolkit (MAT). *## OONI backend* In July 2022, we continued to work on the migration to the Clickhouse database. This involved a series of backend tasks including the testing and switching to more API entry points, numerous configuration changes, and end-to-end testing of the processing of measurements. We fixed various bugs in the fastpath, including the handling of the measurement start time. We worked on the reprocessing of legacy measurements, which involved the deletion of old legacy measurements from the fastpath pipeline table, fixing a bug and adding metrics for the fetching of measurements across API hosts, and improving upon JSONL tables, among many other backend tasks ( https://github.com/ooni/api/pull/297) We improved the API performance by enabling gzip compression of measurements. We created internal dashboards to monitor and test the new ASN-based URL prioritization system (https://github.com/ooni/api/pull/293) We also worked on improvements to the new test helper ( https://github.com/ooni/pipeline/pull/392) and performed security updates to the infrastructure. *## Interviewing Community Coordinator candidates* In July 2022, we carried out and completed the second (and final) round of interviews for the OONI Community Coordinator position ( https://ooni.org/post/2022-job-opening-ooni-community-coordinator/) We were thrilled to have received applications from (and to have the opportunity to meet) many strong candidates. Based on both rounds of interviews, we decided to hire Elizaveta Yachmeneva for the OONI Community Coordinator position (who joined the OONI team in August 2022). *## Test list updates* In July 2022, we updated the test list for the Philippines to include websites that were reportedly blocked ( https://github.com/citizenlab/test-lists/pull/1022) *## New blocks emerged in Indonesia* On 29th July 2022, new blocks emerged in Indonesia. We shared relevant OONI data (and findings) with the #KeepItOn advocacy community. Specifically, the blocked services included: * PayPal: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&… * Yahoo: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&… * Steam: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&… * Origin: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&… * Epic Games: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&… * Dota2: https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&… In many of the above cases, we were able to automatically confirm the blocks based on the IP returned from DNS resolution. In other cases, we were able to confirm the blocks because they were implemented through an HTTP middlebox serving a blockpage ( https://explorer.ooni.org/measurement/20220731T055038Z_webconnectivity_ID_4…) To encourage further testing of the blocked services, we shared an OONI Run link with the #KeepItOn community. *## OONI citations### Digital Defenders Partnership publication* Digital Defenders Partnership published a guide (in Spanish) describing how to document internet blockages and circumvent internet censorship. This publication, which cites OONI and describes OONI tools, is available here: https://www.digitaldefenders.org/wp-content/uploads/2022/09/2207-ISG_final-… *### OPTIMA report on the impact of internet shutdowns on women in Uganda* Sandra Aceng published a research report (as part of Internews’ OPTIMA project) which documents the impact of internet shutdowns on women in Uganda. This study makes use of OONI data collected from Uganda around the country’s 2021 elections. The report is available here: https://preparepreventresist.org/wp-content/uploads/2022/07/Impact-of-Shutd… *## Community activities### OONI training session for journalists in Kenya* On 28th July 2022, OONI’s Maria facilitated an OONI training session for journalists in Kenya. The workshop involved introducing participants to OONI tools and data for investigating internet censorship in Kenya and around the world. *### OONI Community Meeting* On 26th July 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/) during which we discussed the following topics: 1) Using OONI Explorer to group measurements by region 2) Request for community feedback: Do users want to have the option to delete specific measurements manually or automatically? 3) Request for community feedback on write/erase cycle limits *## Userbase* In July 2022, 47,752,189 OONI Probe measurements were collected from 2,892 ASNs in 167 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0

New OONI Code of Conduct
by Elizaveta Yachmeneva 18 Oct '22

18 Oct '22

Hello all! Today OONI published the updated version of the OONI Code of Conduct: https://ooni.org/get-involved/code-of-conduct/ <https://ooni.org/get-involved/code-of-conduct/> This version of the Code of Conduct is a result of the incredible work of our community members, who initiated the discussion and provided numerous suggestions and insights, and of the Incident Response Committee members, who drafted and thoroughly reviewed the whole document together with our team. This document is an ongoing effort, so if you have any suggestions or ideas of how it should be changed to better meet the needs of our community, please get in touch with our Incident Response Committee by sending an email to coc(a)openobservatory.org. Please read the new version of the Code of Conduct <https://ooni.org/get-involved/code-of-conduct/> carefully and follow its guidelines. We believe that this set of rules will help us to keep our community spaces safe and welcoming for people of all backgrounds and to create a supportive environment for everyone. I hope you enjoy the reading and please let us know if you have any feedback or suggestions for this document! <3 Best wishes, Elizaveta

1 0

New online OONI training course launched by Advocacy Assembly
by Maria Xynou 03 Oct '22

03 Oct '22

Hello friends, We are excited to share that a *free, online OONI training course* (“Measuring Internet Censorship with OONI tools”) has been launched on Small Media’s Advocacy Assembly platform! The OONI course is available in: * English: https://advocacyassembly.org/en/courses/63/#/chapter/1/lesson/1 * Farsi: https://advocacyassembly.org/fa/courses/63/#/chapter/1/lesson/1 * Arabic: https://advocacyassembly.org/ar/courses/63/#/chapter/1/lesson/1 * Spanish: https://advocacyassembly.org/es/courses/63/#/chapter/1/lesson/1 Through the above links, you can sign up with Advocacy Assembly and enroll for free! The course is estimated to take around *90 minutes*, and it includes a variety of *videos, screencasts, slides, quizzes and hands-on exercises*. No technical knowledge is required, as the course was designed for human rights defenders, journalists, and researchers. The online OONI training course covers: * Introduction to internet censorship * Measuring internet censorship with OONI Probe * Understanding OONI censorship measurement data * OONI Explorer: Accessing real-time censorship measurement data Notably, the course also includes 2 case study videos by the amazing *Zaituni Njovu* (Zaina Foundation) and *Felicia Anthonio* (Access Now), who discuss measuring internet censorship in Tanzania and as part of the #KeepItOn campaign. Learn more about the course through our blog post: https://ooni.org/post/2022-ooni-training-course-advocacy-assembly/ We have also tweeted about it here: https://twitter.com/OpenObservatory/status/1576888851934892038 We thank Advocacy Assembly for providing us with this amazing opportunity to create an OONI training course for their platform. We also thank translators and everyone else behind the scenes who helped with the creation of the OONI training course. <3 We hope you enroll and find the course useful! Cheers, Maria.

1 0

PT community meetup September [29/09/2022]
by Vasilis 27 Sep '22

27 Sep '22

Dear all, This month Pluggable Transports meetup will take place on Tuesday, September 29. Time: 10:00 AM EDT / 2:00 PM UTC / 16:00 PM CEST Where: Pluggable Transports channel * [Matrix](https://matrix.to/#/#pluggable-transports:matrix.org) * [IRC (no account required)](https://webchat.oftc.net/?channels=pluggable-transports) * [Slack](https://openobservatory.slack.com/messages/pluggable-transports/) * [Mattermost](https://community.internetfreedomfestival.org/community/channel… Please add any topics you would like to discuss during the meeting here: https://pad.riseup.net/p/pt-meetup-keep More info and details on how to join: https://pluggabletransports.info/blog/pt-meetup-sep2022/ Hope to see many of you there. Best, ~Vasilis -- PGP Fingerprint: 8FD5 CF5F 39FC 03EB B382 7470 5FBF 70B1 D126 0162 PGP Public Key: https://keys.openpgp.org/vks/v1/by-fingerprint/8FD5CF5F39FC03EBB38274705FBF…

1 0

OONI Community Meeting: Tomorrow, 27th September 2022
by Maria Xynou 26 Sep '22

26 Sep '22

Hello, The OONI team warmly welcomes you to join us tomorrow for our monthly community meeting. *=> Where? *OONI Slack channel: https://slack.ooni.org/ (bridged with IRC: ircs://irc.oftc.net:6697/#ooni) *=> When?* Tomorrow, *27th September 2022 at 14:00 UTC* (for 1 hour) As part of the meeting, we'll be discussing the *improved version of OONI's Code of Conduct* and we'll be requesting your feedback for further improvements. Please add any additional topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep We hope you can join us! Thanks, ~ OONI team

1 0

New Report: Iran blocks social media, app stores and encrypted DNS amid protests
by Maria Xynou 26 Sep '22

26 Sep '22

Hello, Today OONI published a new report documenting the latest censorship events in Iran amid the ongoing protests. Read the report here: https://ooni.org/post/2022-iran-blocks-social-media-mahsa-amini-protests/ *# Key Findings* OONI data shows new blocking events in Iran amid ongoing protests: ** Increased blocking of encrypted DNS. *As of 20th September 2022, OONI data shows that ISPs in Iran started blocking DNS over HTTPS (DoH) endpoints by means of DNS based tampering (returning bogons), in addition to implementing TLS level interference. Previously reachable DoH endpoints (such as cloudflare-dns.com) are now blocked. ** Blocking of WhatsApp and Instagram. *As of 21st September 2022, OONI data shows that many ISPs in Iran started blocking access to WhatsApp endpoints, web.whatsapp.com, and www.instagram.com. We mostly observe TCP/IP blocking, with additional DNS and TLS blocking. ** Blocking of Google Play Store and Apple App Store. *As of 22nd September 2022, OONI data shows that multiple ISPs in Iran started blocking access to Google Play Store and Apple App Store domains. ** Blocking of Skype and Linkedin. *As of 23rd September 2022, OONI data shows the blocking of Skype, while the blocking of Linkedin is automatically confirmed on at least 10 networks in Iran. In addition to the above blocks (and many other long-term blocks), Iran experienced *5 major outages affecting mobile networks* over the past week, which are visible in the IODA, Cloudflare Radar and Kentik datasets. OONI data suggests that Psiphon and Tor Snowflake work in Iran, and can potentially be used for censorship circumvention. We also share the key findings of our study in this Twitter thread: https://twitter.com/OpenObservatory/status/1574108767226155008 You can continue to monitor censorship events in Iran through real-time OONI data: https://explorer.ooni.org/chart/mat?probe_cc=IR&test_name=web_connectivity&… We thank OONI Probe users in Iran who contributed measurements, supporting this study. <3 ~ OONI team.

1 0

New Report: Azerbaijan and Armenia block TikTok amid border clashes
by Maria Xynou 16 Sep '22

16 Sep '22

Hello, Today OONI and Azerbaijan Internet Watch published a new research report which shares *OONI data on the* *blocking of TikTok in Armenia and Azerbaijan* during the border clashes over the last days. Read our report here: https://ooni.org/post/2022-azerbaijan-and-armenia-blocks-tiktok/ In *Azerbaijan*, OONI data shows: * TikTok website & endpoints blocked on at least 3 networks; * TLS level interference by dropping packets after noticing a disallowed server_name; * ISPs block TikTok based on different lists of TikTok endpoint domains. In *Armenia*, OONI data shows: * TikTok blocked on at least 2 networks; * On most networks, TikTok interference observed at the DNS level by returning an NXDOMAIN error, an IP address associated with www.google.com, or a set of unrelated IP addresses which contain an expired certificate for the server_name it.domain.name; * Not all tested networks in Armenia implemented the TikTok block and censorship techniques varied from network to network; * Several endpoints used by the TikTok app and website are also blocked on at least one network in Armenia. Overall, OONI data shows that Azerbaijan blocked TikTok by means of TLS level interference, while Armenia implemented both DNS level blocks and TLS level interference. Learn more through our report: https://ooni.org/post/2022-azerbaijan-and-armenia-blocks-tiktok/ We thank OONI Probe users in Armenia and Azerbaijan who contributed measurements, supporting this study. You can continue to monitor the reachability of TikTok (and other platforms) in Armenia and Azerbaijan through OONI data: https://explorer.ooni.org/chart/mat Thanks, Maria.

1 0