ooni-talk

ooni-talk@lists.torproject.org

1 participants
451 discussions

OONI Monthly Report: February 2022
by Maria Xynou 18 Apr '22

18 Apr '22

Hello, OONI's February 2022 status report is shared below. *# OONI Monthly Report: February 2022* Throughout February 2022, the OONI team worked on the following sprints: * Sprint 58 (1st - 13th February 2022) * Sprint 59 (14th - 27th February 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## OONI submission for the OHCHR report on internet shutdowns* On 10th February 2022, we sent a submission in response to the UN High Commissioner for Human Rights’s call for submissions in support of OHCHR report on internet shutdowns and human rights to the fiftieth session of the Human Rights Council in June 2022 ( https://www.ohchr.org/en/Issues/CivicSpace/Pages/cfi-report-hrc-50.aspx). In particular, the OONI submission provides information on the occurrence of mandated disruptions of access to social media and messaging platforms over the past 5 years based on empirical OONI network measurement data. Through our submission, we share information on social media blocks that occurred during elections in Uganda, Tanzania, Mali, Benin, Togo, Burundi, and Zambia over the last 5 years (based on OONI data). We also share information on social media blocks that occurred during protests in Pakistan, Jordan, Iran, Zimbabwe, and Cuba. Moreover, we share OONI data and information on social media blocks that emerged during sensitive political time periods in Ethiopia, Myanmar, and Venezuela. In our submission, we also recommend that the OHCHR considers expanding this scope to also report on cases that involve the blocking of websites of marginalized communities. *## OONI Probe Mobile* Throughout February 2022, we worked towards the OONI Probe Android 3.6.0 release. In particular, we worked on the following OONI Probe Android improvements: * Worked on ensuring that the VPN modal is displayed prior to the running of tests (https://github.com/ooni/probe/issues/1982); * Improved the UI for minimizing a running test ( https://github.com/ooni/probe/issues/1998); * Fixed a bug affecting the display of the indication that a proxy is being used (https://github.com/ooni/probe/issues/1919); * Fixed a bug affecting Web Connectivity tests ( https://github.com/ooni/probe/issues/1328); * Improved the performance of the test startup time ( https://github.com/ooni/probe/issues/2012); * Improved the progress bar of tests ( https://github.com/ooni/probe/issues/2021, https://github.com/ooni/probe/issues/1936); * Fixed a bug affecting the OONI backend ( https://github.com/ooni/probe/issues/1922); * Disabled the RiseupVPN test from OONI Probe ( https://github.com/ooni/probe/issues/2001). We also worked on the following OONI Probe iOS improvements: * Added links to the OONI blog and research reports in OONI Probe iOS ( https://github.com/ooni/probe/issues/1850); * Fixed a bug that caused the OONI Probe iOS app to crash with a malformed URL (https://github.com/ooni/probe/issues/1978). In preparation for the upcoming release of our new Tor Snowflake experiment, we wrote a test description, which we published on our website: https://ooni.org/nettest/tor-snowflake/ *## OONI Run* We continued to make incremental improvements to OONI Run based on community feedback. In particular, we fixed a bug affecting the copy-pasting of URLs into the OONI Run platform ( https://github.com/ooni/run/issues/82). *## OONI Probe Desktop* Throughout February 2022, we worked towards the OONI Probe Desktop 3.7.0 release. In late February 2022, we released the release candidate for OONI Probe Desktop 3.7.0: https://github.com/ooni/probe-desktop/releases/tag/v3.7.0-rc.1 In particular, we worked on the following OONI Probe Desktop improvements: * Fixed a bug affecting website testing ( https://github.com/ooni/probe/issues/1848); * Fixed a bug affecting the “test duration limit” (following recent OONI Probe CLI changes); * Fixed a bug affecting the RiseupVPN test result screen ( https://github.com/ooni/probe-desktop/pull/272); * Ensured that OONI Probe Desktop defaults to English on unsupported languages (https://github.com/ooni/probe/issues/1914); * Disabled the RiseupVPN test from OONI Probe Desktop ( https://github.com/ooni/probe/issues/2002). *## OONI Probe Command Line Interface (CLI)* We released OONI Probe CLI 3.14.1 along with new mobile libraries for Android and iOS. You can read the changelog here: https://github.com/ooni/probe-cli/releases/tag/v3.14.1 Some highlights: * We added the DNS Check test to the experimental test suite; * We added the Tor Snowflake test to the experimental test suite; * We made `ooniprobe list` approximately 7 times faster than before; * We fixed our mobile libraries to ensure that all measurements are submitted. Previously, there were cases when the last measurement in a Web Connectivity run would not be submitted when the experiment timeout occurred right before an attempt to submit this measurement (see: https://github.com/ooni/probe/issues/2037); * We fixed a bug which prevented the NDT test from working on Android 6 phones due to the bundling of a very old certification authority ( https://github.com/ooni/probe/issues/2031); * We added a workaround to handle an inconsistency in how Android 10 NXDOMAIN errors are reported which would otherwise cause measurements to be marked as failed. This was done by tweaking our error classifier for this error condition (https://github.com/ooni/probe/issues/2029) and we started working on a more comprehensive fix for v3.15. *## Expanding censorship measurement methodologies* In preparation for the OONI Probe CLI 3.14 release, we performed extensive QA of the new Tor Snowflake experiment ( https://github.com/ooni/probe/issues/1917 and https://github.com/ooni/probe/issues/2004) where we discussed the proper experiment configuration with Tor Snowflake developers. We eventually settled for a configuration where we allow tor to cache on disk its descriptors to increase the chances that tor could successfully bootstrap using Snowflake. We determined that bootstrapping issues are probably caused by the Snowflake bridge being at capacity, which is a problem being addressed upstream. We started researching how we could use data we already collect to detect cases of heavy throttling. We developed a prototype experiment, called “download”, to explicitly investigate these conditions ( https://github.com/ooni/probe/issues/2025). We applied some of the preliminary knowledge we gathered from this new experiment to detect cases of Twitter throttling using data already collected by Web Connectivity in our report on censorship in Russia amid the conflict in Ukraine ( https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/#twitter-throt… ). We discovered and documented issues with the RiseupVPN experiment ( https://github.com/ooni/probe/issues/1999) and decided to temporarily disable the experiment from the OONI Probe apps ( https://github.com/ooni/probe/issues/2000) until those fundamental issues have been resolved. We also discovered an issue with the tor experiment, where security changes in the upstream obfs4 library caused the NTOR handshake to fail with some of the bridges we are currently testing ( https://github.com/ooni/probe/issues/1970 and https://github.com/ooni/probe/issues/1986). We discussed this issue with the obfs4 maintainer, which helped us understand the extent of the change. We concluded that for the OONI Probe CLI 3.14 release, we would not use the most updated version of the obfs4 library, in order to provide bridge operators some extra time to update obfs4. We aim to upgrade obfs4 in the next OONI Probe release. We helped our OTF Information Controls Research Fellow Kathrin Elmenhorst to implement a QUIC Ping experiment aimed at better characterizing cases of QUIC blocking (https://github.com/ooni/probe/issues/1994). The core idea of the experiment is to initiate a QUIC session using an invalid version number, to elicit a version negotiation message from a server. The message sent to initiate the session does not contain valid TLS data, therefore this experiment helps us to distinguish cases where QUIC is blocked from cases where the blocking depends on the TLS data contained in the initial QUIC datagram. *## OONI Explorer localization* Through collaboration with the Localization Lab, OONI Explorer copy was added to Transifex to enable translation of the platform by the global Localization Lab community. OONI Explorer can now be translated through the following Transifex project: https://www.transifex.com/otf/ooni-explorer/ Thanks to the Localization Lab community, OONI Explorer is being translated to 20 languages, and is already fully translated to Persian, Turkish, and French. *## Circumvention Tool Reachability Dashboard* Leading up to the launch (early March 2022), we worked on final improvements to our new Circumvention Tool Reachability Dashboard ( https://explorer.ooni.org/chart/circumvention). This included a series of UI improvements based on review and testing of the platform ( https://github.com/ooni/explorer/pull/653#pullrequestreview-877501889). *## OONI Measurement Aggregation Toolkit (MAT)* We continued to make improvements to our new Measurement Aggregation Toolkit (MAT) in preparation for the public launch (early April 2022). These improvements included improvements to the readability of x-axis charts, and layout changes in charts for longer date ranges (expanding beyond 2 months). We also made backend improvements to the aggregation API to improve how anomalous, confirmed, and failed measurements are grouped and counted in the API (https://github.com/ooni/backend/issues/486). *## OONI backend* We continued to work on migrating the OONI API to the new Clickhouse-based backend in production (https://github.com/ooni/pipeline/pull/371). This involved testing the new Clickhouse-based API ( https://github.com/ooni/backend/issues/552), re-importing the fastpath table with a domain column, comparing statistics generated by the private API, extensive monitoring, testing API endpoints and migrating them, and fixing the counts inside the aggregation API ( https://github.com/ooni/backend/issues/486). We also continued working on a number of other backend activities, such as improving the Prometheus and Grafana dashboards and alerting to monitor the Clickhouse database and its server. We cleaned up various spurious alerts and alerts that emerged due to limited memory on the ams-pg-test host. We ran an experiment predicting incoming traffic using sklearn's (machine learning library) linear regression. The experiment was successful. We set up log forwarding and ingestion using the monitoring server and created an example dashboard on Jupyter. We implemented a tool to run Jupyter notebooks automatically, which allows us to produce charts on an hourly, daily or weekly basis (https://github.com/ooni/backend/issues/564). We created a dashboard to monitor API check-ins. We also worked on adding support to the check-in API for returning geolocation information ( https://github.com/ooni/backend/issues/555). We started a thorough review of measurements that were not processed successfully in the past due to unsupported formats or parsing errors ( https://github.com/ooni/backend/issues/538). As part of this, we also created Jupyter dashboards and started a design document to plan the reprocessing of measurements. We also investigated a bug impacting RiseupVPN measurements ( https://github.com/ooni/pipeline/pull/379). In preparation for a security audit, we also continued to work on extensive threat modeling ( https://github.com/ooni/backend/issues/556). *## Improving OONI data analysis capabilities* We worked on data analysis and research related to extracting per website metrics (https://github.com/ooni/backend/issues/330). As part of this, we wrote an internal design document that outlines the features of various OONI measurements that are useful for performing data analysis tasks ( https://github.com/ooni/ooni.org/issues/1112). This document will support our efforts in extracting data from OONI measurements that is useful for characterizing different forms of internet censorship. We also worked towards consolidating our block page fingerprint collection efforts with the Citizen Lab filtering annotations ( https://github.com/ooni/backend/issues/516). This will enhance our ability to automatically detect and confirm more cases of website blocking around the world. *## Improving third party usage of OONI data* To enable third parties (such as researchers) to make use of OONI data, we worked on implementing a tool for third party download of OONI data. To avoid drifting out of sync, we determined in our design goals that this tool should use the same codebase as the OONI data processing pipeline. Initial work on this new tool was published as a branch of the OONI pipeline repository: https://github.com/ooni/pipeline/pull/377 We also created an internal design document, which outlines some of the current challenges in third party OONI data consumption and shared it for feedback with some key users of OONI data. *## Creating an online OONI Training Course for Advocacy Assembly* Small Media has provided us the opportunity to create an online OONI Training Course for their Advocacy Assembly project ( https://www.advocacyassembly.org/), which features a variety of training courses for human rights defenders. In February 2022, we started working on creating the online OONI training course. We started off by reviewing several past Advocacy Assembly training course scripts and videos in order to better understand how these courses are structured. Based on this, we finalized the structure for the OONI training course. Structurally, the online OONI training course will entail the following 7 chapters: * Introduction to the course * Introduction to internet censorship * Measuring internet censorship * Measuring internet censorship with OONI Probe * Understanding OONI censorship measurement data * OONI Explorer: Accessing real-time censorship measurement data * Conclusion Each chapter will dive into each topic in depth, providing participants with videos, screencasts, slides, quizzes, and hands-on exercises. Throughout February 2022, we created several hands-on exercises for the course, and we wrote the scripts for 8 direct-to-camera videos for the following (sub-)chapters: * Introduction to the course * What is internet censorship? * The problem of internet censorship * What is OONI Probe? * Interpreting OONI data * What is OONI Explorer? * Looking at OONI data in aggregate * Conclusion In April 2022, we will travel to London to record the above videos (based on the scripts we wrote) at a studio with Small Media. *## ISOC Pulse project on Internet shutdowns* OONI is a data partner for the Internet Society (ISOC) Pulse project on Internet shutdowns (https://pulse.internetsociety.org/partners). We aim to contribute OONI data (along with relevant charts and information) for all the “content blocking” events listed on the ISOC Pulse shutdowns timeline ( https://pulse.internetsociety.org/shutdowns). To this end, we started off by creating an inventory of all ISOC Pulse “content blocking” entries listed per year in their timeline (which span from 2018 to date). We then created content for the “content blocking” entries listed for 2022, including relevant OONI data, charts, and interpretation, which we shared with ISOC (to include in their timeline). We also started creating content (short reports) for the 2021 entries listed in the ISOC Pulse shutdown timeline. *## OTF Information Controls Research Fellows* In February 2022, OONI started serving as the host organization for OTF Information Controls Research Fellow, Ain Ghazal. Throughout their 1-year fellowship, Ain will study censorship resistance systems in global VPN infrastructure. Further information about their research fellowship is available here: https://www.opentech.fund/about/people/ain-ghazal/ As of February 2022, OONI is excited to serve as the host organization for a total of 3 OTF Information Controls Research Fellows. We are also hosting: * Kathrin Elmenhorst (investigating HTTP/3 censorship): https://www.opentech.fund/about/people/kathrin-elmenhorst/ * Gurshabad Grover (investigating the role of private ISPs in exacerbating or minimizing the effects of state-ordered censorship): https://www.opentech.fund/about/people/gurshabad-grover/ *## Google Summer of Code (GSoC) 2022 OONI project ideas* Over the past years, we have submitted Google Summer of Code (GSoC) project ideas for OONI via the Tor Project ( https://summerofcode.withgoogle.com/programs/2022/organizations/the-tor-pro…). For the summer of 2022, we submitted the following OONI project ideas for GSoC students: OONI Probe Network Experiments OONI Probe CLI Improvements OONI Explorer Improvements Further information is available here: https://gitlab.torproject.org/tpo/team/-/wikis/GSoC *## Collaboration with Netalitica* Netalitica researchers continued to do excellent work in reviewing and updating the Citizen Lab test lists. In February 2022, we reviewed their updates to the test lists for Hungary ( https://github.com/citizenlab/test-lists/pull/892) and Poland. *## Test list updates* In response to (potential) censorship events, we contributed updates to the following Citizen Lab test lists: * Global: https://github.com/citizenlab/test-lists/pull/889, https://github.com/citizenlab/test-lists/pull/890, https://github.com/citizenlab/test-lists/pull/915 * South Korea: https://github.com/citizenlab/test-lists/pull/891 * Turkey: https://github.com/citizenlab/test-lists/pull/911 We also reviewed and merged several test list pull requests contributed by community members. *## Emerging censorship events### New censorship events in Russia following the invasion of Ukraine* Following the invasion of Ukraine, Russia ramped up its censorship. Real-time OONI data collected from Russia shed light on these new censorship events as they emerged. Starting from 26th February 2022, Russia started to throttle access to Twitter and to block access to several independent news media websites. We started off by sharing OONI findings on social media ( https://twitter.com/OpenObservatory/status/1497951299593281538) and with mailing lists, while performing more extensive data analysis as new blocks emerged. Based on our analysis, we wrote a research report, which we published in early March 2022. *### Twitch blocked in Iran* Twitch was temporarily blocked in Iran on 27th and 28th February 2022. This was detected and reported by community members in Iran who shared relevant OONI data (https://twitter.com/xhdix/status/1497961275225214980/). OONI data involving the block can be accessed and viewed through the following MAT chart: https://explorer.ooni.org/experimental/mat?probe_cc=IR&test_name=web_connec… *## Notable community use of OONI Probe and OONI data### VEsinFiltro report on the blocking of news media websites in Venezuela* On 1st February 2022, our Venezuelan partner, VEsinFiltro, published a report documenting the blocking of news media websites in Venezuela: https://vesinfiltro.com/noticias/2022-02-01-bloqueo-Noticias/ Their report makes use of OONI data, as well as network measurement data collected from their own custom tools. *### VEsinFiltro report on the blocking of circumvention tools in Venezuela* On 7th February 2022, VEsinFiltro published a report documenting the blocking of circumvention tools in Venezuela: https://vesinfiltro.com/noticias/2022-02-07-restriction-circumvention-tools/ Their report makes use of OONI data, involving the analysis of measurements collected from the OONI Probe Tor and Web Connectivity experiments. *### ECOWAS court ruling on internet shutdown in Togo* Togo’s internet shutdown amid protests in September 2017 was challenged in the “Amnesty International Togo and Ors v. The Togolese Republic” case ( https://globalfreedomofexpression.columbia.edu/cases/amnesty-international-…) brought to the Community Court of Justice of the Economic Community of West African States (ECOWAS). In preparation for this case, the OONI team was asked to share relevant technical expertise, along with other experts. We analyzed the internet shutdown and shared a technical analysis in support of the court case ( https://twitter.com/FrancoisPatuel/status/1494030238388563968). As an outcome, the Community Court of Justice of ECOWAS held that the Togolese government violated the applicants’ right to freedom of expression by shutting down the internet during protests in September 2017, and ordered the Respondent State of Togo to take measures to guarantee the “non-occurrence” of a future similar situation. The ECOWAS court ruling on the September 2017 internet shutdown in Togo won the Columbia University Global Freedom of Expression 2022 Significant Legal Ruling Award ( https://globalfreedomofexpression.columbia.edu/prizewinners2022/). *## Community activities### OONI workshop in Tanzania by Zaina Foundation* On 28th February 2022, Zaina Foundation (our partner in Tanzania) facilitated an OONI workshop as part of their broader Digital Security Training for women journalists in Dar es Salaam ( https://twitter.com/ZainaFoundation/status/1498330087258742789). *### OONI FAQ translated to Khmer by DigitalHub101* Cambodian civil society group, DigitalHub101, published a version of the OONI FAQ (https://ooni.org/support/faq/) in Khmer: https://digitalhub101.com/ooni/ This new page features information in Khmer about OONI, OONI Probe, OONI Explorer, and OONI research reports. We linked to their translated version in the main OONI FAQ page to enable more community members to benefit from the translation. *### OONI Community Meeting* On 22nd February 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we discussed the following topics: 1) “Internet Protection Bill” in Iran 2) OONI measurement coverage from Zimbabwe *## Userbase* In February 2022, 36,927,939 OONI Probe measurements were collected from 2,556 AS networks in 158 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0

OONI Monthly Report: January 2022
by Maria Xynou 18 Apr '22

18 Apr '22

Hello, These days we're primarily supported by the DRL, which is why we do quarterly reporting. That said, we'd like to share monthly updates from the OONI team with the community, hence our January 2022 report shared below. I'll also follow-up to share our February and March 2022 reports. *# OONI Monthly Report: January 2022* Throughout January 2022, the OONI team worked on the following sprints: * Sprint 56 (3rd - 16th January 2022) * Sprint 57 (17th - 30th January 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## OONI Probe apps* In January 2022, Norbel Ambanumben joined the OONI team to lead the development of the OONI Probe mobile app. In preparation for the next release, various improvements were made to the OONI Probe mobile app, including: * Annotating measurements coming from VPNs ( https://github.com/ooni/probe/issues/1901); * Faster test startup time; * Improvements to the progress bar tracking; * Improvements to the minimisation of running tests ( https://github.com/ooni/probe/issues/1976, https://github.com/ooni/probe/issues/1981); * Adding links to the OONI blog and research reports in the OONI Probe mobile app (https://github.com/ooni/probe-android/pull/472); * CI fix (https://github.com/ooni/probe/issues/1924). On OONI Probe Desktop, we fixed a bug related to viewing RiseupVPN test results (https://github.com/ooni/probe-desktop/pull/272, https://github.com/ooni/probe/issues/1953). *## OONI Probe Web* We made progress on the backend components required for the operation of OONI Probe Web. As OONI Probe Web cannot use the same strategy as the OONI Probe apps to discover the country code and network of the probe (since we can’t ship a geoIP database in the client-side app), we had to add backend support for doing this operation. Specifically, we added support for returning geoIP information in the OONI check-in API (https://github.com/ooni/api/pull/276) and performed some quality testing of the db-ip geoIP database. We are considering eventually using the same mechanism for OONI Probe desktop and mobile probes too, as it would allow us to update the geoIP database without needing to make a new software release. *## OONI Run* We worked on improvements to the OONI Run platform. In particular, we fixed a bug affecting the generation of OONI Run links ( https://github.com/ooni/run/issues/100, https://github.com/ooni/run/pull/101), we fixed a bug affecting OONI Run links on iOS ( https://github.com/ooni/probe/issues/1978), and we made improvements to how OONI Run is deployed (https://github.com/ooni/run/pull/103). *## Expanding censorship measurement methodologies### Websteps* We worked on refactoring of the probe-engine codebase in preparation for Websteps development (our new experiment for measuring the blocking of websites). This included code improvements following the first Websteps prototype (https://github.com/ooni/probe/issues/1956). We also discovered and fixed a bug that was introduced when Websteps was merged (https://github.com/ooni/probe/issues/1808). This bug stems from a change in the logic used by the measurement engine to decide whether to submit measurements. After merging Websteps, we unconditionally stopped submitting measurements for experiments that returned an error. This behavior is correct in that the experiment should return errors only to indicate fundamental failures (e.g., an experiment expecting that a URL is input is passed a string that is not a valid URL) or missing preconditions. However, because previously the core measurement engine did not enforce this rule, several experiments were actually returning an error for network conditions that would otherwise be associated with interesting network measurements. We discovered this bug while performing extensive QA after merging Websteps. We then needed to ensure that all OONI experiments were correctly returning errors only to signal fundamental failures or missing pre-conditions. As a result, we fixed DASH, ndt7, Psiphon, STUN Reachability and other minor experiments. It is worth noting that no stable OONI release ever shipped with this bug, and we indeed catched the bug because we knew merging Websteps entailed larger changes to the tree, so we chose to perform extensive QA to ensure we were not introducing regressions. *### OONI Probe CLI v3.14.0 pre-release* We released OONI Probe CLI v3.14.0-alpha.1 with support for running our new Tor Snowflake experiment (https://ooni.org/nettest/tor-snowflake/), a refactored Tor experiment, and a series of data quality improvements. Details are available through our pre-release notes: https://github.com/ooni/probe-cli/releases/tag/v3.14.0-alpha.1 Notably, this pre-release includes: * Support for running the new Tor Snowflake experiment on OONI Probe Mobile and Desktop (https://github.com/ooni/probe/issues/1917); * Refactored Tor experiment to use the newer underlying measurement library, which allowed us to cleanup legacy code and reduce the overall complexity of the network measurement engine ( https://github.com/ooni/probe/issues/1688, https://github.com/ooni/probe/issues/1970, https://github.com/ooni/probe/issues/1545 ). *### Other improvements* We also worked on a series of other improvements. Specifically, we: * Started designing a new experiment that pings DNS servers over UDP ( https://github.com/ooni/probe/issues/1987, https://github.com/ooni/probe-cli/pull/674); * Researched application-level transparent proxies to understand whether we could use them to perform more realistic QA checks before shipping releases; * Discovered and discussed a potential data quality issue around how we determine DNS blocking in Web Connectivity, where DNS blocking and HTTP blocking conflict with each other, and HTTP blocking currently takes precedence, chose not to address the bug to keep the overall probe behavior consistent over time, and planned to ensure we don’t have the same issue in websteps (https://github.com/ooni/probe/issues/1975); * Fixed a bug potentially affecting the quality of measurements where we were not correctly detecting the end of a stream due to differences in how we represent errors and how the Go standard library expected errors to be ( https://github.com/ooni/probe/issues/1965); * Refactoring and improvements the code we use to communicate with the OONI backend trying to make the Go API more robust ( https://github.com/ooni/probe/issues/1951); * Refactored the CLI and engine codebases to increase abstraction and testability (https://github.com/ooni/probe/issues/1885); * Refactored code we use to collect low-level measurements to reduce measurement code duplication and improve consistency ( https://github.com/ooni/probe/issues/1957). *## Blog post on measuring HTTP/3 censorship with OONI Probe* In January 2022, we started hosting Kathrin Elmenhorst for her 3-month OTF Information Controls Research Fellowship. We had previously worked with Kathrin on adding HTTP/3 support to OONI Probe for measuring HTTP/3 censorship in China, Iran, India, and Kazakhstan. Based on this research, we collaborated on a research paper (“Web censorship measurements of HTTP/3 over QUIC”) which was submitted to AMC Internet Measurement Conference ( https://dl.acm.org/doi/abs/10.1145/3487552.3487836). To share the findings of this research with our broader community, Kathrin wrote a blog post which summarizes the measurement findings. We published this blog post here: https://ooni.org/post/2022-http3-measurements-paper/ *## OONI Explorer* OONI Explorer performance was significantly enhanced in January 2022. Migrating to the new Clickhouse-based backend has helped ensure that OONI Explorer queries work faster and more reliably. Now database-intense queries on OONI Explorer load within seconds, enabling users to more effectively track censorship events around the world based on real-time OONI data. We made a minor update to the OONI Data Policy to mention that OONI Explorer uses Sentry to log crash reports ( https://ooni.org/about/data-policy). *## New Circumvention Tool Reachability Dashboard* Throughout January 2022, we worked on creating a new Dashboard which presents aggregate views of real-time OONI data based on the reachability testing of circumvention tools ( https://explorer.ooni.org/chart/circumvention). In particular, this dashboard presents charts based on the reachability testing of Psiphon, Tor, and Tor Snowflake in every country around the world (https://github.com/ooni/ooni.org/issues/774). Through these charts, users can gain a bird's-eye view on the testing of these tools in each country over time, and plot charts based on the countries and date range of their choice (https://github.com/ooni/explorer/pull/653). As these charts present aggregates of anomalous measurements, it is possible to infer potential cases of blocking (or cases when the tested tools were unreachable on tested networks due to other reasons). *## OONI backend* Throughout January 2022, we worked on migrating the OONI API to the new Clickhouse-based backend in production ( https://github.com/ooni/pipeline/pull/371). This resulted in a significant improvement to OONI Explorer performance (which relies on the OONI API), as OONI Explorer queries now work much faster and more reliably. We expanded OONI’s data analysis capabilities to support processing STUN reachability experiments (https://github.com/ooni/ooni.org/issues/767). The measurements of these tests are very closely related to Tor Snowflake, since the PT relied on STUN for NAT hole punching. We also did further testing and performance improvements to the data analysis code in preparation for the production launch. Moreover, progress on the OONI API was made to have stable aggregate views for displaying pluggable transport test results ( https://github.com/ooni/ooni.org/issues/769). Specifically, we implemented new API endpoints to return: * The runtime of PT test results; * The number of PT metrics grouped by country (this allows us to filter in the UI the country list to only those for which we have data). We also: * Added backend support for scoring measurements collected from the HTTP version of WhatsApp Web (https://github.com/ooni/backend/issues/546); * Improved the speed of the ooniprobe list command by refactoring the database queries (https://github.com/ooni/probe-cli/pull/662); * Looked into migrating the ooni.org website to another provider ( https://github.com/ooni/ooni.org/issues/1064) to improve its accessibility around the world; * Carried out some research and experimentation related to enabling third-party analysis of OONI data ( https://github.com/ooni/backend/issues/514#issuecomment-1016525686); * Wrote a document that outlines the features of various OONI measurements that are useful for performing data analysis tasks (in order to extract per-website metrics). *## Improving third party usage of OONI data* To enable third-party use of OONI data (particularly by researchers), we carried out some research and collected feedback on how to improve the user experience for third party consumers of OONI data. As part of this, we decided to offer a Command Line Interface (CLI) tool (called oonidata) that researchers can run to download raw data dumps of OONI data. A summary of this research can be found in this comment: https://github.com/ooni/backend/issues/514#issuecomment-1016525686 *## Preparing for a security audit* In preparation for a security audit of OONI software projects, we worked on extensive threat modeling. This involved identifying and mapping all potential threats that could affect various OONI software components, determining the probability, impact, and risk of each potential threat, as well as strategies for mitigation. The documentation that we prepared will support the upcoming security audits of OONI tools. *## Collaboration with Netalitica* Netalitica researchers continued to do excellent work in reviewing and updating the Citizen Lab test lists. In January 2022, we reviewed their updates to the test lists for Singapore, Myanmar, and Vietnam ( https://github.com/citizenlab/test-lists/pull/885), as well as their updates to the test lists for Cuba, Cameroon, and Bolivia ( https://github.com/citizenlab/test-lists/pull/888). We also opened a pull request based on Netalitica updates to the test lists for Nicaragua and Ecuador: https://github.com/citizenlab/test-lists/pull/881 *## Test list updates* In January 2022, we contributed minor updates to several Citizen Lab test lists, as documented through the following pull requests: https://github.com/citizenlab/test-lists/pull/882, https://github.com/citizenlab/test-lists/pull/883, https://github.com/citizenlab/test-lists/pull/884 We reviewed and merged more than 50 pull requests from the Citizen Lab test list repository, some of which included new contributions via our new web platform (https://test-lists.ooni.org/). *## Created an audio recording for a podcast on technical terms* We created a 10-minute audio recording on “What is the Internet?” for the ABC Digital podcast on “A Glossary of Technological Terms”. To create this recording, we wrote a script describing what is the internet, which we then recorded and shared with Conflux Center. This recording was subsequently shared with training participants of the “Technologies and Digital Tools for Peacekeeping and Political Missions: Data, Research, Analysis, & Communication” course organized by Conflux Center ( https://www.confluxcenter.org/wp-content/uploads/2022/01/Virtual-Course-Cat… ). *## Censored Planet data user study* Three OONI team members participated in Censored Planet’s data user study. As part of this user study, we reviewed the new Censored Planet Dashboard and shared feedback through a survey and through interviews with the Censored Planet team. *## Community activities### OONI training for UN peacekeeping missions* On 25th January 2022, OONI’s Maria facilitated an OONI training (“OONI - Practical exercise measuring Internet censorship”) for UN peacekeeping missions as part of a course (“Technologies and Digital Tools for Peacekeeping and Political Missions: Data, Research, Analysis, & Communication”) organized by Conflux Center. As part of the training, Maria introduced participants to OONI censorship measurement tools, provided a live demo of using OONI Probe for measuring internet censorship, and facilitated hands-on exercises for using OONI Probe based on real-world scenarios. Information about the virtual training course is available here: https://www.confluxcenter.org/wp-content/uploads/2022/01/Virtual-Course-Cat… *### OONI Probe workshop for partners in Southeast Asia* On 28th January 2022, OONI’s Maria facilitated an OONI Probe workshop for Sinar Project partners in Southeast Asia participating in the iMap Regional Workshop. As part of this workshop, Maria provided a theoretical overview of OONI Probe and OONI Run, a live demo of using OONI Probe for measuring internet censorship, a live demo for using OONI Run for coordinating censorship testing, and facilitated several hands-on exercises based on real-world scenarios. *### OONI Community Meeting* On 31st January 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we discussed RAACT (Resistance Against Automated Content Takedown): a new project led by Tracking Exposed (https://tracking.exposed/) to measure content takedowns on social media platforms (Facebook, Instagram, TikTok, Youtube). As part of the meeting, we discussed how the OONI community could get involved with the RAACT project. *## Userbase* In January 2022, 39,215,688 OONI Probe measurements were collected from 2,634 AS networks in 171 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0

OONI Community Meeting: Tuesday, 19th April 2022
by Maria Xynou 18 Apr '22

18 Apr '22

Hello, The OONI team warmly welcomes you to join us next Tuesday for our monthly community meeting. *=> Where? *OONI Slack channel: https://slack.ooni.org/ (bridged with IRC: ircs://irc.oftc.net:6697/#ooni) *=> When?* Tuesday, *19th April 2022 at 14:00 UTC* (for 1 hour) Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep The monthly OONI community meetings aim to: * Collect community feedback on OONI tools & methodologies * Address questions in relation to the use of OONI tools and OONI data * Foster discussions on internet censorship issues * Receive updates from the community We hope you can join us! All the best, ~ OONI team

1 1

PT community meetup April [28/04/2022]
by Vasilis 15 Apr '22

15 Apr '22

Dear all, This month Pluggable Transports meetup will take place on Thursday, April 28. Time: 10:00 AM EDT / 2:00 PM UTC / 16:00 PM CEST Where: Pluggable Transports channel * [Matrix](https://matrix.to/#/#pluggable-transports:matrix.org) * [IRC (no account required)](https://webchat.oftc.net/?channels=pluggable-transports) * [Slack](https://openobservatory.slack.com/messages/pluggable-transports/) * [Mattermost](https://community.internetfreedomfestival.org/community/channel… More info and details on how to join: https://pluggabletransports.info/blog/pt-meetup-apr2022/ Hope to see many of you there. Best, ~Vasilis -- PGP Fingerprint: 8FD5 CF5F 39FC 03EB B382 7470 5FBF 70B1 D126 0162 PGP Public Key: https://keys.openpgp.org/vks/v1/by-fingerprint/8FD5CF5F39FC03EBB38274705FBF…

1 0

New Launch: OONI Measurement Aggregation Toolkit (MAT)
by Maria Xynou 12 Apr '22

12 Apr '22

Hello, Today the OONI team is thrilled to announce the public launch of our *new Measurement Aggregation Toolkit (MAT)*! Access the MAT here: https://explorer.ooni.org/chart/mat Learn all about the MAT through our blog post: https://ooni.org/post/2022-ooni-mat/ *# What is the MAT?* Imagine being able to track internet censorship based on real-time data collected from networks around the world. And imagine being able to create your own charts (based on this data) with the click of a button. That's the MAT! :) The MAT is a tool that enables you to *create your own custom charts based on aggregate views of real-time OONI data* collected from around the world. Use the MAT to* track internet censorship worldwide* based on OONI data! *# Who is the MAT for? * The MAT was built for researchers, journalists, and human rights defenders interested in examining internet censorship around the world. *# Why use the MAT?* When examining cases of internet censorship, it's important to *look at many measurements at once* ("in aggregate") in order to identify trends and patterns, and answer key questions like the following: * Does the testing of a service (e.g. Facebook) present signs of blocking every time that it is tested in a country? This can be helpful for ruling out false positives. * What types of websites (e.g. human rights websites) are blocked in each country? * In which countries is a specific website (e.g. bbc.com) blocked? * How does the blocking of different apps (e.g. WhatsApp or Telegram) vary across countries? * How does the blocking of a service vary across ASNs? * How does the blocking of a service change over time? When trying to answer questions like the above, we normally perform relevant data analysis (instead of inspecting measurements one by one). The MAT incorporates our data analysis techniques, enabling you to answer such questions without any data analysis skills, and with the click of a button! Further details on using the MAT and interpreting MAT charts are available through our blog post: https://ooni.org/post/2022-ooni-mat/ We hope the internet freedom community finds the MAT useful for investigating and responding to internet censorship events around the world. Please help spread the word: https://twitter.com/OpenObservatory/status/1513818001153437700 If you have any questions, feedback, or feature requests, we’d love to hear from you! You can open a ticket (https://github.com/ooni/explorer/issues/) or write us an email (contact(a)openobservatory.org). We thank all OONI Probe users worldwide who have contributed (and continue to contribute) measurements, supporting the MAT. We also thank community members for their invaluable feedback. <3 Thanks, OONI team.

1 0

[Event] Obfuscated VPN: Lessons Learned and Future Work
by Vasilis 06 Apr '22

06 Apr '22

Dear all, Internews will be hosting Pop Up PTIM Sessions! This will consist of stand alone sessions, allowing the community to discuss PT related topics and stay up to date on the progress of this community. The first session will be presented by the LEAP Encryption Access Project. They will share the work they have been doing for the past few years on the usage of PTs in the context of a VPN application. This session will take place on April 12th at 14:00 UTC/9:00 ET/16:00 CET. Please use the link below to register for the event. Session Title: Sharing LEAP’s Experience on the usage of PTs in the context of a VPN application. Registration: https://cryptpad.fr/form/#/2/form/view/d5B9a9ZJxBTBXYtbqjbmIeiIyYZ-fC4zDTut… Code of Conduct: https://www.pluggabletransports.info/code-of-conduct/ Announcement post: https://www.pluggabletransports.info/blog/popup-ptim-leap-announcement/ Please feel free to share this event announcement. Hope to see you there! --Vasilis -- PGP Fingerprint: 8FD5 CF5F 39FC 03EB B382 7470 5FBF 70B1 D126 0162 PGP Public Key: https://keys.openpgp.org/vks/v1/by-fingerprint/8FD5CF5F39FC03EBB38274705FBF…

1 0

OONI Community Meeting: Tuesday, 29th March 2022
by Maria Xynou 28 Mar '22

28 Mar '22

Hello, The OONI team warmly welcomes you to join us next Tuesday for our monthly community meeting. *=> Where? *OONI Slack channel: https://slack.ooni.org/ (bridged with IRC: ircs://irc.oftc.net:6697/#ooni) *=> When?* Tuesday, *29th March 2022 at 14:00 UTC* (for 1 hour) Please add topics that you'd like to discuss during the meeting in this pad: https://pad.riseup.net/p/ooni-community-meeting-keep The monthly OONI community meetings aim to: * Collect community feedback on OONI tools & methodologies * Address questions in relation to the use of OONI tools and OONI data * Foster discussions on internet censorship issues * Receive updates from the community We hope you can join us! All the best, ~ OONI team

1 1

PT community meetup March [17/03/2022]
by Vasilis 14 Mar '22

14 Mar '22

Dear all, This month Pluggable Transports meetup will take place on Thursday, March 17. Time: 9:00 AM EST / 2:00 PM UTC Where: Pluggable Transports channel * [Matrix](https://matrix.to/#/#pluggable-transports:matrix.org) * [IRC (no account required)](https://webchat.oftc.net/?channels=pluggable-transports) * [Slack](https://openobservatory.slack.com/messages/pluggable-transports/) * [Mattermost](https://community.internetfreedomfestival.org/community/channel… More info and details on how to join: https://pluggabletransports.info/blog/pt-meetup-mar2022/ Hope to see many of you there. Best, ~Vasilis -- PGP Fingerprint: 8FD5 CF5F 39FC 03EB B382 7470 5FBF 70B1 D126 0162 PGP Public Key: https://keys.openpgp.org/vks/v1/by-fingerprint/8FD5CF5F39FC03EBB38274705FBF…

1 0

New release: OONI Probe Desktop 3.7.0
by Maria Xynou 08 Mar '22

08 Mar '22

Hello friends, I'm reaching out to share that we have released *OONI Probe Desktop 3.7.0. * Please update to the latest version: https://ooni.org/install/desktop OONI Probe Desktop includes our *new Tor Snowflake test* ( https://ooni.org/nettest/tor-snowflake/), along with many other bug fixes and improvements. Please note that we have *temporarily disabled the RiseupVPN test.* We aim to re-enable it once it's improved. Apologies for the inconvenience. Thanks, Maria (on behalf of the OONI team)

1 0

[Report] New blocks emerge in Russia amid war in Ukraine: An OONI network measurement analysis
by Maria Xynou 07 Mar '22

07 Mar '22

Hello, I hope you are all safe and well. My deepest solidarity to all those affected by the war in Ukraine. Today OONI published a *research report on new censorship events in Russia* amid the war in Ukraine, based on OONI data analysis. You can read the report here: https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/ *# Key Findings* *## Russia* ** Media censorship.* Russia recently started blocking access to independent Russian news media websites (such as Dozhd and New Times) and foreign news media websites (such as BBC, Deutsche Welle, Russian version of Voice of America, and RFE/RL service websites). ** Blocking of a website about captured and killed Russian soldiers.* Russia blocked access to 200rf.com, which was created by representatives of the Ministry of Internal Affairs of Ukraine to share information about captured and killed Russian soldiers in Ukraine. ** Centralized throttling of Twitter.* OONI data suggests that access to twitter.com was throttled in Russia between 26th February 2022 to 4th March 2022. As the throttling of twitter.com seems to have stopped across all ISPs in Russia at the same time (~08:00 on 4th March 2022 UTC), it appears to have been centralized. Moreover, interference to twitter.com appears to have changed from throttling to blocking (through the injection of a RST packet) on 4th March 2022. ** Blocking of Facebook and Twitter.* OONI data shows that access to facebook.com and twitter.com was blocked in Russia by 4th March 2022 (primarily through the injection of RST packets). ** Decentralized censorship.* Every Internet Service Provider (ISP) in Russia is responsible for implementing government-mandated blocks independently. As a result, we observe variance in how internet censorship is implemented across Russia, as blocks are not implemented on all networks in the country, and different ISPs adopt different censorship techniques. Some ISPs implement blocks through the use of multiple techniques at the same time, making circumvention harder. ** Different censorship techniques.* To block websites, OONI data shows that Russian ISPs adopt the following censorship techniques (beyond throttling): * DNS manipulation, redirecting in some cases to blockpages * HTTP man-in-the-middle, serving blockpages * TLS man-in-the-middle * Injection of a RST packet after the ClientHello during the TLS handshake (most prevalent) * Timing out the session after the ClientHello during the TLS handshake * Closing the connection after the ClientHello during the TLS handshake *## EU* Following the recent decision of the Council of the European Union to suspend the broadcasting activities of Sputnik and Russia Today (RT) in the EU, some EU countries have started to implement blocks. OONI data shows that EU countries like Greece, France, Romania, Spain, Poland, and Germany have started blocking rt.com. Moreover, we observe the blocking of sputniknews.com in several European countries, such as France, Germany, Spain, and the Czech Republic. Following requests by the European Commission, Google has removed results pertaining to rt.com and sputniknews.com from Google Search for users in the EU. Similarly, Facebook and Twitter are geo-blocking EU users from accessing RTNews. Learn more though our report: https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/ ~ OONI team.

1 0

← Newer
1
...
15
16
17
18
19
20
21
...
46
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

ooni-talk