- tor-project - lists.torproject.org

OONI Monthly Report: February 2022
by Maria Xynou 18 Apr '22

18 Apr '22

Hello, OONI's February 2022 status report is shared below. *# OONI Monthly Report: February 2022* Throughout February 2022, the OONI team worked on the following sprints: * Sprint 58 (1st - 13th February 2022) * Sprint 59 (14th - 27th February 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## OONI submission for the OHCHR report on internet shutdowns* On 10th February 2022, we sent a submission in response to the UN High Commissioner for Human Rights’s call for submissions in support of OHCHR report on internet shutdowns and human rights to the fiftieth session of the Human Rights Council in June 2022 ( https://www.ohchr.org/en/Issues/CivicSpace/Pages/cfi-report-hrc-50.aspx) In particular, the OONI submission provides information on the occurrence of mandated disruptions of access to social media and messaging platforms over the past 5 years based on empirical OONI network measurement data. Through our submission, we share information on social media blocks that occurred during elections in Uganda, Tanzania, Mali, Benin, Togo, Burundi, and Zambia over the last 5 years (based on OONI data). We also share information on social media blocks that occurred during protests in Pakistan, Jordan, Iran, Zimbabwe, and Cuba. Moreover, we share OONI data and information on social media blocks that emerged during sensitive political time periods in Ethiopia, Myanmar, and Venezuela. In our submission, we also recommend that the OHCHR considers expanding this scope to also report on cases that involve the blocking of websites of marginalized communities. *## OONI Probe Mobile* Throughout February 2022, we worked towards the OONI Probe Android 3.6.0 release. In particular, we worked on the following OONI Probe Android improvements: * Worked on ensuring that the VPN modal is displayed prior to the running of tests (https://github.com/ooni/probe/issues/1982) * Improved the UI for minimizing a running test ( https://github.com/ooni/probe/issues/1998) * Fixed a bug affecting the display of the indication that a proxy is being used (https://github.com/ooni/probe/issues/1919) * Fixed a bug affecting Web Connectivity tests ( https://github.com/ooni/probe/issues/1328) * Improved the performance of the test startup time ( https://github.com/ooni/probe/issues/2012) * Improved the progress bar of tests ( https://github.com/ooni/probe/issues/2021, https://github.com/ooni/probe/issues/1936) * Fixed a bug affecting the OONI backend ( https://github.com/ooni/probe/issues/1922) * Disabled the RiseupVPN test from OONI Probe ( https://github.com/ooni/probe/issues/2001) We also worked on the following OONI Probe iOS improvements: * Added links to the OONI blog and research reports in OONI Probe iOS ( https://github.com/ooni/probe/issues/1850) * Fixed a bug that caused the OONI Probe iOS app to crash with a malformed URL (https://github.com/ooni/probe/issues/1978) In preparation for the upcoming release of our new Tor Snowflake experiment, we wrote a test description, which we published on our website: https://ooni.org/nettest/tor-snowflake/ *## OONI Run* We continued to make incremental improvements to OONI Run based on community feedback. In particular, we fixed a bug affecting the copy-pasting of URLs into the OONI Run platform ( https://github.com/ooni/run/issues/82) *## OONI Probe Desktop* Throughout February 2022, we worked towards the OONI Probe Desktop 3.7.0 release. In late February 2022, we released the release candidate for OONI Probe Desktop 3.7.0: https://github.com/ooni/probe-desktop/releases/tag/v3.7.0-rc.1 In particular, we worked on the following OONI Probe Desktop improvements: * Fixed a bug affecting website testing ( https://github.com/ooni/probe/issues/1848) * Fixed a bug affecting the “test duration limit” (following recent OONI Probe CLI changes); * Fixed a bug affecting the RiseupVPN test result screen ( https://github.com/ooni/probe-desktop/pull/272) * Ensured that OONI Probe Desktop defaults to English on unsupported languages (https://github.com/ooni/probe/issues/1914) * Disabled the RiseupVPN test from OONI Probe Desktop ( https://github.com/ooni/probe/issues/2002) *## OONI Probe Command Line Interface (CLI)* We released OONI Probe CLI 3.14.1 along with new mobile libraries for Android and iOS. You can read the changelog here: https://github.com/ooni/probe-cli/releases/tag/v3.14.1 Some highlights: * We added the DNS Check test to the experimental test suite; * We added the Tor Snowflake test to the experimental test suite; * We made `ooniprobe list` approximately 7 times faster than before; * We fixed our mobile libraries to ensure that all measurements are submitted. Previously, there were cases when the last measurement in a Web Connectivity run would not be submitted when the experiment timeout occurred right before an attempt to submit this measurement (see: https://github.com/ooni/probe/issues/2037) * We fixed a bug which prevented the NDT test from working on Android 6 phones due to the bundling of a very old certification authority ( https://github.com/ooni/probe/issues/2031) * We added a workaround to handle an inconsistency in how Android 10 NXDOMAIN errors are reported which would otherwise cause measurements to be marked as failed. This was done by tweaking our error classifier for this error condition (https://github.com/ooni/probe/issues/2029) and we started working on a more comprehensive fix for v3.15. *## Expanding censorship measurement methodologies* In preparation for the OONI Probe CLI 3.14 release, we performed extensive QA of the new Tor Snowflake experiment ( https://github.com/ooni/probe/issues/1917 and https://github.com/ooni/probe/issues/2004) where we discussed the proper experiment configuration with Tor Snowflake developers. We eventually settled for a configuration where we allow tor to cache on disk its descriptors to increase the chances that tor could successfully bootstrap using Snowflake. We determined that bootstrapping issues are probably caused by the Snowflake bridge being at capacity, which is a problem being addressed upstream. We started researching how we could use data we already collect to detect cases of heavy throttling. We developed a prototype experiment, called “download”, to explicitly investigate these conditions ( https://github.com/ooni/probe/issues/2025) We applied some of the preliminary knowledge we gathered from this new experiment to detect cases of Twitter throttling using data already collected by Web Connectivity in our report on censorship in Russia amid the conflict in Ukraine ( https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/#twitter-throt… ). We discovered and documented issues with the RiseupVPN experiment ( https://github.com/ooni/probe/issues/1999) and decided to temporarily disable the experiment from the OONI Probe apps ( https://github.com/ooni/probe/issues/2000) until those fundamental issues have been resolved. We also discovered an issue with the tor experiment, where security changes in the upstream obfs4 library caused the NTOR handshake to fail with some of the bridges we are currently testing ( https://github.com/ooni/probe/issues/1970 and https://github.com/ooni/probe/issues/1986) We discussed this issue with the obfs4 maintainer, which helped us understand the extent of the change. We concluded that for the OONI Probe CLI 3.14 release, we would not use the most updated version of the obfs4 library, in order to provide bridge operators some extra time to update obfs4. We aim to upgrade obfs4 in the next OONI Probe release. We helped our OTF Information Controls Research Fellow Kathrin Elmenhorst to implement a QUIC Ping experiment aimed at better characterizing cases of QUIC blocking (https://github.com/ooni/probe/issues/1994) The core idea of the experiment is to initiate a QUIC session using an invalid version number, to elicit a version negotiation message from a server. The message sent to initiate the session does not contain valid TLS data, therefore this experiment helps us to distinguish cases where QUIC is blocked from cases where the blocking depends on the TLS data contained in the initial QUIC datagram. *## OONI Explorer localization* Through collaboration with the Localization Lab, OONI Explorer copy was added to Transifex to enable translation of the platform by the global Localization Lab community. OONI Explorer can now be translated through the following Transifex project: https://www.transifex.com/otf/ooni-explorer/ Thanks to the Localization Lab community, OONI Explorer is being translated to 20 languages, and is already fully translated to Persian, Turkish, and French. *## Circumvention Tool Reachability Dashboard* Leading up to the launch (early March 2022), we worked on final improvements to our new Circumvention Tool Reachability Dashboard ( https://explorer.ooni.org/chart/circumvention) This included a series of UI improvements based on review and testing of the platform ( https://github.com/ooni/explorer/pull/653#pullrequestreview-877501889) *## OONI Measurement Aggregation Toolkit (MAT)* We continued to make improvements to our new Measurement Aggregation Toolkit (MAT) in preparation for the public launch (early April 2022). These improvements included improvements to the readability of x-axis charts, and layout changes in charts for longer date ranges (expanding beyond 2 months). We also made backend improvements to the aggregation API to improve how anomalous, confirmed, and failed measurements are grouped and counted in the API (https://github.com/ooni/backend/issues/486) *## OONI backend* We continued to work on migrating the OONI API to the new Clickhouse-based backend in production (https://github.com/ooni/pipeline/pull/371) This involved testing the new Clickhouse-based API ( https://github.com/ooni/backend/issues/552) re-importing the fastpath table with a domain column, comparing statistics generated by the private API, extensive monitoring, testing API endpoints and migrating them, and fixing the counts inside the aggregation API ( https://github.com/ooni/backend/issues/486) We also continued working on a number of other backend activities, such as improving the Prometheus and Grafana dashboards and alerting to monitor the Clickhouse database and its server. We cleaned up various spurious alerts and alerts that emerged due to limited memory on the ams-pg-test host. We ran an experiment predicting incoming traffic using sklearn's (machine learning library) linear regression. The experiment was successful. We set up log forwarding and ingestion using the monitoring server and created an example dashboard on Jupyter. We implemented a tool to run Jupyter notebooks automatically, which allows us to produce charts on an hourly, daily or weekly basis (https://github.com/ooni/backend/issues/564) We created a dashboard to monitor API check-ins. We also worked on adding support to the check-in API for returning geolocation information ( https://github.com/ooni/backend/issues/555) We started a thorough review of measurements that were not processed successfully in the past due to unsupported formats or parsing errors ( https://github.com/ooni/backend/issues/538) As part of this, we also created Jupyter dashboards and started a design document to plan the reprocessing of measurements. We also investigated a bug impacting RiseupVPN measurements ( https://github.com/ooni/pipeline/pull/379) In preparation for a security audit, we also continued to work on extensive threat modeling ( https://github.com/ooni/backend/issues/556) *## Improving OONI data analysis capabilities* We worked on data analysis and research related to extracting per website metrics (https://github.com/ooni/backend/issues/330) As part of this, we wrote an internal design document that outlines the features of various OONI measurements that are useful for performing data analysis tasks ( https://github.com/ooni/ooni.org/issues/1112) This document will support our efforts in extracting data from OONI measurements that is useful for characterizing different forms of internet censorship. We also worked towards consolidating our block page fingerprint collection efforts with the Citizen Lab filtering annotations ( https://github.com/ooni/backend/issues/516) This will enhance our ability to automatically detect and confirm more cases of website blocking around the world. *## Improving third party usage of OONI data* To enable third parties (such as researchers) to make use of OONI data, we worked on implementing a tool for third party download of OONI data. To avoid drifting out of sync, we determined in our design goals that this tool should use the same codebase as the OONI data processing pipeline. Initial work on this new tool was published as a branch of the OONI pipeline repository: https://github.com/ooni/pipeline/pull/377 We also created an internal design document, which outlines some of the current challenges in third party OONI data consumption and shared it for feedback with some key users of OONI data. *## Creating an online OONI Training Course for Advocacy Assembly* Small Media has provided us the opportunity to create an online OONI Training Course for their Advocacy Assembly project ( https://www.advocacyassembly.org/) which features a variety of training courses for human rights defenders. In February 2022, we started working on creating the online OONI training course. We started off by reviewing several past Advocacy Assembly training course scripts and videos in order to better understand how these courses are structured. Based on this, we finalized the structure for the OONI training course. Structurally, the online OONI training course will entail the following 7 chapters: * Introduction to the course * Introduction to internet censorship * Measuring internet censorship * Measuring internet censorship with OONI Probe * Understanding OONI censorship measurement data * OONI Explorer: Accessing real-time censorship measurement data * Conclusion Each chapter will dive into each topic in depth, providing participants with videos, screencasts, slides, quizzes, and hands-on exercises. Throughout February 2022, we created several hands-on exercises for the course, and we wrote the scripts for 8 direct-to-camera videos for the following (sub-)chapters: * Introduction to the course * What is internet censorship? * The problem of internet censorship * What is OONI Probe? * Interpreting OONI data * What is OONI Explorer? * Looking at OONI data in aggregate * Conclusion In April 2022, we will travel to London to record the above videos (based on the scripts we wrote) at a studio with Small Media. *## ISOC Pulse project on Internet shutdowns* OONI is a data partner for the Internet Society (ISOC) Pulse project on Internet shutdowns (https://pulse.internetsociety.org/partners) We aim to contribute OONI data (along with relevant charts and information) for all the “content blocking” events listed on the ISOC Pulse shutdowns timeline ( https://pulse.internetsociety.org/shutdowns) To this end, we started off by creating an inventory of all ISOC Pulse “content blocking” entries listed per year in their timeline (which span from 2018 to date). We then created content for the “content blocking” entries listed for 2022, including relevant OONI data, charts, and interpretation, which we shared with ISOC (to include in their timeline). We also started creating content (short reports) for the 2021 entries listed in the ISOC Pulse shutdown timeline. *## OTF Information Controls Research Fellows* In February 2022, OONI started serving as the host organization for OTF Information Controls Research Fellow, Ain Ghazal. Throughout their 1-year fellowship, Ain will study censorship resistance systems in global VPN infrastructure. Further information about their research fellowship is available here: https://www.opentech.fund/about/people/ain-ghazal/ As of February 2022, OONI is excited to serve as the host organization for a total of 3 OTF Information Controls Research Fellows. We are also hosting: * Kathrin Elmenhorst (investigating HTTP/3 censorship): https://www.opentech.fund/about/people/kathrin-elmenhorst/ * Gurshabad Grover (investigating the role of private ISPs in exacerbating or minimizing the effects of state-ordered censorship): https://www.opentech.fund/about/people/gurshabad-grover/ *## Google Summer of Code (GSoC) 2022 OONI project ideas* Over the past years, we have submitted Google Summer of Code (GSoC) project ideas for OONI via the Tor Project ( https://summerofcode.withgoogle.com/programs/2022/organizations/the-tor-pro… ). For the summer of 2022, we submitted the following OONI project ideas for GSoC students: OONI Probe Network Experiments OONI Probe CLI Improvements OONI Explorer Improvements Further information is available here: https://gitlab.torproject.org/tpo/team/-/wikis/GSoC *## Collaboration with Netalitica* Netalitica researchers continued to do excellent work in reviewing and updating the Citizen Lab test lists. In February 2022, we reviewed their updates to the test lists for Hungary ( https://github.com/citizenlab/test-lists/pull/892) and Poland. *## Test list updates* In response to (potential) censorship events, we contributed updates to the following Citizen Lab test lists: * Global: https://github.com/citizenlab/test-lists/pull/889, https://github.com/citizenlab/test-lists/pull/890, https://github.com/citizenlab/test-lists/pull/915 * South Korea: https://github.com/citizenlab/test-lists/pull/891 * Turkey: https://github.com/citizenlab/test-lists/pull/911 We also reviewed and merged several test list pull requests contributed by community members. *## Emerging censorship events### New censorship events in Russia following the invasion of Ukraine* Following the invasion of Ukraine, Russia ramped up its censorship. Real-time OONI data collected from Russia shed light on these new censorship events as they emerged. Starting from 26th February 2022, Russia started to throttle access to Twitter and to block access to several independent news media websites. We started off by sharing OONI findings on social media ( https://twitter.com/OpenObservatory/status/1497951299593281538) and with mailing lists, while performing more extensive data analysis as new blocks emerged. Based on our analysis, we wrote a research report, which we published in early March 2022. *### Twitch blocked in Iran* Twitch was temporarily blocked in Iran on 27th and 28th February 2022. This was detected and reported by community members in Iran who shared relevant OONI data (https://twitter.com/xhdix/status/1497961275225214980/) OONI data involving the block can be accessed and viewed through the following MAT chart: https://explorer.ooni.org/experimental/mat?probe_cc=IR&test_name=web_connec… *## Notable community use of OONI Probe and OONI data### VEsinFiltro report on the blocking of news media websites in Venezuela* On 1st February 2022, our Venezuelan partner, VEsinFiltro, published a report documenting the blocking of news media websites in Venezuela: https://vesinfiltro.com/noticias/2022-02-01-bloqueo-Noticias/ Their report makes use of OONI data, as well as network measurement data collected from their own custom tools. *### VEsinFiltro report on the blocking of circumvention tools in Venezuela* On 7th February 2022, VEsinFiltro published a report documenting the blocking of circumvention tools in Venezuela: https://vesinfiltro.com/noticias/2022-02-07-restriction-circumvention-tools/ Their report makes use of OONI data, involving the analysis of measurements collected from the OONI Probe Tor and Web Connectivity experiments. *### ECOWAS court ruling on internet shutdown in Togo* Togo’s internet shutdown amid protests in September 2017 was challenged in the “Amnesty International Togo and Ors v. The Togolese Republic” case ( https://globalfreedomofexpression.columbia.edu/cases/amnesty-international-…) brought to the Community Court of Justice of the Economic Community of West African States (ECOWAS). In preparation for this case, the OONI team was asked to share relevant technical expertise, along with other experts. We analyzed the internet shutdown and shared a technical analysis in support of the court case ( https://twitter.com/FrancoisPatuel/status/1494030238388563968) As an outcome, the Community Court of Justice of ECOWAS held that the Togolese government violated the applicants’ right to freedom of expression by shutting down the internet during protests in September 2017, and ordered the Respondent State of Togo to take measures to guarantee the “non-occurrence” of a future similar situation. The ECOWAS court ruling on the September 2017 internet shutdown in Togo won the Columbia University Global Freedom of Expression 2022 Significant Legal Ruling Award ( https://globalfreedomofexpression.columbia.edu/prizewinners2022/) *## Community activities### OONI workshop in Tanzania by Zaina Foundation* On 28th February 2022, Zaina Foundation (our partner in Tanzania) facilitated an OONI workshop as part of their broader Digital Security Training for women journalists in Dar es Salaam ( https://twitter.com/ZainaFoundation/status/1498330087258742789) *### OONI FAQ translated to Khmer by DigitalHub101* Cambodian civil society group, DigitalHub101, published a version of the OONI FAQ (https://ooni.org/support/faq/) in Khmer: https://digitalhub101.com/ooni/ This new page features information in Khmer about OONI, OONI Probe, OONI Explorer, and OONI research reports. We linked to their translated version in the main OONI FAQ page to enable more community members to benefit from the translation. *### OONI Community Meeting* On 22nd February 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/) during which we discussed the following topics: 1) “Internet Protection Bill” in Iran 2) OONI measurement coverage from Zimbabwe *## Userbase* In February 2022, 36,927,939 OONI Probe measurements were collected from 2,556 AS networks in 158 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0

OONI Monthly Report: January 2022
by Maria Xynou 18 Apr '22

18 Apr '22

Hello, These days we're primarily supported by the DRL, which is why we do quarterly reporting. That said, we'd like to share monthly updates from the OONI team with the community, hence our January 2022 report shared below. I'll also follow-up to share our February and March 2022 reports. *# OONI Monthly Report: January 2022* Throughout January 2022, the OONI team worked on the following sprints: * Sprint 56 (3rd - 16th January 2022) * Sprint 57 (17th - 30th January 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## OONI Probe apps* In January 2022, Norbel Ambanumben joined the OONI team to lead the development of the OONI Probe mobile app. In preparation for the next release, various improvements were made to the OONI Probe mobile app, including: * Annotating measurements coming from VPNs ( https://github.com/ooni/probe/issues/1901) * Faster test startup time; * Improvements to the progress bar tracking; * Improvements to the minimisation of running tests ( https://github.com/ooni/probe/issues/1976, https://github.com/ooni/probe/issues/1981) * Adding links to the OONI blog and research reports in the OONI Probe mobile app (https://github.com/ooni/probe-android/pull/472) * CI fix (https://github.com/ooni/probe/issues/1924) On OONI Probe Desktop, we fixed a bug related to viewing RiseupVPN test results (https://github.com/ooni/probe-desktop/pull/272, https://github.com/ooni/probe/issues/1953) *## OONI Probe Web* We made progress on the backend components required for the operation of OONI Probe Web. As OONI Probe Web cannot use the same strategy as the OONI Probe apps to discover the country code and network of the probe (since we can’t ship a geoIP database in the client-side app), we had to add backend support for doing this operation. Specifically, we added support for returning geoIP information in the OONI check-in API (https://github.com/ooni/api/pull/276) and performed some quality testing of the db-ip geoIP database. We are considering eventually using the same mechanism for OONI Probe desktop and mobile probes too, as it would allow us to update the geoIP database without needing to make a new software release. *## OONI Run* We worked on improvements to the OONI Run platform. In particular, we fixed a bug affecting the generation of OONI Run links ( https://github.com/ooni/run/issues/100, https://github.com/ooni/run/pull/101) we fixed a bug affecting OONI Run links on iOS ( https://github.com/ooni/probe/issues/1978) and we made improvements to how OONI Run is deployed (https://github.com/ooni/run/pull/103) *## Expanding censorship measurement methodologies### Websteps* We worked on refactoring of the probe-engine codebase in preparation for Websteps development (our new experiment for measuring the blocking of websites). This included code improvements following the first Websteps prototype (https://github.com/ooni/probe/issues/1956) We also discovered and fixed a bug that was introduced when Websteps was merged (https://github.com/ooni/probe/issues/1808) This bug stems from a change in the logic used by the measurement engine to decide whether to submit measurements. After merging Websteps, we unconditionally stopped submitting measurements for experiments that returned an error. This behavior is correct in that the experiment should return errors only to indicate fundamental failures (e.g., an experiment expecting that a URL is input is passed a string that is not a valid URL) or missing preconditions. However, because previously the core measurement engine did not enforce this rule, several experiments were actually returning an error for network conditions that would otherwise be associated with interesting network measurements. We discovered this bug while performing extensive QA after merging Websteps. We then needed to ensure that all OONI experiments were correctly returning errors only to signal fundamental failures or missing pre-conditions. As a result, we fixed DASH, ndt7, Psiphon, STUN Reachability and other minor experiments. It is worth noting that no stable OONI release ever shipped with this bug, and we indeed catched the bug because we knew merging Websteps entailed larger changes to the tree, so we chose to perform extensive QA to ensure we were not introducing regressions. *### OONI Probe CLI v3.14.0 pre-release* We released OONI Probe CLI v3.14.0-alpha.1 with support for running our new Tor Snowflake experiment (https://ooni.org/nettest/tor-snowflake/) a refactored Tor experiment, and a series of data quality improvements. Details are available through our pre-release notes: https://github.com/ooni/probe-cli/releases/tag/v3.14.0-alpha.1 Notably, this pre-release includes: * Support for running the new Tor Snowflake experiment on OONI Probe Mobile and Desktop (https://github.com/ooni/probe/issues/1917) * Refactored Tor experiment to use the newer underlying measurement library, which allowed us to cleanup legacy code and reduce the overall complexity of the network measurement engine ( https://github.com/ooni/probe/issues/1688, https://github.com/ooni/probe/issues/1970, https://github.com/ooni/probe/issues/1545 ). *### Other improvements* We also worked on a series of other improvements. Specifically, we: * Started designing a new experiment that pings DNS servers over UDP ( https://github.com/ooni/probe/issues/1987, https://github.com/ooni/probe-cli/pull/674) * Researched application-level transparent proxies to understand whether we could use them to perform more realistic QA checks before shipping releases; * Discovered and discussed a potential data quality issue around how we determine DNS blocking in Web Connectivity, where DNS blocking and HTTP blocking conflict with each other, and HTTP blocking currently takes precedence, chose not to address the bug to keep the overall probe behavior consistent over time, and planned to ensure we don’t have the same issue in websteps (https://github.com/ooni/probe/issues/1975) * Fixed a bug potentially affecting the quality of measurements where we were not correctly detecting the end of a stream due to differences in how we represent errors and how the Go standard library expected errors to be ( https://github.com/ooni/probe/issues/1965) * Refactoring and improvements the code we use to communicate with the OONI backend trying to make the Go API more robust ( https://github.com/ooni/probe/issues/1951) * Refactored the CLI and engine codebases to increase abstraction and testability (https://github.com/ooni/probe/issues/1885) * Refactored code we use to collect low-level measurements to reduce measurement code duplication and improve consistency ( https://github.com/ooni/probe/issues/1957) *## Blog post on measuring HTTP/3 censorship with OONI Probe* In January 2022, we started hosting Kathrin Elmenhorst for her 3-month OTF Information Controls Research Fellowship. We had previously worked with Kathrin on adding HTTP/3 support to OONI Probe for measuring HTTP/3 censorship in China, Iran, India, and Kazakhstan. Based on this research, we collaborated on a research paper (“Web censorship measurements of HTTP/3 over QUIC”) which was submitted to AMC Internet Measurement Conference ( https://dl.acm.org/doi/abs/10.1145/3487552.3487836) To share the findings of this research with our broader community, Kathrin wrote a blog post which summarizes the measurement findings. We published this blog post here: https://ooni.org/post/2022-http3-measurements-paper/ *## OONI Explorer* OONI Explorer performance was significantly enhanced in January 2022. Migrating to the new Clickhouse-based backend has helped ensure that OONI Explorer queries work faster and more reliably. Now database-intense queries on OONI Explorer load within seconds, enabling users to more effectively track censorship events around the world based on real-time OONI data. We made a minor update to the OONI Data Policy to mention that OONI Explorer uses Sentry to log crash reports ( https://ooni.org/about/data-policy) *## New Circumvention Tool Reachability Dashboard* Throughout January 2022, we worked on creating a new Dashboard which presents aggregate views of real-time OONI data based on the reachability testing of circumvention tools ( https://explorer.ooni.org/chart/circumvention) In particular, this dashboard presents charts based on the reachability testing of Psiphon, Tor, and Tor Snowflake in every country around the world (https://github.com/ooni/ooni.org/issues/774) Through these charts, users can gain a bird's-eye view on the testing of these tools in each country over time, and plot charts based on the countries and date range of their choice (https://github.com/ooni/explorer/pull/653) As these charts present aggregates of anomalous measurements, it is possible to infer potential cases of blocking (or cases when the tested tools were unreachable on tested networks due to other reasons). *## OONI backend* Throughout January 2022, we worked on migrating the OONI API to the new Clickhouse-based backend in production ( https://github.com/ooni/pipeline/pull/371) This resulted in a significant improvement to OONI Explorer performance (which relies on the OONI API), as OONI Explorer queries now work much faster and more reliably. We expanded OONI’s data analysis capabilities to support processing STUN reachability experiments (https://github.com/ooni/ooni.org/issues/767) The measurements of these tests are very closely related to Tor Snowflake, since the PT relied on STUN for NAT hole punching. We also did further testing and performance improvements to the data analysis code in preparation for the production launch. Moreover, progress on the OONI API was made to have stable aggregate views for displaying pluggable transport test results ( https://github.com/ooni/ooni.org/issues/769) Specifically, we implemented new API endpoints to return: * The runtime of PT test results; * The number of PT metrics grouped by country (this allows us to filter in the UI the country list to only those for which we have data). We also: * Added backend support for scoring measurements collected from the HTTP version of WhatsApp Web (https://github.com/ooni/backend/issues/546) * Improved the speed of the ooniprobe list command by refactoring the database queries (https://github.com/ooni/probe-cli/pull/662) * Looked into migrating the ooni.org website to another provider ( https://github.com/ooni/ooni.org/issues/1064) to improve its accessibility around the world; * Carried out some research and experimentation related to enabling third-party analysis of OONI data ( https://github.com/ooni/backend/issues/514#issuecomment-1016525686) * Wrote a document that outlines the features of various OONI measurements that are useful for performing data analysis tasks (in order to extract per-website metrics). *## Improving third party usage of OONI data* To enable third-party use of OONI data (particularly by researchers), we carried out some research and collected feedback on how to improve the user experience for third party consumers of OONI data. As part of this, we decided to offer a Command Line Interface (CLI) tool (called oonidata) that researchers can run to download raw data dumps of OONI data. A summary of this research can be found in this comment: https://github.com/ooni/backend/issues/514#issuecomment-1016525686 *## Preparing for a security audit* In preparation for a security audit of OONI software projects, we worked on extensive threat modeling. This involved identifying and mapping all potential threats that could affect various OONI software components, determining the probability, impact, and risk of each potential threat, as well as strategies for mitigation. The documentation that we prepared will support the upcoming security audits of OONI tools. *## Collaboration with Netalitica* Netalitica researchers continued to do excellent work in reviewing and updating the Citizen Lab test lists. In January 2022, we reviewed their updates to the test lists for Singapore, Myanmar, and Vietnam ( https://github.com/citizenlab/test-lists/pull/885) as well as their updates to the test lists for Cuba, Cameroon, and Bolivia ( https://github.com/citizenlab/test-lists/pull/888) We also opened a pull request based on Netalitica updates to the test lists for Nicaragua and Ecuador: https://github.com/citizenlab/test-lists/pull/881 *## Test list updates* In January 2022, we contributed minor updates to several Citizen Lab test lists, as documented through the following pull requests: https://github.com/citizenlab/test-lists/pull/882, https://github.com/citizenlab/test-lists/pull/883, https://github.com/citizenlab/test-lists/pull/884 We reviewed and merged more than 50 pull requests from the Citizen Lab test list repository, some of which included new contributions via our new web platform (https://test-lists.ooni.org/) *## Created an audio recording for a podcast on technical terms* We created a 10-minute audio recording on “What is the Internet?” for the ABC Digital podcast on “A Glossary of Technological Terms”. To create this recording, we wrote a script describing what is the internet, which we then recorded and shared with Conflux Center. This recording was subsequently shared with training participants of the “Technologies and Digital Tools for Peacekeeping and Political Missions: Data, Research, Analysis, & Communication” course organized by Conflux Center ( https://www.confluxcenter.org/wp-content/uploads/2022/01/Virtual-Course-Cat… ). *## Censored Planet data user study* Three OONI team members participated in Censored Planet’s data user study. As part of this user study, we reviewed the new Censored Planet Dashboard and shared feedback through a survey and through interviews with the Censored Planet team. *## Community activities### OONI training for UN peacekeeping missions* On 25th January 2022, OONI’s Maria facilitated an OONI training (“OONI - Practical exercise measuring Internet censorship”) for UN peacekeeping missions as part of a course (“Technologies and Digital Tools for Peacekeeping and Political Missions: Data, Research, Analysis, & Communication”) organized by Conflux Center. As part of the training, Maria introduced participants to OONI censorship measurement tools, provided a live demo of using OONI Probe for measuring internet censorship, and facilitated hands-on exercises for using OONI Probe based on real-world scenarios. Information about the virtual training course is available here: https://www.confluxcenter.org/wp-content/uploads/2022/01/Virtual-Course-Cat… *### OONI Probe workshop for partners in Southeast Asia* On 28th January 2022, OONI’s Maria facilitated an OONI Probe workshop for Sinar Project partners in Southeast Asia participating in the iMap Regional Workshop. As part of this workshop, Maria provided a theoretical overview of OONI Probe and OONI Run, a live demo of using OONI Probe for measuring internet censorship, a live demo for using OONI Run for coordinating censorship testing, and facilitated several hands-on exercises based on real-world scenarios. *### OONI Community Meeting* On 31st January 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/) during which we discussed RAACT (Resistance Against Automated Content Takedown): a new project led by Tracking Exposed (https://tracking.exposed/) to measure content takedowns on social media platforms (Facebook, Instagram, TikTok, Youtube). As part of the meeting, we discussed how the OONI community could get involved with the RAACT project. *## Userbase* In January 2022, 39,215,688 OONI Probe measurements were collected from 2,634 AS networks in 171 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0

Tor Browser Meeting moved to Tuesday (2022-04-19)
by Richard Pospesel 15 Apr '22

15 Apr '22

Hi Everyone, Many folks are taking Monday off for holiday, so we'll be moving the weekly Tor Browser meeting to 2022-04-19 @ 1500 UTC in #tor-meeting on OFTC. best, -Richard

1 0

Anti-censorship team meeting notes, 2022-04-14
by Shelikhoo 14 Apr '22

14 Apr '22

Hey everyone! Here are our meeting logs: http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-04-14-15.59.html And our meeting pad: Anti-censorship work meeting pad -------------------------------- Next meeting: Thursday April 21th 16:00 UTC Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC (channel is logged while meetings are in progress) == Goal of this meeting == Weekly checkin about the status of anti-censorship work at Tor. Coordinate collaboration between people/teams on anti-censorship at Tor. == Links to Useful documents == * Our anti-censorship roadmap: * Roadmap: https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards * The anti-censorship team's wiki page: * https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home * Past meeting notes can be found at: * https://lists.torproject.org/pipermail/tor-project/ * Tickets that need reviews: from sponsors we are working on: * All needs review tickets: https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s… * Sponsor 28 * must-do tickets: https://gitlab.torproject.org/groups/tpo/-/milestones/10 * possible tickets: https://gitlab.torproject.org/groups/tpo/-/issues?scope=all&utf8=%E2%9C%93&… * Sponsor 96 * https://gitlab.torproject.org/groups/tpo/-/milestones/24 == Announcements == * Roger is working on a Defcon talk submission, around the Russia censorship thing. Soon it will be ready for internal feedback and then we'll submit it. (https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/73) == Discussion == * [Distributed Snowflake Server Support](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transp… is ready for testing * A testing Broker will be setup * A secondary Bridge from dcf will be used for testing * Once testing is concluded, get help from other orgs to run other secondary snowflake bridges(like in Asia) * shelikhoo will create a ticket for an automated bridge setup system == Actions == * == Interesting links == * https://ntc.party/t/reported-shutdown-in-turkmenistan-2022-04-11/2224 * Seems to be over now https://twitter.com/CloudflareRadar/status/1513907409579876360 https://radar.cloudflare.com/tm == Reading group == * We will discuss "Blocking of HTTP/3 (QUIC) in Russia" on Apr/28th/2022 * https://github.com/net4people/bbs/issues/108 * Questions to ask and goals to have: * What aspects of the paper are questionable? * Are there immediate actions we can take based on this work? * Are there long-term actions we can take based on this work? * Is there future work that we want to call out, in hopes that others will pick it up? == Updates == Name: This week: - What you worked on this week. Next week: - What you are planning to work on next week. Help with: - Something you need help with. anadahz: 2022-01-27 Last week: - Increase timeout check cycles for default-bridge-felix-1 and default-bridge-felix-2 as they have been generating too many alerts: https://gitlab.torproject.org/tpo/anti-censorship/monit-configuration/-/mer… cecylia (cohosh): last updated 2022-04-14 Last week: - moved tor-specific snowflake code out of client library (snowflake#40124) - merge request: snowflake!85 - bump version of webrtc dependency (snowflake#40127) - merge request: snowflake!86 - commented on snowflake web workflow (snowflake#40125) - opened issue to bump version of snowflake and webrtc in tor browser (tor-browser-build#40474) - work on conjure test environment setup This week: - continued work on conjure PT - continue snowflake maintenance tasks Needs help with: dcf: 2022-04-14 Last week: - monitored the transition of the snowflake bridge to its new server https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Next week: - look at STATUS VERSION proposal https://gitlab.torproject.org/tpo/core/torspec/-/merge_requests/63 - install second snowflake bridge site https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Help with: agix: 2021-02-10 Last week: - Continued work on gettor-twitter Next week: - Hopefully finish the task Help with: - arlolra: 2022-04-07 Last week: - Merged the rest of snowflake !81 Next week: - Get to snowflake-webext #10 Evergreen: - Figure out where in pion/webrtc ALPN should be configured and used - Maybe add Chacha20Poly1305 to pion/dtls https://github.com/pion/dtls#planned-features https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Help with: - maxb: 2021-09-23 Last week: - Worked on https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… re: utls for broker negotiation - Had conversation with someone about upstream utls http round tripper https://github.com/refraction-networking/utls/pull/74 - Too busy with work :/ Next week: - _Really_ want to get a PR for utls round tripper meskio: 2022-04-07 Last week: - document circumvention settings API (bridgedb#40043) - accept bridge distributor request changes (rdsys#104) - investigate bridgestrap reporting odd timing (tor#40592) - review cross compile docker snowflake-proxy (docker-snowflake-proxy!6) Next week: AFK Shelikhoo: 2022-04-14 Last Week: - [Merge Request Awaiting] Add SOCKS5 forward proxy support to snowflake (snowflake!64) - [Coding & Deployment] Proposal: Centralized Probe Result Collector (anti-censorship/team#54) - [Discussion] Centralized Probe Log Collection Ascension Request - [Discussion] Hosting Centralized Probe Log Collection Server on TPA managed VPS - [Discussion] Proposal: Support for Dynamic IP obfs4 bridges with unattended proxy information update(aka "Subscription") - [Discussion+Coding] Support Automatic Bridge Info update(shelikhoo/LogCollectorAncillary#1) - [Discussion+Coding] Remove Inactive Bridge from Test Result(shelikhoo/LogCollectorAncillary#2) - [Coding] Distributed Snowflake Bridges - Broker - [Coding] Distributed Snowflake Bridges Testing Environment - Dockerlized Next Week: - [Coding] Distributed Snowflake Bridges (continue) - [Deployment] Distributed Snowflake Bridges Testing Broker Itchy Onion: 2022-04-14 Last week: - pair programming with Cecylia on the general workflow for updating racecar plugin - resolving a compatible issue introduced in racecar 2.1.0 This week: - continue working on racecar 2.1.0 (need to rebase snowflake) - set up LDAP HackerNCoder: 2021-12-16 This week: Last/done: Setup web mirror on tor.encryptionin.space Next: Get (new VPs with) new IP and setup new web mirror on new domain hanneloresx: 2021-3-4 Last week: - Submitted MR for bridgestrap issue #14 Next week: - Finish bridgestrap #14 - Find new issue to work on Help with: - ln5: 2022-03-24 Last week: - Received hardware for a new home for snowflake.tpn; discussed OS configuration with dcf in private email Next week: - Install Debian 11 on new snowflake.tpn - Possibly rack new snowflake.tpn, if I can get access and find time to do the transportation Need help with: - Input on urgency appreciated, for my planning - Input on preferred CPU configuration, esp wrt HyperThreading -- lacking input before deployment in data centre, the machine will *not* have HT disabled and CPU bug mitigation will thus have to be carried out by the kernel

1 0

New Launch: OONI Measurement Aggregation Toolkit (MAT)
by Maria Xynou 12 Apr '22

12 Apr '22

Hello, Today the OONI team is thrilled to announce the public launch of our *new Measurement Aggregation Toolkit (MAT)*! Access the MAT here: https://explorer.ooni.org/chart/mat Learn all about the MAT through our blog post: https://ooni.org/post/2022-ooni-mat/ *# What is the MAT?* Imagine being able to track internet censorship based on real-time data collected from networks around the world. And imagine being able to create your own charts (based on this data) with the click of a button. That's the MAT! :) The MAT is a tool that enables you to *create your own custom charts based on aggregate views of real-time OONI data* collected from around the world. Use the MAT to* track internet censorship worldwide* based on OONI data! *# Who is the MAT for? * The MAT was built for researchers, journalists, and human rights defenders interested in examining internet censorship around the world. *# Why use the MAT?* When examining cases of internet censorship, it's important to *look at many measurements at once* ("in aggregate") in order to identify trends and patterns, and answer key questions like the following: * Does the testing of a service (e.g. Facebook) present signs of blocking every time that it is tested in a country? This can be helpful for ruling out false positives. * What types of websites (e.g. human rights websites) are blocked in each country? * In which countries is a specific website (e.g. bbc.com) blocked? * How does the blocking of different apps (e.g. WhatsApp or Telegram) vary across countries? * How does the blocking of a service vary across ASNs? * How does the blocking of a service change over time? When trying to answer questions like the above, we normally perform relevant data analysis (instead of inspecting measurements one by one). The MAT incorporates our data analysis techniques, enabling you to answer such questions without any data analysis skills, and with the click of a button! Further details on using the MAT and interpreting MAT charts are available through our blog post: https://ooni.org/post/2022-ooni-mat/ We hope the internet freedom community finds the MAT useful for investigating and responding to internet censorship events around the world. Please help spread the word: https://twitter.com/OpenObservatory/status/1513818001153437700 If you have any questions, feedback, or feature requests, we’d love to hear from you! You can open a ticket (https://github.com/ooni/explorer/issues/) or write us an email (contact(a)openobservatory.org) We thank all OONI Probe users worldwide who have contributed (and continue to contribute) measurements, supporting the MAT. We also thank community members for their invaluable feedback. <3 Thanks, OONI team.

1 0

Next network-health meeting on 04/25/2022 1600 UTC
by Georg Koppen 12 Apr '22

12 Apr '22

Hello everyone! For those of you being interested in network-health related topics and work: due to the upcoming Easter holidays we'll skip our weekly meeting on 04/18/2022. The next regular sync will be on 04/25/2022 at 1600 UTC in OFTC's #tor-meeting as usual. See you there, Georg

1 0

Anti-censorship team meeting notes, 2022-04-07
by Shelikhoo 07 Apr '22

07 Apr '22

Hey everyone! Here are our meeting logs: http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-04-07-15.59.html And our meeting pad: Anti-censorship work meeting pad -------------------------------- Anti-censorship work meeting pad -------------------------------- Next meeting: Thursday April 14th 16:00 UTC Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC (channel is logged while meetings are in progress) == Goal of this meeting == Weekly checkin about the status of anti-censorship work at Tor. Coordinate collaboration between people/teams on anti-censorship at Tor. == Links to Useful documents == * Our anti-censorship roadmap: * Roadmap: https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards * The anti-censorship team's wiki page: * https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home * Past meeting notes can be found at: * https://lists.torproject.org/pipermail/tor-project/ * Tickets that need reviews: from sponsors we are working on: * All needs review tickets: https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s… * Sponsor 28 * must-do tickets: https://gitlab.torproject.org/groups/tpo/-/milestones/10 * possible tickets: https://gitlab.torproject.org/groups/tpo/-/issues?scope=all&utf8=%E2%9C%93&… * Sponsor 96 * https://gitlab.torproject.org/groups/tpo/-/milestones/24 == Announcements == * Next snowflake bridge migration scheduled for next week https://gitlab.torproject.org/tpo/tpa/team/-/issues/40716 == Discussion == * Nickname for second bridge site? https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… * The existing bridge site has nickname "flakey" (actually flakey1, flakey2, etc.), since the second bridge site will have a different fingerprint it should have a different nickname * Can leave suggestions on the issue * Discuss about cooperation with Greatfire (by serene) * keroserene and shelikhoo had a call with Greatfire * one of the topics discussed was collaboration regarding FreeBrowser (https://freebrowser.org/, https://github.com/greatfire/freebrowser) and Snowflake * Some notes from the call: https://pad.riseup.net/p/greatfire-snowflake-notes == Actions == * == Interesting links == * Extended DNS Error provides more error codes to indicate why a DNS query failed; code 16 means "censored". (Like status code 451 in HTTP.) https://www.rfc-editor.org/rfc/rfc8914.html#section-4.17 * Ukraine-focused circumvention VPN, only unblocks a handful of sites like VK, Mail.ru * https://zaborona.help/ * https://ntc.party/t/ukraine/2135/3 * Uses static SOCKS proxies? E.g. srv1.vpn.zaborona.help https://zaborona.help/graph.html == Reading group == * We will discuss "Balboa: Bobbing and Weaving around Network Censorship" on April 7 * https://www.usenix.org/system/files/sec21-rosen.pdf * https://censorbib.nymity.ch/#Rosen2021a * Questions to ask and goals to have: * What aspects of the paper are questionable? * Are there immediate actions we can take based on this work? * Are there long-term actions we can take based on this work? * Is there future work that we want to call out, in hopes that others will pick it up? == Updates == Name: This week: - What you worked on this week. Next week: - What you are planning to work on next week. Help with: - Something you need help with. anadahz: 2022-01-27 Last week: - Increase timeout check cycles for default-bridge-felix-1 and default-bridge-felix-2 as they have been generating too many alerts: https://gitlab.torproject.org/tpo/anti-censorship/monit-configuration/-/mer… cecylia (cohosh): last updated 2022-03-31 Last week: - onboarded itchy onion onto s28 tasks - reviews - work on conjure PT This week: - continued work on conjure PT - continue to monitor snowflake broker stats Needs help with: dcf: 2022-04-07 Last week: - set up the snowflake bridge's permanent server and scheduled a time for the migration https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… https://gitlab.torproject.org/tpo/tpa/team/-/issues/40716 - opened an issue for tracking the installation of a second bridge site https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - approved last piece of forward-fingerprint patch https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - posted hints about CPU profiling snowflake-webext in Chrome https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - more correspondence with OTF about a rapid response grant for the snowflake bridge Next week: - look at STATUS VERSION proposal https://gitlab.torproject.org/tpo/core/torspec/-/merge_requests/63 - install second snowflake bridge site https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Help with: agix: 2021-02-10 Last week: - Continued work on gettor-twitter Next week: - Hopefully finish the task Help with: - arlolra: 2022-04-07 Last week: - Merged the rest of snowflake !81 Next week: - Get to snowflake-webext #10 Evergreen: - Figure out where in pion/webrtc ALPN should be configured and used - Maybe add Chacha20Poly1305 to pion/dtls https://github.com/pion/dtls#planned-features https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Help with: - maxb: 2021-09-23 Last week: - Worked on https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… re: utls for broker negotiation - Had conversation with someone about upstream utls http round tripper https://github.com/refraction-networking/utls/pull/74 - Too busy with work :/ Next week: - _Really_ want to get a PR for utls round tripper meskio: 2022-04-07 Last week: - document circumvention settings API (bridgedb#40043) - accept bridge distributor request changes (rdsys#104) - investigate bridgestrap reporting odd timing (tor#40592) - review cross compile docker snowflake-proxy (docker-snowflake-proxy!6) Next week: AFK Shelikhoo: 2022-04-07 Last Week: - [Merge Request Awaiting] Add SOCKS5 forward proxy support to snowflake (snowflake!64) - [Coding & Deployment] Proposal: Centralized Probe Result Collector (anti-censorship/team#54) - [Discussion] Centralized Probe Log Collection Ascension Request - [Discussion] Hosting Centralized Probe Log Collection Server on TPA managed VPS - [Discussion] Bridges should report implementation versions of their pluggable transports - [Coding] Add SOCKS5 forward proxy support to snowflake (snowflake!64) - built-in DNS - [Coding] Distributed Snowflake Bridges - Broker(continue) - (Reply a lot of emails) Next Week: - [Coding] Distributed Snowflake Bridges - Broker (continue) Itchy Onion: 2022-04-07 Last week: - snowflake !84 - racecar onboarding This week: - racecar local testing - set up LDAP Help with: - upgrade to be compatible with racecar 2.1.0 HackerNCoder: 2021-12-16 This week: Last/done: Setup web mirror on tor.encryptionin.space Next: Get (new VPs with) new IP and setup new web mirror on new domain hanneloresx: 2021-3-4 Last week: - Submitted MR for bridgestrap issue #14 Next week: - Finish bridgestrap #14 - Find new issue to work on Help with: - ln5: 2022-03-24 Last week: - Received hardware for a new home for snowflake.tpn; discussed OS configuration with dcf in private email Next week: - Install Debian 11 on new snowflake.tpn - Possibly rack new snowflake.tpn, if I can get access and find time to do the transportation Need help with: - Input on urgency appreciated, for my planning - Input on preferred CPU configuration, esp wrt HyperThreading -- lacking input before deployment in data centre, the machine will *not* have HT disabled and CPU bug mitigation will thus have to be carried out by the kernel

1 0

Community Team meeting notes - April 04 2022
by gus 06 Apr '22

06 Apr '22

Hi, Here's the Community Team meeting logs: http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-04-04-13.59.log… And our meeting pad: ## Tor Community Team meeting pad Next meeting: Monday, April 11, 2022 - 1400 UTC Weekly meetings, every Monday at 14:00 UTC, in #tor-meeting at OFTC (channel is logged while meetings are in progress) ## Goal of this meeting Weekly checkin about the status of Community Team work at Tor. ## Links to Useful documents * Previous meetings: https://forum.torproject.net/tag/community-team * Monthly l10n hangout: * https://community.torproject.org/localization/hangouts/ * Known issues with Tor Browser: * https://tb-manual.torproject.org/known-issues/ and Tor Browser for Android: https://tb-manual.torproject.org/mobile-tor/#known-issues ## Discussion * Ukraine spike - * https://gitlab.torproject.org/tpo/community/support/-/issues/40068 ## Updates Gus: Last week: - Organizing Tor trainings for S30 (Mexico and Brazil) - Joined the Tor relay operator meetup - 1:1s with the team - Sponsor9 meeting - Sponsor96 meeting This week: - Getting back to some relay operators - Send the relay operator meetup notes to the mailing list - Organizing the last three trainings in Brazil for S30 - Sponsor9 - preparing the LATAM meetup - Meeting with Rhatto and others to discuss onion sre plans Joydeep: Last week: - support work on RT, Forum, cdr.link - helped with S125 report - tags gardening (on Tor Forum) https://gitlab.torproject.org/tpo/community/support/-/issues/40063 - Joined the Tor Relay Operator Meetup This week: - S125 Meeting - Tor Browser 10.0.10 and TBA 11.0.11 testing and post-release support work emmapeel: Last week: - Prepare translation project for sponsor123, review languages. - Improve documentation for translators - also because of new user-password thingie - Ask for screenshots for Tor Browser - get new translators for russian This week: - Add screenshots to transifex - translation russian sposor125 raya: Last week: - s123: worked on march 2022 narrative report - s123: reviewed wireframes, check them out! https://www.figma.com/proto/BydkbTJ87pfo7xhAGHsPGo/USAGM-Landing-Page - 1:1 with gus! :-) - s123: bi-weekly meeting - s123: onion support repo cleanup - s123: fixing language priorities on progress tracker - joined first ever tor relay meetup! This week: - close: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/56 - close: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/35 - close: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/63 - work on: https://gitlab.torproject.org/tpo/web/community/-/issues/265 - s123: coordinate next steps with partners - s123: follow up on landing page development Help with: - getting metrics for narrative report Nina: Last week: - User support on RT, Telegram, IRC. - Finising s125 report - Checking the russian mail services on how do they procced with TP emails This week: - User support. - s125 final meeting rhatto: Last week: - S123 - Onionprobe enhancements: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/58 This week: - 1:1 with Gus - S123 - Onionprobe testbed: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/59 - S123 - Review and merge Oniongroove specs into main branch: https://gitlab.torproject.org/tpo/onion-services/oniongroove/-/merge_reques… - Start to organize the new Onion Services Gitlab Group: https://gitlab.torproject.org/tpo/onion-services - Continue Research on Onion Services UX: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/64 -- The Tor Project Community Team Lead

1 0

Joydeep's Monthly Status Report for March 2022
by Joydeep Sen Gupta 06 Apr '22

06 Apr '22

Hello everyone! This is my monthly report for the month of March. Much like in February, we saw a even greater uptick in the number of requests for unlisted obfs4 bridges. We received 72 unique requests in one month. Understandably so, since some of the obfs4 bridges we were sharing with users went offline at the end of February. Within a month of launching the Telegram User Support Channel for users in Russia, we are seeing more and more questions from users who are new to the Tor ecosystem and curious how all of it works. I am mostly helping with these questions and other technical questions related with little-t-tor, snowflake, Tor Browser design and usability. There were a number of Tor Browser releases last month, namely, 11.5.a5 (Android), 11.0.7, 11.5.a7 (Android), 11.5.a6, 11.0.8 (Android), 11.0.9 and 11.5.a8. A lot of my work was around testing and user support around these specific releases. We received a few reports of the "Tor Browser for Android 11.0.8 crashing on launch" bug [1] and after consulting with the applications team, I posted the workaround [2] on the forum / blog post. I also authored a article (shout-out to @nina13 for the translation!) on the forum [3] encouraging users in Russia to keep their Tor browser up-to-date in light of (1) critical security updates that shipped with Tor Browser 11.0.7 and (2) a lot of users in Russia were unable to update and GetTor for some time was handing out links to Tor Browser 11.0.4. In the past month, I was also able to help some new Tor relay operators setup bridges. That pretty much sums up the highlights from last month and following are some quick stats and highlights from our official support channels. ## Frontdesk Timeline : 01 Mar - 31 Mar 2022 Tickets: new: 26 open: 15 resolved: 792 Breakdown of number of RT tickets received with respect to operating system: (Note: This includes tickets where the user mentioned the operating system or it was evident from the issue they were running into and/or enclosed screenshots.) Windows - 11 macOS - 6 GNU/Linux - 6 Android- 13 Breakdown of most frequent tickets (at least 3 RT tickets): 1. 443 RT Tickets - How to use Tor Bridges in Russia. [4] 2. 72 RT Tickets - Private Bridge requests. This is not related to the .ru censorship but requests from Tor users in China, Iran, etc. [5] 3. 7 RT Tickets - How to circumvent geo-based restrictions with Tor? 4. 8 RT Tickets- How to change the default SE and add a new search engine to Tor Browser? [6] 5. 6 RT Tickets - Tor Browser Android 11.0.8 crashes on launch [1] 6. 5 RT Tickets - Help with setting up a Tor Bridge. ## Tor Forum Most popular topics in the Support category (in terms of no. of views): 1. "What is still blocking my IP address? (IP address on various block lists since running a non-exit Tor relay)". [7] 2. "Can we expect a stable Tor Browser release soon? ESR 91.6.1 appears to fix some critical security issues". [8] 3. "Have I Correctly Set Up My Bridge?" [9] 4. "Snowflake does not work anymore" [10] 5. "I have seen x unique clients (on my Tor Bridge)" [11] Thanks, -- Joydeep [1]: https://gitlab.torproject.org/tpo/applications/fenix/-/issues/40212 [2]: https://forum.torproject.net/t/new-release-tor-browser-11-0-8-android/2527/4 [3]: https://forum.torproject.net/t/tor-browser-11-0-7/2484 [4]: https://forum.torproject.net/t/tor-blocked-in-russia-how-to-circumvent-cens… [5]: https://support.torproject.org/censorship/connecting-from-china/ [6]: https://forum.torproject.net/t/custom-search-engine-in-tor-browser/2574 [7]: https://forum.torproject.net/t/what-is-still-blocking-my-ip-address/2564 [8]: https://forum.torproject.net/t/can-we-expect-a-stable-tor-browser-release-s… [9]: https://forum.torproject.net/t/have-i-correctly-set-up-my-bridge/2637/ [10]: https://forum.torproject.net/t/snowflake-does-not-work-anymore/2650 [11]: https://forum.torproject.net/t/i-have-seen-x-unique-clients/2755

1 0

Sponsor 38 (shadow simulator) update
by Jim Newsome 05 Apr '22

05 Apr '22

I've just shared the 2022-04 update in the Shadow forum: https://github.com/shadow/shadow/discussions/2007 Also it looks like I forgot to share the previous 2021-12 update here: https://github.com/shadow/shadow/discussions/1824 Both mirrored below: 2022-04 update This is part of a series of periodic updates of development in Shadow. This work is sponsored by the NSF <https://github.com/shadow/shadow/blob/main/docs/nsf_sponsorship.md>. Previous update: 2021-12 <https://github.com/shadow/shadow/discussions/1824>. We've merged 82 non-dependabot pull requests <https://github.com/shadow/shadow/pulls?q=is%3Apr+merged%3A2021-12-17..2022-…> and closed 18 issues <https://github.com/shadow/shadow/issues?q=closed%3A2021-12-17..2022-04-05+i…> since our previous update. Release status We are continuing to work on Shadow 2.1 <https://github.com/shadow/shadow/projects/5>. The biggest user-facing goal for this release is to support running golang programs in Shadow <https://github.com/shadow/shadow/milestone/30>, especially tor simulations using the snowflake pluggable transport <https://github.com/shadow/shadow/issues/1549>. We've also begun planning the Shadow 2.2 <https://github.com/shadow/shadow/projects/6> release, which will largely be a push to refactor and migrate more of the core Shadow code to Rust. Notable change since last update Emulation accuracy * Optionally move time forward in non-blocking syscalls <https://github.com/shadow/shadow/pull/1995>. Historically Shadow doesn't move time forward except when explicitly waiting for an event, such as for a deadline to pass or for data to arrive over the network. Conceptually, this emulates a system with an infinite number of infinitely fast CPUs. Normally this is sufficient for modeling networks where CPUs aren't expected to be a bottleneck. Unfortunately as we've explored running more software under Shadow, we've found a growing number of examples of code with "busy loops" <https://github.com/shadow/shadow/issues/1792>, which deadlock in this model. This new feature optionally models every syscall taking some small amount of time (e.g. a microsecond), which allows the simulation to escape most such loops. We are still testing and improving this feature, and expect some version of it to be enabled by default in the next release. * Fixed a corner case in getaddrinfo <https://github.com/shadow/shadow/pull/1998>. * Fixed several bugs in handling file descriptors: o Fixed pipe writable state when reader is closed <https://github.com/shadow/shadow/pull/1985>. o Have pipe writers return EPIPE if there are no readers <https://github.com/shadow/shadow/pull/1983>. o Fixed errno for socket with SOCK_SEQPACKET type <https://github.com/shadow/shadow/pull/1972>. o Don't fail on TCP_NODELAY <https://github.com/shadow/shadow/pull/1936>. * Reliably intercept time via vdso <https://github.com/shadow/shadow/pull/1951>. Previously we relied on intercepting calls to VDSO functions (such as |gettimeofday|) to be intercepted at the libc level via |LD_PRELOAD|. However, this doesn't work when the VDSO is used more directly, such as in golang, which would cause the program to get the real-world time instead of the simulated time. We now patch the VDSO itself at program start to reliably intercept these functions in such cases. * Implemented basic signal emulation <https://github.com/shadow/shadow/pull/1881>. This allows managed code to install signal handlers and send and receive signals within the simulation. (Sending a signal from outside the simulation to a managed process is still not supported). Notably this support is required to handle golang programs, and allows simulated processes to be shut down cleanly by scheduling |kill| processes to send appropriate signals. * Implemented the |select| system call <https://github.com/shadow/shadow/pull/1910>. Usability * Added an experimental option for strace-style logging <https://github.com/shadow/shadow/pull/1886> of syscalls made by managed code. * Stabilized the |--progress| <https://github.com/shadow/shadow/pull/1937> option, which periodically updates stderr with the simulation progress. Performance * Nightly shadow benchmarks are now being run to help detect performance regressions. We also have the ability to run the benchmark on our own development branches to investigate performance changes before merging these branches into Shadow. Benchmark results for a 5% Tor network are published publicly <https://github.com/shadow/benchmark-results/tree/master/tor>, but are only intended to be useful for Shadow developers. * Added a library for overriding crypto functions <https://github.com/shadow/shadow/pull/1853>. When enabled, this option overrides some openssl APIs with "no-op" implementations. This is a reimplementation of a feature previously available in Shadow's "tor plugin". It is intended primarily to improve the performance of tor simulations on hardware without accelerated AES operations, and is only supported on Debian 11. Happy simulating! The Shadow team 2021-12 update This is part of a series of periodic updates of development in Shadow. This work is sponsored by the NSF <https://github.com/shadow/shadow/blob/main/docs/nsf_sponsorship.md>. Previous update: 2021-10 <https://github.com/shadow/shadow/discussions/1695>. We've merged 78 pull requests <https://github.com/shadow/shadow/pulls?q=is%3Apr+merged%3A%3E%3D2021-09-30> and closed 17 issues <https://github.com/shadow/shadow/issues?q=closed%3A%3E%3D2021-09-30+is%3Ais…> since our previous update. Release status We have released Shadow 2.0 <https://github.com/shadow/shadow/releases/tag/v2.0.0>! Give it a try, and let us know if you run into any issues! We have begun work on Shadow 2.1 <https://github.com/shadow/shadow/projects/5>. Notable planned features are support for signals (which is needed <https://github.com/shadow/shadow/issues/1549> to reliably run golang programs under Shadow), and improved Unix socket support. Notable change since last update Emulation accuracy * Add unnamed unix sockets created with socketpair() <https://github.com/shadow/shadow/pull/1787> * Add support for the 'O_DIRECT' flag to pipes (packet mode) <https://github.com/shadow/shadow/pull/1746> * poll and epoll: wait for timeout even with no fd's <https://github.com/shadow/shadow/pull/1697> Performance improvements * Set default min runahead to 1ms <https://github.com/shadow/shadow/pull/1711>. This fixes a performance regression <https://github.com/shadow/shadow/issues/1699> in many-host simulations using multiple CPUs. Stability * Detect and handle death by signals <https://github.com/shadow/shadow/pull/1722> * Handle process exit while it has events scheduled <https://github.com/shadow/shadow/pull/1704> UI improvements * Move host heartbeat options to the experimental options, and disable by default <https://github.com/shadow/shadow/pull/1703> Shadow at Tor We have been collaborating with The Tor Project to use Shadow in its development and testing. We have been running Shadow simulations inside a Gitlab CI pipeline <https://gitlab.torproject.org/jnewsome/sponsor-61-sims> to help develop and tune improved congestion control algorithms in the upcoming 0.4.7 <https://lists.torproject.org/pipermail/tor-project/2021-December/003241.html> release. This is the first major application of Shadow inside the Tor Project itself, and we plan to use the pipeline we've developed for further testing and profiling. We have also been making progress on running Arti (the experimental new Rust implemenation of tor) under Shadow: https://gitlab.torproject.org/tpo/core/arti/-/issues/174. In the course of this work we have fixed some bugs in Arti, fixed an upstream bug in the async-io <https://github.com/smol-rs/async-io/pull/73> crate, and found and identified and fixed several emulation accuracy bugs in Shadow. Happy simulating! The Shadow team

1 0