- tor-project - lists.torproject.org

Joydeep's Monthly Status Report for October 2022
by Joydeep Sen Gupta 06 Nov '22

06 Nov '22

Hello everyone! These are my updates from October. Much of my work was concentrated around helping users in Iran circumvent censorship[1] and learning (and investigating) what usability and connectivity issues they were facing amidst the heavy censorship. This usually involved helping users with using pluggable transports with apps in the Tor ecosystem and investigating Tor logs to figure out if and how Tor was failing to connect. I have been able to respond to more than 500 support tickets from Iran in the last month but this is very much work in progress as internet is heavily censored in Iran right now. With the launch of the Year End Campaign, as always, we have generated a good deal of interest on our user support and discussion platforms and I have been responding to these queries. A lot of general Tor questions, questions about using Tor Browser, etc. We had 4 Tor Browser releases in the past month and I have been involved with some testing and also wrote some documentation for the Tor Browser Manual adding instructions for using the GetBridges Telegram bot[2] and a dedicated help section[3]. A quick overview of our user support channels: Timeline: 01 - 31 October 2022 # Frontdesk Tickets: open: 50 resolved: 766 Breakdown of most frequent tickets by numbers: 1. 99 RT Tickets - Circumventing Censorship with Tor in Iran. 2. 22 RT Tickets - Private Bridge requests from China[4] 3. 43 RT Tickets - How to use a Tor Bridge in Russia[5] # Telegram and Signal Support channel (cdr.link) Tickets resolved: 1475 The other most frequent ticket we received have been about: 1. 1140 tickets: Circumventing censorship in Iran. 2. 169 tickets: Circumventing censorship in Russia 3. 16 tickets - Using unlisted obfs4 bridges in China 4. 12 tickets - Questions about using Snowflake and setting up Snowflake proxies # Tor Forum Most popular topics in the Support category (in terms of no. of views): 1. "Standalone proxy - traffic relayed almost zero with restricted NAT type"[6] 2. "How can I use Snowflake with tor (service)?"[7] 3. "Why has Tor slowed down so much recently?"[8] 4. "Bookmarks in my Tor Browser"[9] 5. "Bridge or Snowflake? low RAM VPS"[10] Thanks, -- Joydeep [1]: https://forum.torproject.net/t/iran-circumventing-censorship-with-tor/4590 [2]: https://tb-manual.torproject.org/bridges/ [3]: https://gitlab.torproject.org/tpo/web/manual/-/issues/131 [4]: https://support.torproject.org/censorship/connecting-from-china/ [5]: https://forum.torproject.net/t/tor-blocked-in-russia-how-to-circumvent-cens… [6]: https://forum.torproject.net/t/standalone-proxy-traffic-relayed-almost-zero… [7]: https://forum.torproject.net/t/how-can-i-use-snowflake-with-tor-service/504… [8]: https://forum.torproject.net/t/why-has-tor-slowed-down-so-much-recently/502… [9]: https://forum.torproject.net/t/bookmarks-in-my-tor-browser/5310/7 [10]: https://forum.torproject.net/t/bridge-or-snowflake-low-ram-vps/5019/6

1 0

Cecylia's monthly status report, October 2022
by Cecylia Bocovich 05 Nov '22

05 Nov '22

Hi everyone, This is my status report for contract work done in October 2022. 1. Conjure development * Completed tor-browser-build changes needed for tor browser integration of conjure on desktop and mobile (not merged yet due to usability concerns) * wrote a guide for integrating PTs with Tor Browser builds [0] * Small client-side fixes for storing the assets directory 2. Snowflake web extension * Worked on fixes for website and extension translations * Updated polling frequency to help with proxy pool shortages * Made progress on Firefox MV3 updates * Reviewed and merged open MRs 3. Bridge reputation system * Wrote a plan for client-side development options [0] https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Tor-Browser-…

1 0

Rhatto's Monthly Status Report, October 2022
by rhatto 04 Nov '22

04 Nov '22

Hi all :) This is my monthly status report for October 2022. Main activities during the period: 0. Documentation: * Improved documentation about Onion Services UX enhancements. Now these docs are more organized to help research and discussion: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/wikis/Docu… https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/64 * Created a small timeline to track the major changes/developments in the Onion Services technology: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/wikis/Docu… I'd like to have some suggestions on what to include, with references if possible :) 1. [Onion Plan][]: * Drafted a roadmap proposal for making Onion Services easier to deploy, maintain, and advertise. * It's a follow up from the Onion Plan session from September: https://gitlab.torproject.org/tpo/team/-/wikis/202209MeetingOnionPlan * The idea is to release this plan when it's ready to gather general feedback. 2. [Onion Launchpad][], an Onion Service landing page template: * Created an automation script for deploying instances on GitLab: https://gitlab.torproject.org/tpo/onion-services/onion-launchpad/-/merge_re… 3. Usual Sponsor 123 deployments and maintenance. [Onion Plan]: https://gitlab.torproject.org/tpo/onion-services/onion-support/-/wikis/Docu… [Onion Launchpad]: https://gitlab.torproject.org/tpo/onion-services/onion-launchpad -- Silvio Rhatto pronouns he/him

1 0

Nina's Monthly status report for October'22
by Nina 04 Nov '22

04 Nov '22

Hi! This is my status report for October 2022. This month, I successfully resolved 1422 tickets. On Telegram (@TorProjectSupportBot) - 1313 On RT (frontdesk@tpo) - 109 Due to censorship events in Iran, most requests (>1000) continue coming from that country. In October, more users applied to us for assistance in connecting to Tor and troubleshooting. One hundred sixty-nine (169) Telegram requests came from Russia. Users mostly asked for bridges or assistance in cases when bridges did not work for them. Also, more than 20 requests came from Turkmenistan, where internet censorship is getting even more severe. In October, some changes were implemented to the @TorProjectSupportBot - now, it allows users to pick the country from the list. Currently on the list are Iran, China, Russia and Turkmenistan. That makes communication easier and faster. We continue getting requests from Kazakhstan users concerned about internet censorship events that can happen in November 2022, when the presidential election takes place. So we started to prepare a small campaign offering Kazakhstan users to install Tor Browser beforehand [1]. [1] https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/155

1 0

Anti-censorship team meeting notes, 2022-11-03
by Shelikhoo 03 Nov '22

03 Nov '22

Hey everyone! Here are our meeting logs: http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-11-03-15.58.html <http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-03-31-15.59.html> And our meeting pad: Anti-censorship work meeting pad -------------------------------- Next meeting: Thursday Nov 10 16:00 UTC Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC (channel is logged while meetings are in progress) == Goal of this meeting == Weekly check-in about the status of anti-censorship work at Tor. Coordinate collaboration between people/teams on anti-censorship at the Tor Project and Tor community. == Links to Useful documents == * Our anti-censorship roadmap: * Roadmap: https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards * The anti-censorship team's wiki page: * https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home * Past meeting notes can be found at: * https://lists.torproject.org/pipermail/tor-project/ * Tickets that need reviews: from sponsors we are working on: * All needs review tickets: * https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s… * Sponsor 28 * must-do tickets: https://gitlab.torproject.org/groups/tpo/-/milestones/10 * possible tickets: https://gitlab.torproject.org/groups/tpo/-/issues?scope=all&utf8=%E2%9C%93&… * Sponsor 96 * https://gitlab.torproject.org/groups/tpo/-/milestones/24 == Announcements == * == Discussion == * Lantern and snowflake * lantern is working on a p2p volunteer proxy system similar to snowflake * a highly concurrent client (capable of managing many simultaneous peer connections) that could do N:M multiplexing between downstream and upstream connections * very observable so that it could send lots of status information to the UI * Distributed Snowflake Server Support will be included in Tor Browser Testing Channel(Alpha) Soon: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_re… * snowflake-01 got 20% increase of usage the last 24h, after being stable for few weeks * PT args size issue: https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/104 == Actions == == Interesting links == == Reading group == * We will discuss "" on * * Questions to ask and goals to have: * What aspects of the paper are questionable? * Are there immediate actions we can take based on this work? * Are there long-term actions we can take based on this work? * Is there future work that we want to call out in hopes that others will pick it up? == Updates == Name: This week: - What you worked on this week. Next week: - What you are planning to work on next week. Help with: - Something you need help with. cecylia (cohosh): last updated 2022-11-03 Last week: - set conjure/tapdance assets directory to pt dir (conjure#11) - reviewed and merge fixed to right-adjusted langs (snowflake-webext#64) - fixed webext and badge build to only include 90%+ complete locales (snowflake-webext#62) - more progress on mozilla mv3 prep - discovered problem with webext translations (snowflake-webext#66) - reviewed and merged fix to close stale 0 byte transfer connections (snowflake-webext!54) - reviewed webext performance improvements (snowflake-webext!39) This week: - website/extension release (have to do extension translations by hand) - try again to reach out to conjure developers - start working on salmon Needs help with: dcf: 2022-11-03 Last week: - opened an issue to abbreviate `ice` specifications in snowflake bridge lines https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - contributed information about snowflake usage and Tor Metrics graphs https://gitlab.torproject.org/tpo/network-health/metrics/analysis/-/issues/… - did some review of the presentation of right-to-left languages in the snowflake webext https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Next week: - disable non-WireGuard SSH access to snowflake-02 https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - migrate goptlib to gitlab https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/86#note_282… - try Conjure PT development version https://forum.torproject.net/t/tor-dev-introducing-a-conjure-pt-for-tor/4429 - break up snowflake-server performance improvements into separate merge requests https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - document recent performance optimizations in Snowflake bridge installation guide Help with: meskio: 2022-11-03 Last week: - gettor email loop fix (rdsys#133) - gettor imap reconnection fixes (rdsys#134) - open a dicussion about PT arg passing (team#104) - send a tor-relays obfs4proxy upgrade reminder - review outreachy merge requests in rdsys Next week: - obfs4proxy 0.0.14 in Tor Browser Shelikhoo: 2022-11-03 Last Week: - [Merge Request Awaiting] Add SOCKS5 forward proxy support to snowflake (snowflake!64) - [Discussion & Deployment] Rollout of Distributed Snowflake Support - [Coding & Deployment] Proposal: Centralized Probe Result Collector (anti-censorship/team#54) - [Research] HTTPT Planning https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/http… - Rollout distributed snowflake (include definition of secondary bridge on broker) https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - Rollout distributed snowflake (get bridgeline into tor browser) https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_re… Next Week: - [Research] WebTunnel Planning (Continue) - [Research] Fix vantage point summary upload in China - Release New version of Snowflake WebExt - [Research] Censorship analysis for UDP traffic between Iran and rest of Internet: 2022 Q4: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… (Continue) - Finish all the reviews Itchy Onion: 2022-11-03 Last week: - Increase the number of snowflake proxies spawned in RACE and ran it against the CI test. I was able to pass every time (6 times in total). But since Tuesday I've been having trouble starting a new deployment with Rib. I've been in talk with TwoSix but so far nothing has helped. This week: - Fixed the Rib issue. Now snowflake passes CI tests 100%.

1 0

PieroV's Monthly Status Report, October 2022
by Pier Angelo Vendrame 01 Nov '22

01 Nov '22

Hi everyone! Here is my status report for October 2022. At the beginning of the month, I continued the unification of our desktop and Android patchset [0]. We had a few errors to correct, but now we use the same branch from tor-browser.git for all our platforms. One of the immediate consequences is that the Onion-Location header is now recognized on Android, too. Starting with Tor Browser 12.0a4, you can tell Tor Browser to prefer the .onion when possible [1]. After that, I worked on multi-lingual support for Tor Browser [2]. Starting with Tor Browser 12.0a4, we will ship only one package/installer for each architecture with all the languages we support. This change will make mirroring Tor Browser much easier, slightly decrease the time to build releases, and dramatically reduce the time to sign them. Of course, 36 additional languages increase the archive size. Therefore, we tried to remove files that were not actually used, like some images and fonts in the offline copy of the manual [3]. Eventually, we even reduced the final build sizes by some hundreds of kB! For the rest of the month, I have been resolving several smaller problems, such as the bookmarks window not working on macOS [4], problems that the new tor-launcher implementation caused, improvements for the multi-lingual UX, typos, and more. We are very excited about the coming 12.0, and we would love to hear any feedback from you on the latest alphas [5]]! Thanks, Pier [0] https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41308 [1] https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41394 [2] https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/17400 [3] https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/4… [4] https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/4… [5] https://www.torproject.org/download/alpha/

1 0

Anti-censorship team meeting notes, 2022-10-27
by meskio 27 Oct '22

27 Oct '22

Hey everyone! Here are our meeting logs: http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-10-27-15.59.html And our meeting pad: Anti-censorship work meeting pad -------------------------------- Next meeting: Thursday Nov 3 16:00 UTC Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC (channel is logged while meetings are in progress) == Goal of this meeting == Weekly check-in about the status of anti-censorship work at Tor. Coordinate collaboration between people/teams on anti-censorship at the Tor Project and Tor community. == Links to Useful documents == * Our anti-censorship roadmap: * Roadmap: https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards * The anti-censorship team's wiki page: * https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home * Past meeting notes can be found at: * https://lists.torproject.org/pipermail/tor-project/ * Tickets that need reviews: from sponsors we are working on: * All needs review tickets: * https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s… * Sponsor 28 * must-do tickets: https://gitlab.torproject.org/groups/tpo/-/milestones/10 * possible tickets: https://gitlab.torproject.org/groups/tpo/-/issues?scope=all&utf8=%E2%9C%93&… * Sponsor 96 * https://gitlab.torproject.org/groups/tpo/-/milestones/24 == Announcements == * Tor Browser 11.5.5/11.5.6 restore a working meek bridge and enable uTLS for Snowflake. == Discussion == * Blocking by TLS fingerprint in Iran * There is plenty of evidence now that there is blocking based on TLS fingerprint in Iran * It likely affects snowflake-client's connections to the broker and may be responsible for the sudden loss of traffic on 2022-10-04 * https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… * Likely to affect mainly Orbot, and not Tor Browser for desktop or Tor Browser for Android * Orbot has updated using uTLS and is now circumventing the block * Censorship analysis for UDP traffic between Iran and rest of Internet: 2022 Q4 src shell * https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… * https://github.com/net4people/bbs/issues/140 * shell is investigating it * builtin bridges and their usage * https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/102 * builtin bridges tend to work better than distributed ones * we want for now to keep using builtin bridges for the situations where they are useful * we need to improve the situation to where we feel comfortable to recomend settings bridges on those cases * investigating what is the churn rate of bridges * subscription model * quality of settings bridges * we are hitting the size limit for args in bridgelines * https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/4… * PT spec 2.0 was meant to solve this * https://github.com/Pluggable-Transports/Pluggable-Transports-spec * https://gitlab.torproject.org/tpo/core/tor/-/issues/21816 * https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/trac… * do we want to propose any changes to arti to have a future with longer args? * does arti want to keep the PT IPC model (i.e., will the future even use SOCKS the way it is used now)? * arti is implementing the SOCKS model for now * the bulk of snowflake bridge lines is the ice=stun:... list. each entry of the list has a "stun:" scheme and a port number. we could abbreviate the list by making the scheme and port number implicit if not specified. * meskio will create an issue to discuss proposals into the pt-spec to solve the issue * snowflake-02 bridge is now usable through the whole pipeline: just use `fingerprint=8838024498816A039FCBBAB14E6F40A0843051FA` in the bridge line. Do we want to encourage people to test this configuration? * https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… * it's okay to start testing it * we'll add this bridge into TB alpha * we have some concerns on the load on unrestricted proxies and the broker by tor connecting to both bridges at once if we configure both == Actions == == Interesting links == == Reading group == * We will discuss "" on * * Questions to ask and goals to have: * What aspects of the paper are questionable? * Are there immediate actions we can take based on this work? * Are there long-term actions we can take based on this work? * Is there future work that we want to call out in hopes that others will pick it up? == Updates == Name: This week: - What you worked on this week. Next week: - What you are planning to work on next week. Help with: - Something you need help with. cecylia (cohosh): last updated 2022-10-27 Last week: - wrote a guide for integrating PTs into tor browser - https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Tor-Browser-… - talked with brave about snowflake web extension - more work on conjure client - reached out about station reliability issues - responded to questions about prometheus metrics for the standalone proxy This week: - wrap up snowflake translation work (blocked) - followups to proxy fixes (blocked) - continue Conjure work - wrap up manifest v3 candidate Needs help with: dcf: 2022-10-27 Last week: - explained two different Client Hellos in the connections of certain uTLS fingerprints https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/obfs… - debugged a problem with snowflake-client failure with certain uTLS fingerprints and opened an issue https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - opined on merging the current draft of STATUS TYPE=version in the PT spec https://gitlab.torproject.org/tpo/core/torspec/-/merge_requests/63#note_284… - diagnosed the problem with snowflake bridge line length in Tor Browser 11.5.5 and helped with the emergency fix in 11.5.6 https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/4… https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_re… - with Linus, distributed snowflake-01's outbound traffic over multiple IP addresses, in an attempt to mitigate DDoS false detections https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - tested using the snowflake-02 bridge https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Next week: - disable non-WireGuard SSH access to snowflake-02 https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - migrate goptlib to gitlab https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/86#note_282… - try Conjure PT development version https://forum.torproject.net/t/tor-dev-introducing-a-conjure-pt-for-tor/4429 - break up snowflake-server performance improvements into separate merge requests https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - document recent performance optimizations in Snowflake bridge installation guide Help with: meskio: 2022-10-27 Last week: - deprecate dymcru builtin bridges (team#98) - checkout our experiment using obfs4 bridges in china and hong kong (team#99) - help outreachy applicants and review their merge requests - investigate why gettor had stopped replying emails (rdsys#129) - telegram gettor stopped working after the release (onionsproutsbot#45) - write sponsor 96 report - research why uTLS HelloFirefox_auto doesn't work agains azure (obfs4#40008) Next week: - gettor bugs (rdsys#133 rdsys#129) Shelikhoo: 2022-10-27 Last Week: - [Merge Request Awaiting] Add SOCKS5 forward proxy support to snowflake (snowflake!64) - [Discussion & Deployment] Rollout of Distributed Snowflake Support - [Coding & Deployment] Proposal: Centralized Probe Result Collector (anti-censorship/team#54) - [Research] HTTPT Planning https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/http… - [Research] Censorship analysis for UDP traffic between Iran and rest of Internet: 2022 Q4: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… - Generate Charts for presention: https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/92#note_283… (Continue) - Rollout distributed snowflake (include definition of secondary bridge on broker) https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Next Week: - [Research] WebTunnel Planning (Continue) - [Research] Fix vantage point summary upload in China - Release New version of Snowflake WebExt - [Research] Censorship analysis for UDP traffic between Iran and rest of Internet: 2022 Q4: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… (Continue) Itchy Onion: 2022-10-27 Last week: - Made some breakthrough. RACE Snowflake started to fail in 2.2.0 because the test load is increased by 5-fold and there is a 30 seconds timeout. So it takes snowflake too long to finish. So far I've observed high variance of flight time from snowflake proxy to server and the worst case it takes ~45 seconds to send. - Confirmed the issue was because of snowflake proxies running out in RACE This week: - Increase the number of snowflake proxies spawned in RACE and ran it against the CI test. I was able to pass every time (6 times in total). But since Tuesday I've been having trouble starting a new deployment with Rib. I've been in talk with TwoSix but so far nothing has helped. - Built and pushed snowflake plugin binary with the fix to 2.3.2:prod -- meskio | https://meskio.net/ -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- My contact info: https://meskio.net/crypto.txt -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan.

1 0

Anti-censorship team meeting notes, 2022-10-20
by meskio 20 Oct '22

20 Oct '22

Hey everyone! Here are our meeting logs: http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-10-20-15.58.html And our meeting pad: Anti-censorship work meeting pad -------------------------------- Next meeting: Thursday Oct 27 16:00 UTC Weekly meetings, every Thursday at 16:00 UTC, in #tor-meeting at OFTC (channel is logged while meetings are in progress) == Goal of this meeting == Weekly check-in about the status of anti-censorship work at Tor. Coordinate collaboration between people/teams on anti-censorship at the Tor Project and Tor community. == Links to Useful documents == * Our anti-censorship roadmap: * Roadmap: https://gitlab.torproject.org/groups/tpo/anti-censorship/-/boards * The anti-censorship team's wiki page: * https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/home * Past meeting notes can be found at: * https://lists.torproject.org/pipermail/tor-project/ * Tickets that need reviews: from sponsors we are working on: * All needs review tickets: * https://gitlab.torproject.org/groups/tpo/anti-censorship/-/merge_requests?s… * Sponsor 28 * must-do tickets: https://gitlab.torproject.org/groups/tpo/-/milestones/10 * possible tickets: https://gitlab.torproject.org/groups/tpo/-/issues?scope=all&utf8=%E2%9C%93&… * Sponsor 96 * https://gitlab.torproject.org/groups/tpo/-/milestones/24 == Announcements == * == Discussion == * Blocking by TLS fingerprint in Iran * There is plenty of evidence now that there is blocking based on TLS fingerprint in Iran * It likely affects snowflake-client's connections to the broker and may be responsible for the sudden loss of traffic on 2022-10-04 * https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… * Likely to affect mainly Orbot, and not Tor Browser for desktop or Tor Browser for Android * One variant of the native Go crypto/tls fingerprint is known to be blocked: the one that prioritizes non-AES ciphersuites and has a minimum TLS version of TLS 1.0 * Other versions of the fingerprint (AES ciphersuites prioritized, minimum version of TLS 1.2) are not currently blocked * https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/96#note_284… * Tor Browser for desktop: currently not blocked (uses AES priority ciphersuites) * Tor Browser for Android: currently not blocked (uses minimum TLS version of 1.2, because compiled by go1.18) * Orbot: available released versions are blocked * Orbot preparing a new release with utls enabled https://github.com/guardianproject/orbot/releases/tag/16.6.3-BETA-2-tor.0.4… * Would be nice if Orbot could use the Circumvention Settings API. That would likely take a little work because internally Orbot currently does not support custom bridge lines other than obfs4: https://github.com/net4people/bbs/issues/131#issuecomment-1272120924 * should snowflake use uTLS by default? * https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_re… * there are some concerns of active censors being able to test unimplemented TLS extensions claimed by uTLS, but haven't being seeing in the wild yet * an example is certificate compression https://datatracker.ietf.org/doc/rfc8879/ * yawning's utls fork dealt with that long ago; it's also now part of the main upstream utls in v1.1.2+: https://github.com/refraction-networking/utls/pull/95 * yes, we'll move to use uTLS * snowflake load after revert broker change src shell * https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… * the revert is not notizable in the graphs (broker polling and bridge bandwidth graphs) * we can go back to multi-bridge support * shell will revert the revert * snowflake broker secondary bridge info src shell * shell will enable snowflake-02 at the broker on Monday 2022-10-24 * Censorship analysis for UDP traffic between Iran and rest of Internet: 2022 Q4 src shell * https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… * https://github.com/net4people/bbs/issues/140 * shell is investigating it * obfs4proxy meek utls patches * https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/obfs… * meskio will merge this mr today and move forward to include it in the next TB version * keeping with HelloFirefox_auto so as not to change too much at once * meskio has been checking pcaps and testing compatibility with the meek and moat domain fronts * Testing new PTs * is conjure ready to be tested? not yet * will be included in TB alpha in early November, and will be wellcome testers * any kind of testers will be nice, might not be ready to really resist censorship * Sometimes in RACE it takes snowflake longer than 45 seconds to transfer a message. We want to make it less than 30. Does it depend on the availability/quality of snowflake proxies or is this something we have full control of programmatically? --- for next week --- * builtin bridges and their usage * https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/102 == Actions == == Interesting links == == Reading group == * We will discuss "" on * * Questions to ask and goals to have: * What aspects of the paper are questionable? * Are there immediate actions we can take based on this work? * Are there long-term actions we can take based on this work? * Is there future work that we want to call out in hopes that others will pick it up? == Updates == Name: This week: - What you worked on this week. Next week: - What you are planning to work on next week. Help with: - Something you need help with. cecylia (cohosh): last updated 2022-10-20 Last week: - more work on translations of webextension and snowflake.tpo - https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - completed integration of conjure into tor browser - https://gitlab.torproject.org/cohosh/tor-browser-build/-/commits/conjure - worked on standalone proxy issues - https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - set up a new meek server and handed it off to the new operator - http://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_req… This week: - wrap up snowflake translation work - followups to proxy fixes - continue Conjure work - wrap up manifest v3 candidate Needs help with: dcf: 2022-10-20 Last week: - thought more about loss of traffic at the snowflake broker and bridge and came ot the new working hypothesis that it *is* due to a block in Iran, that apparent effects in other countries are geoip errors, and that the mechanism of blocking is TLS fingerprinting using at least two identified features: ciphersuite order and minimum supported TLS version https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… https://github.com/net4people/bbs/issues/125#issuecomment-1284602875 - gave instructions for enabling uTLS in snowflake clients and solicited experience reports https://github.com/net4people/bbs/issues/131#issuecomment-1280391051 - helped with enabling utls by default for future snowflake releases https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/96#note_284… https://gitlab.torproject.org/tpo/anti-censorship/rdsys-admin/-/merge_reque… https://gitlab.torproject.org/tpo/anti-censorship/rdsys-admin/-/merge_reque… https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_re… - gave advice on meek bridge setup for https://bugs.torproject.org/tpo/anti-censorship/team/100 - explained a snowflake build failure on old CentOS https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - diagnosed an error caused by running an outdated snowflake proxy https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… - helped troubleshoot reported snowflake proxy inactivity https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Next week: - disable non-WireGuard SSH access to snowflake-02 - migrate goptlib to gitlab https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/86#note_282… - try Conjure PT development version https://forum.torproject.net/t/tor-dev-introducing-a-conjure-pt-for-tor/4429 - break up snowflake-server performance improvements into separate merge requests https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snow… Help with: meskio: 2022-10-20 Last week: - get uTLS back on obfs4proxy meek (obfs4#40008) - discuss the relation between IPtProxy and snowflake client API (snowflake#40218) - enable uTLS by default in snowflake (https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_re…) - make a callout for bridge operators to upgrade their version of obfs4proxy (obfs4#40008) - use randomized uTLS in snowflake in IR (rdsys-admin!8) - experiment with obfs4 bridges in china and hong kong (team#99) - review snowflake webextension patches on ephemeral ports (snowflake-webext!107) - do the process in debian to become a Debian Maintainer, so I can upload packages without a mentor Next week: - deprecate dymcru builtin bridges (team#98) - fix bridgedb https translations (bridgedb#40058) Shelikhoo: 2022-10-13 Last Week: - [Merge Request Awaiting] Add SOCKS5 forward proxy support to snowflake (snowflake!64) - [Discussion & Deployment] Rollout of Distributed Snowflake Support - [Coding & Deployment] Proposal: Centralized Probe Result Collector (anti-censorship/team#54) - [Research] HTTPT Planning https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/http… - [Research] Censorship analysis for UDP traffic between Iran and rest of Internet: 2022 Q4: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… Next Week: - [Research] WebTunnel Planning (Continue) - Generate Charts for presention: https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/92#note_283… (Continue) - [Research] Fix vantage point summary upload in China - Release New version of Snowflake WebExt - Rollout distributed snowflake (include definition of secondary bridge on broker) - [Research] Censorship analysis for UDP traffic between Iran and rest of Internet: 2022 Q4: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/iss… (Continue) Itchy Onion: 2022-10-20 Last week: - bump snowflake plugin to version 2.3.2 - trying to trace where the message dropping happens in the snowflake library used by RACE. (I've been back and forth on this one, but now I believe message dropping and unclosed TCP sockets are not the same issue. The CI tests that are failing doesn't send that many messages for a system resource issue to kick in. I've traced the message in the plugin code, and see they are all sent to the snowflake library code without dropping. So maybe an issue with the version of snowflake lib that's used in RACE) This week: - Made some breakthrough. RACE Snowflake started to fail in 2.2.0 because the test load is increased by 5-fold and there is a 30 seconds timeout. So it takes snowflake too long to finish. So far I've observed high variance of flight time from snowflake proxy to server and the worst case it takes ~45 seconds to send. -- meskio | https://meskio.net/ -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- My contact info: https://meskio.net/crypto.txt -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan.

1 0

OONI Monthly Report: September 2022
by Maria Xynou 19 Oct '22

19 Oct '22

Hello, OONI's September 2022 status report is shared below. *# OONI Monthly Report: September 2022* Throughout September 2022, the OONI team worked on the following sprints: * Sprint 73 (1st-11th September 2022) * Sprint 74 (12th-25th September 2022) * Sprint 75 (26th-30th September 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## Published report on new censorship events in Iran* On 25th September 2022, we published a new research report documenting the latest censorship events in Iran amid the ongoing protests. Read the report here: https://ooni.org/post/2022-iran-blocks-social-media-mahsa-amini-protests/ In this report, we share OONI network measurement findings from Iran on the blocking of WhatsApp, Instagram, Linkedin, Skype, Google Play Store, Apple App Store, and encrypted DNS (DNS over HTTPS). We also share IODA and Cloudflare Radar data on mobile network outages. On 27th September 2022, we published an updated version of the report (“Network outages” section) including contributions from our partner, IODA: https://ooni.org/post/2022-iran-blocks-social-media-mahsa-amini-protests/#n… This report received press coverage from a data journalism piece by Tagesspiegel (which also makes use of raw OONI data): https://interaktiv.tagesspiegel.de/lab/internet-sperren-im-iran-blackouts-g… *## Published report on TikTok block in Azerbaijan and Armenia* On 16th September 2022, in collaboration with Azerbaijan Internet Watch, we published a new research report which shares OONI data on the blocking of TikTok in Armenia and Azerbaijan during the border clashes. Read our report here: https://ooni.org/post/2022-azerbaijan-and-armenia-blocks-tiktok/ Overall, OONI data shows that Azerbaijan blocked TikTok by means of TLS level interference, while Armenia implemented both DNS level blocks and TLS level interference. *## Report for Azerbaijan Internet Watch* In support of our partner, Azerbaijan Internet Watch ( https://ooni.org/partners/azerbaijan-internet-watch/) we analyzed OONI measurements collected from Azerbaijan between May 2022 to September 2022. Based on this analysis, we produced a report with the key findings, which Azerbaijan Internet Watch published on their website: https://www.az-netwatch.org/wp-content/uploads/2022/10/2022-09-OONI-Report-… *## OONI Probe Mobile* During September 2022, we made progress on implementing a functionality that enables OONI Probe Mobile users to view all logs ( https://github.com/ooni/probe-android/pull/516, https://github.com/ooni/probe-ios/pull/499) This change stems from user requests. While OONI Probe already implements some form of logging, the app is not able to collect logs generated during the engine bootstrap. Crucially, these logs are key to debug and understand cases in which OONI services are blocked. The new design, instead, allows us to observe the engine bootstrap logs. Additionally, we added support for sharing the logs with OONI developers in an easier way, using the operating systems’ share functionality. We also made progress in implementing new Experimental settings under the Test Options settings of the OONI Probe apps ( https://github.com/ooni/probe-android/pull/543, https://github.com/ooni/probe-ios/pull/501) This will enable OONI Probe users to enable or disable experimental tests ( https://github.com/ooni/probe/issues/2266) As these settings would apply to both manual and automated runs, we added relevant copy for clarification (https://github.com/ooni/probe/issues/2268) This work addresses community feedback that the OONI Probe Experimental card includes long-running tests which users may want to have disabled as part of automated testing. We are currently performing QA of these new mobile functionalities and we are confident we will be able to include them in the next OONI Probe mobile releases. As part of our work on using Flutter for a shared codebase between the OONI Probe mobile and desktop apps, we wrote an internal guide to document the migration steps for all OONI Probe apps from Native Platforms to Flutter Application using OONI Test Descriptors. Through this document, we aim to capture all components of the system that need to be changed in the process in order to attain complete migration of Android, iOS and Desktop applications to the respective Flutter versions. *## OONI Run* We added support to miniooni for running an OONI Run v2 descriptor saved on the local disk, as opposed to fetching it from a remote URL: https://github.com/ooni/probe-cli/pull/966 *## OONI Probe Desktop* We released OONI Probe Desktop 3.8.0: https://github.com/ooni/probe-desktop/releases/tag/v3.8.0 This release uses the latest version of OONI Probe CLI and improvements to the Experimental card. *## OONI Probe localization* Thanks to the Localization Lab community, new OONI Probe strings were translated into the following languages: Farsi, Russian, Chinese, Turkish, Spanish, French, and Greek (https://explore.transifex.com/otf/ooniprobe/) *## Updated localization guidelines* We worked towards updating the OONI Probe localization guidelines, and writing new localization guidelines for OONI Explorer. This work is documented through the following pull request: https://github.com/ooni/translations/pull/26 The new localization guidelines are available through the following links: * OONI Probe localization guidelines: https://github.com/ooni/translations/blob/master/Guidelines%20for%20OONI%20… * OONI Explorer localization guidelines: https://github.com/ooni/translations/blob/master/Guidelines%20for%20OONI%20… *## Published updated versions of the OONI Probe user guides* In September 2022, we published updated versions (with extensive edits, and many new screenshots and steps) of the OONI Probe user guides: * User Guide for OONI Probe Mobile: https://ooni.org/support/ooni-probe-mobile * User Guide for OONI Probe Desktop: https://ooni.org/support/ooni-probe-desktop The specific updates can be viewed through this pull request: https://github.com/ooni/ooni.org/pull/1202 Now both the OONI Probe Mobile and Desktop user guides cover all the latest features up until version 3.7.0. We subsequently further edited the OONI Probe Mobile ( https://github.com/ooni/ooni.org/pull/1239) and Desktop ( https://github.com/ooni/ooni.org/pull/1240) user guides to clarify settings related to automated testing. *## Localizing the OONI Probe user guides* To enable community members around the world to use OONI Probe, we coordinated with the Localization Lab (https://www.localizationlab.org/) on creating new Transifex projects for the translation of the OONI Probe user guides. Community members can now localize the OONI Probe user guides via the following Transifex projects: * OONI Probe Mobile User Guide: https://www.transifex.com/otf/ooni-probe-mobile-user-guide/ * OONI Probe Desktop User Guide: https://www.transifex.com/otf/ooni-probe-desktop-user-guide/ *## Updated the OONI Data Policy* We made a minor update to OONI’s Data Policy ( https://github.com/ooni/ooni.org/pull/1221) reiterating our use of Sentry in the "Third party services" section. The updated version of OONI’s Data Policy is available here: https://ooni.org/about/data-policy *## Expanding censorship measurement methodologies### uTLS-light* We wrote a prototype fork (“uTLS-light”) of Golang’s crypto/tls library that is able to parrot the ClientHello of Chrome browser. This prototype is specifically designed to make merging from upstream as simple and as straightforward as possible. The main drawback of this prototype is that, compared to similar efforts such as uTLS, it is not configurable, as users of uTLS-light always use the Chrome browser’s configuration (which is what we need as OONI to perform measurements). Our work on developing uTLS-light is available here: https://github.com/hellais/utls-light *### Improving Web Connectivity experiment* We continued testing Web Connectivity v0.5 with support from users in our community. Based on testing and community feedback, we made a series of improvements: * https://github.com/ooni/probe-cli/pull/933 * https://github.com/ooni/probe-cli/pull/935 * https://github.com/ooni/probe-cli/pull/943 * https://github.com/ooni/probe-cli/pull/948 * https://github.com/ooni/probe-cli/pull/951 * https://github.com/ooni/probe-cli/pull/952 * https://github.com/ooni/probe-cli/pull/953 * https://github.com/ooni/probe-cli/pull/957 * https://github.com/ooni/probe-cli/pull/958 * https://github.com/ooni/probe-cli/pull/959 * https://github.com/ooni/probe-cli/pull/960 * https://github.com/ooni/probe-cli/pull/961 * https://github.com/ooni/probe-cli/pull/965 The main focus of these changes was to improve the data quality and correctly flag edge cases to minimize false positives and detect cases where websites are down. In the meanwhile, we spent time testing the test helper improvements to support Web Connectivity v0.5. We performed A/B testing for a few days and then decided to switch all users to the new test helper version. We also improved our measurement scrubbing procedure, by extending the scrubbing to all the fields of a measurement, as opposed to limiting it to the test keys (https://github.com/ooni/probe-cli/pull/956) All this work culminated in the release of OONI Probe CLI v3.16.3: https://github.com/ooni/probe-cli/releases/tag/v3.16.3 *### Experiments developed by Google Summer of Code (GSoC) student* We merged the TLS middlebox experiment ( https://github.com/ooni/probe-cli/pull/817) and port filtering experiment ( https://github.com/ooni/probe-cli/pull/891) contributed by Mehul Gulati, as part of his Google Summer of Code (GSoC) with OONI. We wrapped up the GSoC with a comprehensive recap of the work that was done in the following ticket: https://github.com/ooni/probe/issues/2272 *## OONI Explorer* In September 2022, we completed most development work for the new domain-centric pages of OONI Explorer ( https://github.com/ooni/explorer/pull/762) We refactored the OONI Explorer Search tool to move measurement requests to the client side ( https://github.com/ooni/explorer/pull/799) We continued to make progress towards building the user feedback reporting system, and enabling localization on OONI Explorer. We also replaced Babel with SWC ( https://github.com/ooni/explorer/pull/804) fixed builds, and reviewed pull requests. *## OONI backend* In September 2022, we worked on the following main backend activities: * Improvements to API rate limiting (https://github.com/ooni/api/pull/307) * Added support for filtering by IP address in the aggregation API ( https://github.com/ooni/api/pull/308) * Added database query prioritization in the API ( https://github.com/ooni/api/pull/304) and fastpath ( https://github.com/ooni/pipeline/pull/397) * Improved upon the backend prototype for the user feedback reporting system (https://github.com/ooni/api/pull/302) * Added bot mitigations in the API using robots.txt; * Added API metrics and dashboards; * Investigated test failures; * Implemented an initial scoring table for OpenVPN and obfs4 ( https://github.com/ooni/pipeline/pull/399) *## Published user guide for Test Lists Editor* In September 2022, we published a user guide for our new Test Lists Editor ( https://test-lists.ooni.org/) Our user guide is available here: https://ooni.org/support/test-lists-editor This user guide provides step-by-step instructions (with screenshots) on how to use our new Test Lists Editor for contributing to the lists of websites tested by OONI Probe users around the world. *## Published test list guide for GitHub users* Previously, we used to include instructions for updating the Citizen Lab test lists (https://github.com/citizenlab/test-lists/tree/master/lists) via GitHub in our general test lists documentation. In September 2022, we created a separate guide for GitHub users with updated instructions. Our user guide for updating test lists via GitHub is available here: https://ooni.org/support/github-test-lists *## Published updated test list documentation* We updated our general test list documentation: https://ooni.org/get-involved/contribute-test-lists This documentation explains what test lists are, why and how to contribute to test lists (while linking to more detailed guides), and how to approach test list research. *## Test list updates* In September 2022, we contributed updates to the following test lists: * Global (https://github.com/citizenlab/test-lists/pull/1107, https://github.com/citizenlab/test-lists/pull/1128; https://github.com/citizenlab/test-lists/pull/1114) * Azerbaijan (https://github.com/citizenlab/test-lists/pull/1111) We reviewed test list updates provided by Netalitica researchers and, on their behalf, opened pull requests for the following test lists: * Hong Kong (https://github.com/citizenlab/test-lists/pull/1115) * Poland (https://github.com/citizenlab/test-lists/pull/1116) We opened a pull request with updates to the test list for Kyrgyzstan, provided by the School of Peacemaking ( https://github.com/citizenlab/test-lists/pull/1124) We also reviewed and merged multiple pull requests ( https://github.com/citizenlab/test-lists/pulls?q=is%3Apr+is%3Aclosed) submitted by community members (via both GitHub and our new Test Lists Editor). *## Creating a new OONI Outreach Kit* To better meet community needs (particularly around OONI community engagement), we worked towards creating multiple new materials and resources that will eventually get published as part of a new OONI Outreach Kit. These resources include OONI-related brochures, fact sheets, workshop slides, and screencasts. *## Launch of Advocacy Assembly OONI training course* On 30th September 2022, Small Media’s Advocacy Assembly launched an online OONI training course ( https://advocacyassembly.org/en/courses/63/#/chapter/1/lesson/1) We had previously collaborated with Advocacy Assembly on creating the content of the OONI training course, while they provided amazing support for everything related to editing, designing, and translating the course. Prior to the launch, Advocacy Assembly published a blog post which includes an interview with the OONI team about the course: https://advocacyassembly.org/en/news/228/ *## Improving OONI’s Code of Conduct* Based on feedback from community meetings and discussions, we re-wrote OONI’s Code of Conduct (in collaboration with OONI’s new Incident Response Committee). The updated version of OONI’s Code of Conduct is available here: https://deploy-preview-1245--ooni.netlify.app/get-involved/code-of-conduct/ We shared the updated version with community members during the September 2022 OONI Community Meeting, with the goal of soliciting further community review and feedback. Based on the latest community feedback, we plan to make further improvements to the Code of Conduct before it is published. *## Ford Foundation Retreat* Between 19th-22nd September 2022, OONI’s Maria attended the Ford Foundation Retreat in Atlanta, USA. As part of her participation at the retreat, Maria facilitated a session with the goal of collecting feedback on how internet measurement projects can better meet advocacy needs. *## Community activities### OONI hackathon at DEMHACK 2022* On 24th and 25th September 2022, OONI’s Elizaveta organized an OONI hackathon as part of Roskomsvoboda’s DEMHACK 2022 hackathon ( https://demhack.ru/) in Russia. Hackathon participants were invited to work on the following OONI hackathon challenges (https://demhack.ru/tasks) * Updating the lists of websites (“test lists”) measured for censorship; * Developing a tool for the automatic clean-up of test lists (e.g. removing expired domains); * Analyzing OONI data to investigate internet censorship. As part of the hackathon, OONI’s Maria provided a live demo on using OONI’s Measurement Aggregation Toolkit (https://explorer.ooni.org/chart/mat) encouraging participants to use the tool as a starting point for further analyzing OONI data. *### OONI workshops at FIFAfrica 2022* Between 26th-29th September 2022, OONI’s Arturo traveled to Lusaka, Zambia, to participate in the Forum on Internet Freedom in Africa (FIFAfrica) 2022 conference organized by CIPESA – one of Africa’s largest digital rights events (https://cipesa.org/fifafrica/) As part of his participation (https://cipesa.org/fifafrica/agenda/) Arturo: * Facilitated an OONI workshop (“Investigating Internet censorship in Africa through OONI tools and open data”); * Co-facilitated a workshop with Access Now (“Strengthening Advocacy to Fight Internet Shutdowns in Africa”). *### OONI Community Meeting* On 27th September 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/) during which we discussed the following topics: 1) Improving OONI’s Code of Conduct: Request for community feedback (based on the updated version) 2) Reconsidering test list policies around expired or parked domains *## Userbase* In September 2022, 60,831,677 OONI Probe measurements were collected from 2,765 AS networks in 170 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0

OONI Monthly Report: August 2022
by Maria Xynou 19 Oct '22

19 Oct '22

Hello, OONI's August 2022 status report is shared below. *# OONI Monthly Report: August 2022* Throughout August 2022, the OONI team worked on the following sprints: * Sprint 71 (1st-14th August 2022) * Sprint 72 (15th-28th August 2022) Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni Highlights are shared in this report below. *## New OONI Team member* In August 2022, Elizaveta Yachmeneva joined the OONI team as our new Community Coordinator! Elizaveta will support the global OONI community and help expand our community work. *## OONI Probe Mobile* In August 2022, we released OONI Probe Android 3.7.1 with bug fixes and improvements (https://github.com/ooni/probe-android/releases/tag/v3.7.1) We continued to work on a prototype of the OONI Probe app using Flutter, which will allow us to have a shared codebase between the OONI Probe mobile and desktop apps. Specifically, we experimented with connecting Flutter with the OONI Probe engine (https://github.com/ooni/probe/issues/2197) In particular, we experimented with three distinct approaches ( https://github.com/ooni/probe-cli/pull/849, https://github.com/ooni/probe-cli/pull/850, https://github.com/ooni/probe-cli/pull/851) to expose the OONI Probe engine to code written in Dart (which is the language used by Flutter). Once that work was done, we adapted our app prototypes to use the recent OONI Probe engine changes and generate PoC apps for mobile and desktop using Flutter ( https://github.com/ooni/probe-cli/pull/880) We also looked into Sentry issues (https://github.com/ooni/probe/issues/2187 , https://github.com/ooni/probe/issues/2186) Google Play errors, and failing CI on OONI Probe Android. *## OONI Run* Following the implementation of OONI Run v2 for miniooni users (in July 2022), we wrote a design document which describes the miniooni OONI Run v2 functionality (https://github.com/ooni/probe-cli/pull/916) *## OONI Probe Desktop* In response to community feedback, we fixed the summary table for Tor measurements on OONI Probe Desktop ( https://github.com/ooni/probe/issues/2209) and we looked into fixing inconsistencies when displaying results ( https://github.com/ooni/probe/issues/2210) *## Worked on updating the OONI Probe user guides* Throughout August 2022, we worked on extensive updates to the OONI Probe mobile and desktop user guides to ensure that they include information and screenshots for features based on the latest releases. This work is documented through the following pull request: https://github.com/ooni/ooni.org/pull/1202 *## Test Lists Editor* Following the launch of OONI’s Test Lists Editor in July 2022 ( https://ooni.org/post/2022-test-lists-editor/) we continued to work on improvements to the platform. Specifically, we improved the UI of the “Add new URL” section (https://github.com/ooni/test-lists-ui/issues/61) we fixed an issue with repeated background requests to account metadata endpoint (https://github.com/ooni/test-lists-ui/issues/62) and we adapted the login request to work with the new API changes. *## Expanding censorship measurement methodologies* In previous months, we created Websteps ( https://github.com/bassosimone/websteps-illustrated/) which includes significant improvements to how we measure website blocking. But shipping Websteps would create a wide range of complexities for interdependent OONI software components and data structures. We therefore decided that rather than shipping Websteps, we will instead improve upon our Web Connectivity experiment based on Websteps. To this end, we drafted and implemented a plan ( https://github.com/ooni/probe/issues/2237) for improving upon our Web Connectivity experiment by incrementally integrating selected features and experiments from our Websteps prototype ( https://github.com/bassosimone/websteps-illustrated/) into Web Connectivity. During August 2022, we created a new version of Web Connectivity (“Web Connectivity v0.5”) based on the “step-by-step” style ( https://github.com/ooni/probe-cli/pull/814) which aims to incrementally integrate features from Websteps. The initial implementation of Web Connectivity v0.5 was introduced through the following pull request: https://github.com/ooni/probe-cli/pull/882 Subsequent changes focused on collecting late, duplicate DNS replies, which often provide a clear indication of censorship ( https://github.com/ooni/probe-cli/pull/878, https://github.com/ooni/probe-cli/pull/883) as well as on improving the Web Connectivity test helper to support v0.5 measurements ( https://github.com/ooni/probe-cli/compare/c3964e43b33d36b7cba371e9906d5ed0b…) One of these patches (https://github.com/ooni/probe-cli/pull/878) was contributed by Mehul Gulati, one of our Google Summer of Code (GSoC) students. We improved the code used by Web Connectivity v0.5 to simplify the task of splitting measurements into observations ( https://github.com/ooni/probe-cli/pull/879, https://github.com/ooni/probe-cli/pull/873) thus enabling efforts aimed at automating the detection and characterization of more forms of website censorship. We also improved DNS measurement code to collect CNAMEs from DNS responses (https://github.com/ooni/probe-cli/pull/875, https://github.com/ooni/probe-cli/pull/876, https://github.com/ooni/probe-cli/pull/877) To write this new version of Web Connectivity in a more automated way, we specified and drafted a code generation tool called “boilerplate” ( https://github.com/ooni/probe-cli/pull/853) This tool should simplify maintaining code used for performing specific measurements using the “step-by-step” style. As part of research on censorship events in Indonesia, we added support for running measurements in parallel (https://github.com/ooni/probe-cli/pull/848) which helps with running our new experiment (“DNS Scan”) for scanning open DNS resolvers to detect blocking. ( https://github.com/ooni/probe-cli/pull/847) Lastly, the work on allowing Dart clients to use the OONI Probe engine (described above) highlighted the need to cleanup OONI Probe CLI build scripts. To this end, we refactored the Makefile to defer most of its work to support scripts (https://github.com/ooni/probe/issues/2217, https://github.com/ooni/probe/issues/2218) *## OONI Explorer### Domain-centric pages* Throughout August 2022, we made progress on developing domain-centric pages for OONI Explorer (https://github.com/ooni/explorer/pull/762) We also implemented a new API endpoint for returning domain metadata that we can display in the new domain-centric pages ( https://github.com/ooni/api/pull/300) *### OONI accounts* We wrote an internal design document which outlines our plans for implementing authentication to OONI services for OONI accounts. As part of this, we documented the primary use cases (user stories) for these accounts and the implementation details. Generally, the OONI accounts are being created to support the following use cases: * Presenting website measurements to account users; * Crowdsourcing false positives and fingerprints for blocking detection; * Enabling OONI Probe users to view their own measurements; * Monitoring the success of measurement campaigns; * Creating and monitoring OONI Run links; * Enabling users to request higher testing priority for specific URLs; * URL submission and management. Based on our design document, we implemented a modal for the login system and worked on relevant code refactoring. We also added relevant backend support for authentication (https://github.com/ooni/api/pull/298) *### User feedback reporting system* We created a design document for the (upcoming) OONI Explorer user feedback reporting system, which will enable users to share feedback via OONI Explorer to report false positives, or share other information that is useful for evaluating measurements (such as information about blocked websites). Based on this, we implemented logic functionality in OONI Explorer as a base for our (upcoming) user feedback reporting system ( https://github.com/ooni/explorer/issues/774) We also implemented an initial prototype in the backend which enables registered users to submit, view, and edit feedback for measurements ( https://github.com/ooni/api/pull/302) *### Other OONI Explorer improvements* We continued to work towards adding localization support to OONI Explorer by updating Transifex strings, coordinating with the Localization Lab, and fixing relevant issues (https://github.com/ooni/explorer/pull/778) We also improved upon OONI Explorer by removing links to external pages on measurement pages (https://github.com/ooni/explorer/issues/783) bug fixing (https://github.com/ooni/explorer/pull/777) including failed measurements in the MAT tooltip (https://github.com/ooni/explorer/issues/779) and by upgrading dependencies. *## OONI backend* Notably, we switched all our infrastructure over to the new backend host and we completed the migration to the Clickhouse database ( https://github.com/ooni/pipeline/pull/380) We continued to test and make improvements to the new ASN-based URL prioritization system, we monitored the availability of our test helpers and improved internal dashboards, we fixed measurement bugs, we improved upon the API uploader, and we continued to improve upon JSONL tables ( https://github.com/ooni/api/pull/297/) We handled an incident that involved an attack on our infrastructure. Specifically, we added IP-address based filtering as a temporary mitigation, then more strict filtering on inputs on API calls ( https://github.com/ooni/api/pull/301) and also extended API caching to more entry points. We fixed a backend bug (https://github.com/ooni/pipeline/pull/396) that was impacting the correct annotation of measurements ( https://github.com/ooni/explorer/issues/764) We added DNS fingerprints to our database to automatically detect and confirm more cases of website blocking in Iran (https://github.com/ooni/pipeline/pull/390, https://github.com/ooni/backend/issues/543) We also created a diagram which shows the process through which measurements are processed and published (https://github.com/ooni/pipeline/pull/391) *## Advancing OONI data analysis capabilities* We have been working on a set of tools (https://github.com/ooni/data) that make it easier for us and third parties to analyze OONI data as part of research and to expand the capabilities of our data processing pipeline. In particular, we wrote parsers for several common data formats used in OONI measurements and evaluated the performance ( https://github.com/hellais/oonidata-bench) of several approaches to building python data class objects based on these definitions. One of the benefits of having a well-defined schema for all OONI probe tests is that people interested in analyzing OONI data can benefit from type hints and code completion for accessing all the necessary attributes of a particular test, without needing to refer to OONI specifications. Moreover, it takes care of handling some inconsistencies across various revisions of the data generated by older probes so that the access patterns are as smooth as possible. Lastly, the data is being validated to be conformant with the specification, so it’s easier to identify issues affecting data quality. Other aspects worked on include adding the ability to break up some common types of tests into distinct observations and writing them to either a Clickhouse database backend or a flat CSV file on disk. We also started outlining a system for taking these observations and generating verdicts related to censorship around the world. Since we are also using it heavily internally for doing research with OONI data, our goal is to eventually consolidate all the knowledge and patterns we develop into it, so that it can be beneficial to the wider public. We also created a new GitHub repository for the collection of blocking fingerprints (https://github.com/ooni/blocking-fingerprints) with the goal of enhancing our capabilities to automatically confirm more website blocks. *## Data requests* In response to community requests, we shared relevant OONI data (and interpretation) on censorship events. Specifically, we responded to Freedom on the Net’s data requests, providing relevant OONI data and information pertaining to 11 censorship cases. We also coordinated with community members in the Philippines on interpreting relevant OONI data, and we continued to coordinate with members of the #KeepItOn campaign on monitoring blocks. *## Creating a new OONI Outreach Kit* To better meet community needs (particularly around OONI community engagement), we started working towards creating new materials and resources that will eventually get published as part of a new OONI Outreach Kit. In August 2022, we worked on several OONI fact sheets and related brochures. *## OONI blog post on encrypted-DNS censorship published by RIPE Labs* On 4th August 2022, RIPE Labs published our blog post where we discuss our research on encrypted-DNS censorship. This blog post is available here: https://labs.ripe.net/author/bassosimone/measuring-encrypted-dns-censorship… *## Test list updates* In August 2022, we contributed updates to the following test lists: * Philippines (https://github.com/citizenlab/test-lists/pull/1055) * Kenya (https://github.com/citizenlab/test-lists/pull/1051) * Russia (https://github.com/citizenlab/test-lists/pull/1032) * Tajikistan (https://github.com/citizenlab/test-lists/pull/1034) * Georgia (https://github.com/citizenlab/test-lists/pull/1038) * Lebanon and Global (https://github.com/citizenlab/test-lists/pull/1059) *## Started planning the OONI Team Meeting for 2022* In August 2022, we started logistical preparations for organizing the first (since the start of the pandemic) in-person OONI Team Meeting, which will take place between 2nd-4th November 2022 in Rome, Italy. This meeting will bring all OONI team members and OTF research fellows hosted by OONI together for a 3-day event to plan strategic priorities and roadmaps. *## Created a new OONI Code of Conduct Incident Response Committee* In August 2022, we created a new Incident Response Committee with the goal of improving OONI’s Code of Conduct (CoC) and addressing any CoC violations when they occur. We are excited to share that our new CoC Incident Response Committee comprises of the following members: 1) Elizaveta Yachmeneva (OONI) 2) Amanda Meng (IODA, Georgia Tech) 3) Felicia Anthonio (Access Now) 4) Kelly Koh (Sinar Project) 5) Maria Xynou (OONI) 6) Gurshabad Grover (OTF Information Controls Fellow) 7) Federico Ceratto (OONI) Throughout August 2022, we organized several meetings to discuss the goals and responsibilities of the committee, how to improve upon OONI’s Code of Conduct (CoC), and how to handle CoC violation reports. We also worked towards re-writing OONI’s Code of Conduct based on initial community feedback collected during the June 2022 OONI Community Meeting, as well as feedback shared by Code of Conduct experts in our field. *## Community activities### OONI training session for election observers* On 4th August 2022, OONI’s Maria facilitated an OONI training session for election observers. This training introduced participants to OONI Probe and OONI data, particularly with the goal of monitoring emerging censorship events during elections around the world. *### OONI training session for researchers in Egypt* On 10th August 2022, OONI’s Maria facilitated a 3-hour OONI training session for researchers in Egypt. This workshop provided an introduction to internet censorship, as well as to OONI tools and data for measuring internet censorship. *## Userbase* In August 2022, 60,352,386 OONI Probe measurements were collected from 2,798 ASNs in 173 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/ ~ OONI team.

1 0