tor-project

tor-project@lists.torproject.org

17 participants
2326 discussions

OONI Monthly Report: February 2020
by Maria Xynou 09 Mar '20

09 Mar '20

Hello, Below is OONI's progress report from February 2020. ## OONI Probe support for running circumvention tool tests During February 2020, we added support to OONI Probe for testing the availability of the Tor, obfs4proxy, and Psiphon circumvention tools. These tests have been integrated into the OONI Probe engine (https://github.com/ooni/probe-engine) and are available to run via the OONI Probe desktop app. Below are the specifications for each test: * Tor test: https://github.com/ooni/spec/blob/master/nettests/ts-023-tor.md * Psiphon test: https://github.com/ooni/spec/blob/master/nettests/ts-015-psiphon.md As part of our work on integrating these tests into OONI Probe, we: * Implemented the Tor test UI in the OONI Probe desktop app: https://github.com/ooni/probe/issues/975 * Improved the UX of the Tor test results in the OONI Probe desktop app: https://github.com/ooni/probe/issues/990 * Created an animation that appears when OONI Probe users run circumvention tool tests: https://github.com/ooni/probe/issues/1029 * Configured the OONI Probe CLI (which the desktop app relies on) to submit measurements, even if an experiment fails: https://github.com/ooni/probe-engine/issues/316 ## Analysis of circumvention tool test data and presentation in OONI Explorer We added support for analyzing the new OONI Probe circumvention tool test results, and we integrated them into OONI Explorer and the OONI API. In February 2020, we worked on presenting the measurements in OONI Explorer for the following new OONI Probe tests: * Psiphon: https://github.com/ooni/explorer/issues/332 * Tor: https://github.com/ooni/explorer/issues/333 We also investigated how the batch pipeline deals with the new OONI Probe experiments for Tor and Psiphon: https://github.com/ooni/backend/issues/301 ## Making OONI Probe reporting logic more resilient to censorship To collect OONI measurements from around the world, OONI Probe clients need to upload measurements to OONI servers. As some censors may seek to block connections to OONI servers (in an attempt to prevent the collection of measurements), we have adopted various strategies over the years (such as using onion services), but they have presented various limitations and challenges. As part of our ongoing efforts to make OONI Probe’s reporting logic more resilient to censorship, we researched whether requesting OONI Probe users in censored environments to submit their measurements via instant messaging app messages would be a feasible and desirable option. This is documented through the following ticket: https://github.com/ooni/probe/issues/976 We also explored the possibility of enabling OONI Probe clients to submit their measurements via email, as documented through the following ticket: https://github.com/ooni/probe/issues/992 ## OONI Probe orchestration logic for circumvention tool testing As part of our work on developing OONI Probe orchestration logic that is specific to circumvention tool testing, we updated the canonical location of Tor test targets: https://github.com/ooni/backend/issues/299 These Tor testing targets are then given out to OONI Probe clients by using the OONI Probe orchestration system. ## OONI Probe desktop app We released the 4th release candidate of the OONI Probe desktop app 3.0.0: https://github.com/ooni/probe-desktop/releases/tag/v3.0.0-rc.4 Throughout February 2020, we researched how to best run OONI Probe automatically on desktop (https://github.com/ooni/probe/issues/1011), and we made a series of improvements to the OONI Probe desktop app. https://github.com/ooni/probe-desktop/pull/97 https://github.com/ooni/probe-desktop/pull/98 https://github.com/ooni/probe-desktop/pull/99 https://github.com/ooni/probe-desktop/pull/100 https://github.com/ooni/probe-desktop/pull/101 https://github.com/ooni/probe-desktop/pull/102 https://github.com/ooni/probe-desktop/pull/103 https://github.com/ooni/probe-desktop/pull/104 https://github.com/ooni/probe-desktop/pull/105 https://github.com/ooni/probe-desktop/pull/106 https://github.com/ooni/probe-desktop/pull/107 We also designed mock-ups for the UI elements of the OONI Probe desktop app: https://github.com/ooni/probe/issues/1009#issuecomment-590459771 https://github.com/ooni/probe/issues/966#issuecomment-585128461 https://github.com/ooni/probe/issues/1006#issuecomment-590354947 ## OONI Probe Command Line Interface (CLI) In February 2020, we made 2 OONI Probe CLI releases: * OONI Probe CLI v3.0.0-rc.9: https://github.com/ooni/probe-cli/releases/tag/v3.0.0-rc.9 * OONI Probe CLI v3.0.0-rc.10: https://github.com/ooni/probe-cli/releases/tag/v3.0.0-rc.10 These releases enable users to stop running experiments, specify test list categories, and they address a critical bug in how measurements are stored on disk. Throughout the month, we made a series of improvements to the OONI Probe CLI: https://github.com/ooni/probe-cli/pull/104 https://github.com/ooni/probe-cli/pull/106 https://github.com/ooni/probe-cli/pull/107 https://github.com/ooni/probe-cli/pull/108 https://github.com/ooni/probe-cli/pull/109 https://github.com/ooni/probe-cli/pull/110 https://github.com/ooni/probe-cli/pull/111 https://github.com/ooni/probe-cli/pull/112 https://github.com/ooni/probe-cli/pull/113 https://github.com/ooni/probe-cli/pull/114 https://github.com/ooni/probe-cli/pull/115 https://github.com/ooni/probe-cli/pull/116 ## OONI Probe mobile app Throughout February 2020, we made a series of improvements to the OONI Probe mobile app. On Android: https://github.com/ooni/probe-android/pull/289 https://github.com/ooni/probe-android/pull/290 https://github.com/ooni/probe-android/pull/291 https://github.com/ooni/probe-android/pull/292 https://github.com/ooni/probe-android/pull/293 https://github.com/ooni/probe-android/pull/295 https://github.com/ooni/probe-android/pull/296 https://github.com/ooni/probe-android/pull/297 On iOS: https://github.com/ooni/probe-ios/pull/336 https://github.com/ooni/probe-ios/pull/337 https://github.com/ooni/probe-ios/pull/338 https://github.com/ooni/probe-ios/pull/339 https://github.com/ooni/probe-ios/pull/340 https://github.com/ooni/probe-ios/pull/341 These improvements were in preparation for the release of OONI Probe 2.3.0, including a fix to the OONI Probe WhatsApp test which was presenting false positives following recent changes to WhatsApp’s infrastructure. ## Research on push notifications We researched how to best acquire push notification permissions from OONI Probe mobile app users (see https://github.com/ooni/probe/issues/813). As part of this, we wrote a document which outlines the problem that we are trying to solve, the core questions that we need to answer, and it proposes a potential implementation solution. This document is available here: https://docs.google.com/document/d/1x1jMjMq4-e8Xb4USegBSUV6V_cpNUWb2GwfBE5j… We also deployed a test backend for countly (https://github.com/ooni/sysadmin/commit/d69f9975a16fbb7c4b7a29ec0a0be28c135…) and did some testing to check how well it could fit our use case as a push notification backend. ## OONI Probe measurement engine We implemented padding for DNS over HTTPS (DoH) and DNS over TLS (DoT) in the OONI Probe measurement engine (https://github.com/ooni/probe-engine/issues/285). We also explored test helper options for the OONI Probe SNI blocking experiment (https://github.com/ooni/probe-engine/issues/303). OONI Probe has relied on the MaxMind database (https://www.maxmind.com/) for IP geolocation, but they recently changed their license. As part of our efforts to manage this change (https://github.com/ooni/probe-engine/issues/269), we explored the possibility of generating our own ASN database that would be compatible with the MaxMindDB (https://github.com/ooni/probe-engine/issues/336). We made a series of improvements to the OONI Probe engine throughout the month, as documented through the following pull requests. https://github.com/ooni/probe-engine/pull/313 https://github.com/ooni/probe-engine/pull/318 https://github.com/ooni/probe-engine/pull/319 https://github.com/ooni/probe-engine/pull/322 https://github.com/ooni/probe-engine/pull/323 https://github.com/ooni/probe-engine/pull/324 https://github.com/ooni/probe-engine/pull/329 https://github.com/ooni/probe-engine/pull/333 https://github.com/ooni/probe-engine/pull/342 https://github.com/ooni/probe-engine/pull/343 https://github.com/ooni/probe-engine/pull/344 https://github.com/ooni/probe-engine/pull/345 https://github.com/ooni/probe-engine/pull/346 https://github.com/ooni/probe-engine/pull/347 https://github.com/ooni/probe-engine/pull/354 https://github.com/ooni/probe-engine/pull/360 https://github.com/ooni/probe-engine/pull/361 https://github.com/ooni/probe-engine/pull/364 https://github.com/ooni/probe-engine/pull/368 https://github.com/ooni/probe-engine/pull/369 https://github.com/ooni/probe-engine/pull/370 https://github.com/ooni/probe-engine/pull/373 https://github.com/ooni/probe-engine/pull/374 https://github.com/ooni/probe-engine/pull/375 ## Making the OONI Probe apps rely entirely on the OONI Probe golang engine We made considerable progress on making the OONI Probe mobile and desktop apps rely entirely on the golang based probe-engine. To this end, we: * Investigated the use of certifi/gocertifi to support shipping SSL certificates using Go: https://github.com/ooni/probe-engine/issues/296 * Replaced most of the HTTP code in measurement-kit with the golang based ooni/netx: https://github.com/ooni/probe-engine/issues/302 * Implemented a golang-based replacement for the measurement-kit method called MKAsyncTask: https://github.com/ooni/probe-engine/issues/339 * Ensured we never emit empty fields in the golang code: https://github.com/ooni/probe-engine/issues/348 ## OONI Explorer Throughout February 2020, we made a series of improvements to OONI Explorer based on GitHub tickets opened by community members and our team over the last months. Our work on improving OONI Explorer is available through the following pull requests. https://github.com/ooni/explorer/pull/387 https://github.com/ooni/explorer/pull/393 https://github.com/ooni/explorer/pull/394 https://github.com/ooni/explorer/pull/395 https://github.com/ooni/explorer/pull/397 https://github.com/ooni/explorer/pull/398 https://github.com/ooni/explorer/pull/401 https://github.com/ooni/explorer/pull/404 https://github.com/ooni/explorer/pull/405 https://github.com/ooni/explorer/pull/408 https://github.com/ooni/explorer/pull/409 https://github.com/ooni/explorer/pull/411 https://github.com/ooni/explorer/pull/412 https://github.com/ooni/explorer/pull/415 We also started implementing a strategy for performing end-to-end testing of OONI Explorer to ensure that everything works as expected. See: https://github.com/ooni/explorer/issues/159 & https://github.com/ooni/explorer/pull/417. This is a big step in the direction of ensuring better quality of the OONI software ecosystem end-to-end. ## OONI API In order to improve the performance of the OONI API and of the services that rely on it (such as OONI Explorer), we investigated how to better log long-running queries (https://github.com/ooni/backend/issues/325). As part of this, we created an internal dashboard that generates metrics on query runtime and enables us to better track heavy queries (https://github.com/ooni/api/pull/162). We also implemented limits to the database queries (https://github.com/ooni/api/pull/165), and made a series of other improvements, as documented through the following pull requests. https://github.com/ooni/api/pull/163 https://github.com/ooni/api/pull/164 https://github.com/ooni/api/pull/166 https://github.com/ooni/api/pull/167 https://github.com/ooni/api/pull/168 https://github.com/ooni/api/pull/169 https://github.com/ooni/api/pull/170 https://github.com/ooni/api/pull/171 https://github.com/ooni/api/pull/172 ## OONI data processing pipeline In order for OONI Explorer to present “confirmed blocked” cases, we need to add the fingerprints of block pages to the OONI database. Thanks to the support of community members -- and, especially, Vasilis Ververis from the Magma project -- many GitHub tickets have been filed providing block page fingerprints. Throughout February 2020, we completed the task of adding all the remaining fingerprints to the OONI database: https://github.com/ooni/backend/issues/281. This improves the analytics needed to extract per-website metrics. We implemented a watchdog system that monitors the amount of rows generated in our fast-path pipeline and compares them with the measurement table. This enables us to see if we’re missing any measurements and to otherwise better understand issues related to OONI data processing. We also implemented an internal dashboard that generates charts and alarms based on deltas (https://github.com/ooni/backend/issues/219). All of this work is available through the following pull request: https://github.com/ooni/pipeline/pull/292 We also made several other improvements to the OONI data processing pipeline throughout February 2020, as documented through the following pull requests. https://github.com/ooni/pipeline/pull/290 https://github.com/ooni/pipeline/pull/291 https://github.com/ooni/pipeline/pull/292 https://github.com/ooni/pipeline/pull/295 https://github.com/ooni/pipeline/pull/296 https://github.com/ooni/pipeline/pull/297 https://github.com/ooni/pipeline/pull/299 ## OONI infrastructure We made several important improvements to the OONI server infrastructure. Specifically, we: * Consolidated the pattern used to deploy web-connectivity test helpers: https://github.com/ooni/backend/issues/300 * Upgraded our monitoring software to the latest version, prometheus 2.15.2: https://github.com/ooni/backend/issues/302 * Dealt with several incidents affecting our infrastructure and discussed how to better document them going forward: https://github.com/ooni/backend/issues/343 * Added support for logging long-running queries and plotted them in an internal dashboard: https://github.com/ooni/backend/issues/325 ## OONI website In collaboration with community members, we: * Improved the SEO of the OONI website: https://github.com/ooni/ooni.org/issues/314 * Replaced the CSS framework of the OONI website: https://github.com/ooni/ooni.org/issues/343 * Fixed the HTML <title> tag for certain pages on our website, improving how our website appears in search engines: https://github.com/ooni/ooni.org/issues/296 ## Google Summer of Code We submitted the following OONI project ideas for the Google Summer of Code (GSoC): * OONI Probe network experiments: https://community.torproject.org/gsoc/ooni-probe-experiments/ * OONI Explorer Advanced Search: https://community.torproject.org/gsoc/ooni-explorer-advanced-search/ * Privacy aware geo-lookup: https://community.torproject.org/gsoc/privacy-aware-geo-lookup/ ## Report on censorship in Togo We published a research report on the blocking of instant messaging apps in Togo amid the 2020 presidential election. Our report is available here: https://ooni.org/post/2020-togo-blocks-instant-messaging-apps/ Thanks to measurements contributed by OONI Probe users in Togo, we found that access to the WhatsApp mobile app, Telegram Web, and Facebook Messenger were blocked on 2 networks (Togo Telecom & Atlantique Telecom) in Togo on election day. They were, however, accessible on the Canalbox network, suggesting that internet censorship varies across ISPs in Togo. VPNCompare published a blog post discussing our report on Togo: https://www.vpncompare.co.uk/internet-censorship-togo/ ## Supported OTF Fellow Over the last year, we have hosted an OTF Information Controls Fellow who has been researching internet censorship in certain states of India. Throughout February 2020, we worked closely with the fellow to analyze relevant OONI measurements and document the findings. ## Collaboration with Netalitica Netalitica researchers continue to do an excellent job updating the Citizen Lab test lists! In February 2020, we reviewed Netalitica updates to the Citizen Lab test lists for Kazakhstan (https://github.com/citizenlab/test-lists/pull/585), Turkey, and Ukraine. In response to a data analysis request from Netalitica, we extracted and analyzed Web Connectivity measurements from 19 countries and generated charts, all of which are available through the following ticket: https://github.com/ooni/ooni.org/issues/333. ## Other test list updates We opened pull requests with several other test list updates as well. More specifically, we: * Updated the global and Chinese test lists to include coronavirus-related websites: https://github.com/citizenlab/test-lists/pull/571 & https://github.com/citizenlab/test-lists/pull/572 * Updated the global test list to add a circumvention tool site: https://github.com/citizenlab/test-lists/pull/569 * Updated the Iranian test list: https://github.com/citizenlab/test-lists/pull/575 & https://github.com/citizenlab/test-lists/pull/574 ## Reviewed RightsCon session proposals This year, OONI’s Maria is a Program Committee Co-Chair for the Network Connectivity and Internet Shutdowns track at RightsCon 2020. As part of this capacity, Maria reviewed 27 RightsCon session proposals submitted under the Network Connectivity and Internet Shutdowns track (excluding session proposals submitted by OONI). ## Organization of IFF Internet Measurement Village Throughout February 2020, we continued to coordinate efforts around the organization of the Internet Measurement Village at the Internet Freedom Festival (IFF). Our coordination efforts involved reaching out to community members to encourage them to submit session proposals, discussing session ideas with community members, collecting session ideas in order to create an agenda, and participating in bi-weekly calls with other village organizers. Unfortunately though the IFF 2020 has been cancelled as a result of the escalating impact of COVID-19 (coronavirus) in Europe and around the world. Nonetheless, we hope to co-organize the Internet Measurement Village at the IFF next year. Meanwhile, we would like to explore opportunities to facilitate measurement sessions remotely online (beyond the online monthly OONI community meetings). ## Community use of OONI data ### Censored Planet & Citizen Lab research paper Censored Planet and the Citizen Lab collaborated on a research paper, titled “Measuring the Deployment of Network Censorship Filters at Global Scale”, which makes use of OONI data. Their research paper (available here: https://censoredplanet.org/assets/filtermap.pdf) will appear in Network and Distributed System Security Symposium (NDSS), 2020. Censored Planet also published a report summarizing the findings: https://censoredplanet.org/filtermap As part of their research, Censored Planet and the Citizen Lab used OONI data (along with Censored Planet data) to gather blockpages from filter deployments around the world. ### Blocking of Tutanota in Russia In mid-February 2020, OONI measurements collected from Russia revealed the blocking of email provider tutanota.com. Once Tutanota was notified of the blocking of their site in Russia, they independently coordinated OONI Probe testing on Twitter to gather more measurements: https://twitter.com/TutanotaTeam/status/1228735217768292352 Roskomsvoboda followed up to share the relevant blocklist (https://reestr.rublacklist.net/search/?q=tutanota) and they published a post on the blocking of tutanota.com in Russia: https://roskomsvoboda.org/55478/ A news article, citing OONI Explorer measurements on the blocking of tutanota.com in Russia, was also published: https://www.clubic.com/pro/legislation-loi-internet/donnees-personnelles/ac… ## Community activities ### FOSDEM OONI’s Federico traveled to Brussels to attend FOSDEM (https://fosdem.org/2020/) on 1st & 2nd February 2020. ### Community meeting We facilitated the monthly OONI Community Meeting on 25th February 2020 on our Slack channel (https://slack.ooni.org/), during which we discussed: 1. Measuring throttling 2. Measuring the blocking of IP addresses 3. Mining OONI data to examine the blocking of the Tor network ## Userbase In February 2020, 7,669,694 OONI Probe measurements were collected from 5,511 networks in 214 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on monthly coverage worldwide): http://explorer.ooni.org/ ~ The OONI team. -- Maria Xynou Research & Partnerships Director Open Observatory of Network Interference (OONI) https://ooni.org/ PGP Key Fingerprint: 2DC8 AFB6 CA11 B552 1081 FBDE 2131 B3BE 70CA 417E

1 0

Network Team Meeting Notes, 24 February and 4 March
by Alexander Færøy 09 Mar '20

09 Mar '20

Hello folks, The meeting we had last Wednesday on the 4th of March had so few participants that we went over the weekly items we always have, but postponed discussions until more members of the team could be present. Logs from the meeting the 4th of March: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-03-04-22.59.html Meeting notes from the meeting on the 24th of February: 1) We started out with roadmap. 2) We had discussions on reviews and in particular how to do reviews of TROVE security issues. 3) We went over our 0.4.3 status page. 4) We discussed how to do the integration with Shadow in Tor. 5) Swati joined our meeting to say bye after she finished her work as a Google Season of Docs student. Swati have done some excellent work on re-structuring our Unix man pages and worked close together with Taylor on this. 6) Nobody had anything else for this meeting and the meeting was ended. --- end of summary --- You can read the network team meeting log at: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-02-24-17.59.html Below are the contents of our meeting pad: Nick: Week of 17 Feb (planned): - Lots of code review - Continue bug retrospective? - Work on relay/dirauth modularization stuff - Any C style work to do? Week of 17 Feb (actual): - Worked on lots of patches to make more relay code disabled when --disable-relay-mode - lots of review and merge - continued bug retrospective - review and merge - far too many meetings - Revised pending patches Week of 24 Feb (planned): - Review and merge - Try to advance TROVE-2020-00{1,2,3} - Prepare for March - Write policy proposals for adopting current acting policies, and adding a "critical" scurity level - C style work as possible - Try to finish bug retrospective? - Try to work on smaller code-cleanup patches and any pending 043 issues - Try to do a personal roadmap for next 3-4 months catalyst: week of 02/17 (2020-W08) (planned): - TPI holiday 02/17 - reviews - more gsod stuff as needed - writing feedback - create follow up ticket(s) for stuff out of scope for #32921 - dig a bit more into #32622 week of 02/17 (2020-W08) (actual): - TPI holiday 02/17 - more gsod review (#33275) - gsod: reviewed TOC and cross-ref linking (#33369) - more review of #32921 week of 02/24 (2020-W09) (planned): - reviews - gsod wrap up (final reports, etc) - writing feedback ahf: Week of 17/2 2020 (planned) - Write email about a discussion I had with Peter Stuge and Linus at FOSDEM. - Hack on 0.4.3 tickets. - If the new GL instance comes up before Friday, try to do a run of ticket migrations to it to see if things are better. Write email to tor-project@ with an update. - Look into crash issue in #32729 Week of 17/2 2020 (actually) - Many meetings and post-meeting conversations/TODO list items. - Tried to reproduce #32340 (I think we have had this ticket on all platforms now at some point) - Hiro said GL instance is progressing. Worked on Trac Wiki to Markdown to GL wiki migration. - Didn't get around to write Linus/Peter email. Week of 24/2 2020 (planned) - Get minimal test case working for TROVE issue. - Do patch for #32165. - Maybe write that Linus/Peter email :-) - Figure out what is up and down with the Outreachy project(s). asn: Week of 17/02 (planned): - Get back to OBv3. - Get #32709 merged. Week of 17/02 (actual): - Got #32709 merged! - Started working on OBv3 documentation. Week of 24/02 (planned): - Merge first round of OBv3 documentation. - Finish OBv3 integration with travis/readthedocs/etc. - Reviews. dgoulet: Week of Feb 17th (actual): - Reviews and merges. - Patch for #33361 - Requested testing on longclaw/moria for #33029. We got it merged finally. We can move on to #33072. - Tracing branch #32910 discussed with nickm. Experimented with various possible fixes to nickm's comment. - Spent a considerable amount of time analyzing Freenode tor's onion service log to understand why the service stopped publishing its descriptors. So I revived an old patch from #24346 and made it to #33400. - Confirming that #32672 will affect bridges. - Paper revisions with Rob for our NetDev 0x14 submissions. Worked on the slide outline as well. Week of Feb 24th (planned): - Work on #33072, #33400 (if need be) - Revised #32910 (tracing branch from nickm's feedback). - Do a little bit of slides for NetDev. - Feedback writing. - Need to think about the next things I'll work on for the next months. Gaba: Last week (actual): - reduce scalabiltiy project to fit new requirements Week of February 24th (planned) - scalabiltiy project risks - run behind trac - s55 follow up teor: (online first meeting of the month, offline at the usual meeting time) Week of 10 February (planned): Important: - Write Draft of Proposal 313: Relay IPv6 Statistics (#33159) - Revise Proposal 313 based on reviews - respond to emails and IRC Roadmap: - open Sponsor 55 tickets for required work - based on proposals - with updated estimates - make existing tickets children of Sponsor 55 parent tickets Other: - Ticket triage, backport deciding, quick code reviews - Python 3 reviews / helping new contributors Week of 10 February (actual): Important: - Write Proposal 313: Relay IPv6 Statistics (#33159) - Revise Proposal 313 based on reviews - Merge Proposal 313 - Create Sponsor 55 tickets, doing estimates, and writing a rough implementation order - respond to emails Roadmap: - made some existing tickets children of Sponsor 55 parent tickets - found a relay bandwidth stats bug that affects our bandwidth and connection stats (#33201), so I guess it's Sponsor 55. - Turns out it's not a bug, it's a misleading comment. - test IPv6 in CI (#33195 and children) - fix protocol version sorting issue (#33285) - help tom with info about IPv6 relays for consensus health (#33266) - Start designing chutney reachability tests (#33232) Other: - Ticket triage, backport deciding, quick code reviews, merges - Python 3 reviews / helping new contributors - Help with manual page rewrite (#33188) - Help with git scripts (#32121, #33284) - Help with 0.4.3 logging fix (#33087) Week of 17 February (planned): Important: Roadmap: - chutney relay IP4 reachability self-tests (#33232) - Add an ipv6 mixed network to chutney and tor - #32588 Setting ORPort [ipv6]:auto mistakenly advertises port 94 - Maybe #33220 Prop 311: 3. Allow Relay IPv6 Extends Other: - Ticket triage, backport deciding, quick code reviews and ticket help - Back to normal code reviews (thanks everyone for letting me focus on Sponsor 55 proposals!) - Python 3 reviews / helping new contributors Week of 17 February (actual): Important: - tor-relays moderation - list moderation was taking a lot of time, and I need to focus on Sponsor 55, so I'm stepping away from list moderation for a while Roadmap: - Make consensus voting more resistant to late votes (#4631) - chutney relay IP4 reachability self-tests (#33232) - Add an ipv6 mixed network to chutney and tor (#33334) - Update the default chutney networks for new tor features (#33376) - Use chutney's diagnostics in Tor's CI (#33353, #32792) - Require all nodes to bootstrap (#33378) - Working on: Require all relays in the consensus - Next up is: Remove AssumeReachable 1 from relays - chutney and tor changes to improve chutney tests Other: - Ticket triage, backport deciding, quick code reviews and ticket help - Back to normal code reviews (thanks everyone for letting me focus on Sponsor 55 proposals!) - Python 3 reviews / helping new contributors - Some quick bug fixes Mike: week of 19 Feb (planned): - Investigate/reproduce circpad shutdown bugs (#30992 and chutney warns); plan some fixes - Mull over Research Janitor responsibilities and priorites - Maybe review metrics-team tickets week of 19 Feb (actual): - Still discussing Reasearch Janitor role priorities - Performance funding proposal work week of 24 Feb (planned): - Cashing in some banked overtime dayz (Monday+Tuesday) - Probably keep discussing Reasearch Janitor role priorities - Performance funding proposal work - Metrics-team flashflow experiment+sbws review - circpad bugs I hope? Coin toss for that vs metrics-team review (though the metrics work is pretty cold now anyway... maybe it can wait some more) jnewsome: week of 02/03 (2020-W06) (actual): - Onboarding logistics - Fleshed out Shadow sponsor page a bit https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor38 - Fixed some Shadow compiler warnings and bugs (https://github.com/shadow/shadow/issues/711 and related issues) week of 02/10 (2020-W07) (planned): - Enable -Werror in Shadow, suppressing existing warnings if necessary (https://github.com/shadow/shadow/issues/711) - Add clang compile to CI - Fix remaining tests that listen on hard-coded ports (https://github.com/shadow/shadow/issues/718) - Start planning Shadow TCP redesign week of 02/10 (2020-W07) (actual): - Fixed some additional warnings in Shadow, but haven't enabled -Werror yet. Now planning to only enable it in CI - Investigated possibility of running Shadow CI inside Docker containers. Ran into issues with elfloader. Deferring for now since elfloader is planned to go away. - Created Github CI action for shadow-plugin-tor (shadow-plugin-tor@head against with fixed versions of shadow and tgen) week of 02/17 (2020-W08) (planned): - Extract shadow-plugin-tor CI core logic into a shell script - Set up shadow repository to also run the shadow-plugin-tor CI (shadow@head against fixed versions of shadow-plugin-tor and tgen) - Suppress remaining compiler warnings in Shadow and enable -Werror in CI - Start looking at TCP module and planning refactor/redesign week of 02/17 (2020-W08) (actual): - TPI holiday on Monday; vacation on Friday - Extracted shadow-plugin-tor CI core logic into a shell script https://github.com/shadow/shadow-plugin-tor/pull/86 - Fixed remaining compiler warnings in Shadow (https://github.com/shadow/shadow/pull/725) and enabled -Werror in CI (https://github.com/shadow/shadow/pull/726) - Started looking at TCP module and planning refactor/redesign, with an eye towards migrating it to Rust week of 02/24 (2020-W08) (planned): - Enable running the shadow-plugin-tor CI from the shadow repo (should be trivial after last week's refactor) - Continue looking at TCP module and planning refactor/redesign, with an eye towards migrating it to Rust - Phantom kickoff meeting this week (migrating Shadow to process-per-simulated-host) - Create a tor trac ticket for "build for shadow", capturing the requirements All the best, Alex. -- Alexander Færøy

1 0

Changes to the Tor Exit List service
by Iain Learmonth 09 Mar '20

09 Mar '20

Hi, We will shortly be decommissioning the server that ran TorDNSEL and Tor Check until now. The TorDNSEL software will be replaced on the new server with software we will be able to more easily maintain going forward. Service operators using the Tor Exit List service may need to take action to avoid service interruption, details are contained later in this post. ## For exit relay operators: The primary change that comes with the new software is that exit policies are no longer considered when deciding if an IP address is to be included in the list. If we have observed an exit relay using an IP address through our active measurements, this will be listed as an exit relay in the new service regardless of the exit policy. For exit relay operators that also route their own traffic via their exit relay's IP address and have exit policies specifically to deny access to services that block Tor traffic, such that those services will allow you to connect without Tor, this will no longer work. We recommend using a dedicated IP address for your exit relay. That allows your ISP to more easily recognize that abuse complaints and DMCA notices can be forwarded to you to be quickly responded to with a boilerplate response, as opposed to cutting off your Internet access or providing your personal information to the copyright cartels. ## For service operators: Depending on how you obtain exit address information, you may need to take action to avoid service interruption. If you are using CollecTor or Onionoo, these interfaces will remain unchanged and are already compatible with the new setup. If you are using the DNS exit list service or the bulk exit list exporter then you may need to make changes. ### DNS exit list The good news is that this service now behaves closer to a typical DNS-based list service and so it may be easier to integrate into your use-case now, and no longer require a custom implementation. The old DNS exit list would have lookups that look like: <reverse client ip>.<server port>.<reverse server ip>.ip-port.exitlist.torproject.org For services that are accessed via multiple IP addresses, e.g. IRC networks with multiple servers or websites behind load balancers, this leads to service operators needing to perform multiple lookups in order to have confidence that an IP address is not an exit relay. Instead, services can now use this simplified service: <reverse client ip>.dnsel.torproject.org, just like other DNS list services (https://en.wikipedia.org/wiki/Domain_Name_System-based_Blackhole_List ). If a client IP address is a Tor exit relay, the service will return with an A record of 127.0.0.2. You'll also be able to look up a TXT record with the fingerprint of the relay to learn more about the individual relay. Note that some IP addresses are shared between multiple exit relays, they will still only have one A record but may have multiple TXT records, one for each fingerprint. If an IP address is not known in the Tor network, the response will contain a NXDOMAIN (no such domain) status. For example: ------------------------------------------------------------------------ $ dig +noall +comments +answer 199.72.247.162.dnsel.torproject.org ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46707 199.72.247.162.dnsel.torproject.org. 1080 IN A 127.0.0.2 ------------------------------------------------------------------------ $ dig +noall +comments +answer 199.72.247.162.dnsel.torproject.org txt ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29451 199.72.247.162.dnsel.torproject.org. 1095 IN TXT "B34CC9056250847D1980F08285B01CF0B718C0B6" ------------------------------------------------------------------------ dig +noall +comments +answer 198.81.40.188.dnsel.torproject.org txt ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 2678 ------------------------------------------------------------------------ *The old DNS exit list service will be turned off on the 1st April 2020. Please ensure you have updated to the new service before this time.* ### Bulk exit list changes The bulk exit list exporter also used to consider exit policies, but will now return all exit relay IP addresses regardless of the query made. You should not need to make any modifications if you are using this service but you may find that you can remove some complexity from your client if you were filtering by service before. You can fetch a list of all exit IP addresses seen from: https://check.torproject.org/torbulkexitlist This list is updated at most once every 40 minutes depending on the number of exit relays in the network at the time, fetching it every hour would be reasonable. Thanks, Iain.

1 0

Network Team Meeting Time Changes
by Alexander Færøy 09 Mar '20

09 Mar '20

Hello all, The US have moved from Standard Time to Daylight Time and we have updated the meeting times for the weekly network team meetings. Please see an updated list at https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam All the best, Alex. -- Alexander Færøy

1 0

Notes from Vegas Meeting MARCH 5 2020
by Erin Wyatt 06 Mar '20

06 Mar '20

Vegas Team Meeting Notes March 5 2020: + NEW BUSINESS - Should we cancel or reschedule the NY gathering from April or not? Resolution: Maybe we'll move it to May and hope for the best. - SVN people SVN! Can we move it to nextcloud? Resolution: tl;dr: we’ll keep it for now. + GENERAL NOTES Georg 1. Network health work 2. Alex's and my talk at foss-north got accepted (https://foss-north.se/2020/schedule.html#slot19), yay! Antonela 1. Donation flow changes in www and tor browser 2. S27 is almost done 3. Slowly starting with S30 4. End of the month duties 5. Sent peer-feedback 6. Writing onion browser review 7. Reviewing OTF Learning Lab work Anarcat 1. more work on libvirt/ganeti migration (#33085) 2. more work on check.tpo replacement 3. peer feedback work 4. taxes form wrangling 5. more gitlab patch review, hiro is finalizing the migration of the test instance to a new server right now! 6. znc irc bouncer debugging (#33483) 7. CiviCRM mail rate expansion monitoring (#33189) 8. more routine work (PGP key expiries, mail delivery problems #33413, etc) Roger 1. Mike: I told Jim Newsome that when you have your act together, you'll be reaching out to him to describe our use cases and wishlists and goals for what we want from Shadow. That way he can have them in mind as he makes Shadow improvements, and who knows maybe he'll even lead the activities you ask for. 2. I'm planning to not submit a Defcon talk this year, since "every other year" is a fine rate for me. I am imagining to submit a CCC talk instead (finally). If you have opinions, please help me think through what is best to submit. Gaba 1. Traveling: should we cancel or reschedule the NY gathering from April or not? 2. SVN people SVN! Can we move it to nextcloud? 3. Sponsor 30 check-in 4. Trac is up to date in my inbox 5. or library size reduction coordination 6. update time allocation usage 7. network team retrospective 8. peer feedback 9. gitlab follow up Alex: 0. I will not be at this meeting as I'm traveling home from a conference in the other side of DK at this time. 1. Feedback writing and making sure I have received everything from folks as it comes in. 2. Network Team retrospective went well. 3. Going to dive into our new Gitlab setup on Friday. 4. End of month tasks. 5. Talking about Tor today in Aarhus with the Danish engineering union and on Wednesday at the University College in Odense. 6. Some follow ups from 1:1's, but the team seems to be doing good. Steph 1. Finishing peer and self feedback 2. Catching up on DoH stuff so I feel more comfortable contributing. Matt and I will be talking about it more today so we can write a response. 3. Wrote an op-ed for Bitcoin Magazine 4. Newsletter went out Friday. GR did some testing and now it goes out in 3 hours and not 24. 5. Wrapped up Outreachy mentoring with annalee_! She completed projects of updating the press list and fixing blog tags. Submitted final feedback 6. Working with OTF learning lab on dark web demystifying. Also working internally to connect larger onion services goals. 7. Social media 8. Helping with NY event coordination Sarah 1. Moving ahead with details for San Francisco and NYC events while monitoring covid19 developments. 2. Peer feedback. 3. Grants team working on DRL and OTF proposals. 4. Taking over fundraising related tasks from Jon. isabela 1. 1st round of interview for ED assistant candidates (2nd round will be this coming week) 2. Reviewing proposals due this week (DRL Scalability work / DRL w/ tails and guardian project / OTF browser work) 3. continue to work on invites for EFF party and creating a plan b in case of covid-19 intervention :) 4. Writing self feedback 5. multiples 1:1 - including irl 1:1 w/ matt in nyc pili 1. End/Start of month maintenance tasks: Tor Browser february report, trac backlog review, etc... 2. S9 sync 3. Moving github sponsors forward 4. Dealing with GSoC queries and updating ideas list accordingly 5. DRL Proposal 6. OTF Proposal revisions 7. Applied to the Learning Lab to showcase new features from Onion Services project 8. Trying to figure out if we're going to have someone from Tor attending a writing sprint for an online digital security training course 9. Some work on the dev portal - Dev teams should expect a request for feedback in the next few weeks :) 10. Peer and self feedback 11. Tor Browser Release meeting this week ewyatt: 1. Reminder: all written feedback due tomorrow, March 6th (self, team lead, and peer) 2. Traveling, scrambling to get things together. I will be working remotely for most days (my time is on the AFK calendar) 3. Making progress on Exec Assistant recruiting 4. Still haven't gotten PTO's out, but it's still in top 3 spots on my list, hoping to get to it soon! 5. Contracts updates (still working on) Karsten: 1. Exit scanner transition almost complete. \o/ 2. Added dirbytes reported by dirauths (#33065), deployment still in progress. Mike: 1. Helping with DRL proposal 2. Working on employee feedback 3. Untangling a decade of Tor organizational dysfunction, so I can get back to fixing its technical dysfunction. Gus: 1. Reviewing Cleopatra's work (RT and documentation updates). Final feedback is due March 9. 2. Contacted partners that submitted proposals for our Tor Village and asked them if they want to be part of remote training program. 3. Traveling tonight for a workshop with journalists. 4. Submitted Tor activity to CryptoRave 5. Folks from a non-profit asked us if we can write a comment about strong encryption and human rights 6. Sponsor9 sync meeting 7. Still writing peer feedback 8. Outreachy: new contribution period starting today. We have two projects this round: https://trac.torproject.org/projects/tor/wiki/Outreachy 9. Reviewing DRL proposal. 10. Community Team is organizing a new Documentation Hackathon. plz open more tickets: https://lists.torproject.org/pipermail/tor-project/2020-March/002748.html Nick 1. Hoping for releases in 5-10 days. 2. Starting on walking onions writeup 3. Wrapping up feedback. I might be late, but not by more than a day or two 4. Went to event on decentralized web yesterday. Interesting people, but I might have gained more had I known which days other stuff was happening. I should learn from this. 5. NYC event- go or no go? Matt: 1. Collect team feedback 2. Moar Tor Browser releases 3. OTF Tor Browser proposal 4. (Beginning) W3C involvement 5. Conversation about engaging in IETF discussions around IP privacy \o/ 6. Writing peer feeback

1 0

Metrics Meeting Notes, March 5th 2020
by Gaba 05 Mar '20

05 Mar '20

Hi! We had our weekly metrics team meeting today. The next one is going to be on March 12th at 1500 UTC. The pad for the meetings in 2020 is at https://pad.riseup.net/p/tor-metricsteam-2020.1-keep The agenda and notes for today are here: ---------------------------------- Agenda Thursday, March 5th, 15 UTC ---------------------------------- Feedback due tomorrow (karsten) Exit List Service Change Announcement (irl) https://trac.torproject.org/projects/tor/ticket/33510 https://pad.riseup.net/p/exit-scanner-transistion-announcement-keep Needs review tickets (karsten) https://trac.torproject.org/projects/tor/query?status=needs_review&componen… Defect tickets (karsten) https://trac.torproject.org/projects/tor/query?status=accepted&status=assig… Roadmap (gaba) https://trello.com/b/Mu5fYg53/tor-metrics-roadmap Log: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-03-05-15.02.log… -- Project Manager: Network, Anti-Censorship and Metrics teams gaba at torproject.org she/her are my pronouns GPG Fingerprint EE3F DF5C AD91 643C 21BE 8370 180D B06C 59CA BD19

1 0

Anti-censorship team meeting notes 05 March 2020
by Cecylia Bocovich 05 Mar '20

05 Mar '20

Here is our meeting log: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-03-05-17.59.html And our meeting pad: Anti-censorship work meeting pad -------------------------------- Next meeting: Thursday March 5th 18:00 UTC Weekly meetings, every Thursday at 18:00 UTC, in #tor-meeting at OFTC (channel is logged while meetings are in progress). == Goal of this meeting == Weekly checkin about the status of anti-censorship work at Tor. Coordinate collaboration between people/teams on anti-censorship at Tor. == Links to Useful documents == Our anti-censorship roadmap: https://dip.torproject.org/torproject/anti-censorship/roadmap/boards Our roadmap consists of a subset of trac tickets. For 2020 Q1, the trac keyword is anti-censorship-roadmap-2020Q1 The anti-censorship team's wiki page: https://trac.torproject.org/projects/tor/wiki/org/teams/AntiCensorshipTeam Past meetings' notes can be found at: https://lists.torproject.org/pipermail/tor-project/ GetTor's roadmap: https://dip.torproject.org/groups/torproject/anti-censorship/gettor-project… Tickets that need reviews: https://trac.torproject.org/projects/tor/query?status=needs_review&componen… Projects from sponsors we are working on: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor30 https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor28 ------------------------- --- March 5th 2020 -- ------------------------- == Announcements == Snowflake got a mention on privacytools.io: https://www.privacytools.io/browsers/#addons Tor participating in Google Summer of Code, ideas page: https://community.torproject.org/gsoc/ == Discussion == - We want to run a session during IFF to get people to circumvent censorship. We were thinking of getting people to connect to a censored network, e.g by providing a router that censors in different way; and try to access the internet or some sites or something. We would need some advice/help to set something like this up -- Pili & Gus - Source code from 2017 experiment (few hundred users, Windows only, simulated censorship with a local firewall, used an instrumented Tor Browser to record mouse clicks): - https://github.com/lindanlee/PETS2017-paper - see experiment/firewall.bat and experiment/gen-firewall - see the screenrecording directory - see the session/pre/videos directory for videos and detailed notes of a pre-study - see the setup directory for other setup notes (don't remember what exactly) - paper: https://censorbib.nymity.ch/#Lee2017a - Source code from 2015 experiment (5 users, one on one, no censorship, just browser usability): - git clone https://www.bamsoftware.com/repo.eecs.berkeley.edu/tor-ux.git/ (see README and tasks.html) - videos: https://people.torproject.org/~dcf/uxsprint2015/ - blog post: https://blog.torproject.org/blog/ux-sprint-2015-wrapup == Actions == == Interesting links == https://ooni.org/post/2020-iran-blocks-farsi-wikipedia/ "Over the last days, between 2nd to 3rd March 2020, OONI measurements collected from 6 different networks in Iran showed that access to the Farsi edition of Wikipedia (fa.wikipedia.org) was temporarily blocked by means of DNS tampering and SNI filtering." "The Farsi edition of Wikinews (fa.wikinews.org) has remained blocked ... It’s worth highlighting that the mobile version of the Farsi language edition of Wikipedia (fa.m.wikipedia.org) remains accessible, as does the English version of Wikipedia (en.wikipedia.org)." == Updates == Name: This week: - What you worked on this week. Next week: - What you are planning to work on next week (related to anti-censorship work). Help with: - Something you may need help with. Hiro: (last updated before 2020-01-09) Started restoring Twitter Responder. When retrieving tweets I need to check for new tweets only otherwise Gettor will end up responding to old back log and already answered messages. <-- is this #27330 ? Next Week: - Work on gettor specs <-- Is this #3781? - More with review of strings and website content and translation phw: This week (2020-02-20): Will be gone until our Mar 12 meeting. Merged and deployed BridgeDB's Python 3 port (#30946). Retired scramblesuit, obfs3, and fte from BridgeDB. Refactored email distributor check tool for #12802. Wrote OONI patches for new default bridge. Reviewed #23225. Next week: Write a summary of our current BridgeDB distribution mechanisms and brainstorm new ones Help with: Gaba: () Last week (): This week (planned): cecylia (cohosh): last updated 2020-03-05 Last week: - implemented snowflake throughput check (#32938) - updated snowflake webextension and badge with new translations - wrote a patch for gitlab issues with gettor (#33034) - added a go module to snowflake (#33330) - grant edits - responded to GSoC emails - looked at old pcaps for clues about why some proxies fail This week: - maybe continue work on throughput check (#32938) - tbb patch for #33330 - implement update notification for proxy-go instances (#32677) - Aggregate and write up a report of snowflake throughput changes (#32545) - revise GetTor message body (#9036) Needs help with: - review of #33034 (hiro?) - review of snowflake tickets #32938 and #33330 arlolra: 2020-02-13 Last week: - merged the client part of #19026 Next week: - maybe the proxy parts of #19026 - pick another task? Help with: - dcf: 2020-03-05 Last week: - turned off the unused snowflake-reg-test.appspot.com reflector (#33429) - posted a workaround for snowflake-turbotunnel packages updating themselves (#33336) - helped diagnose a snowflake stall when there are multiple SOCKS connections (#33519) Next week: - review go.mod for snowflake (#33330) - review snowflake throughput tests (#32938) - work on a fix for multiple SOCKS connections (#33519) - maybe make new packages with 9.5a7 when that's out Help with: cjb: 2019-03-05 Last week: - went ahead with removing the bridgeauth-side change from #31011 so that ticket can be split out Next week: - poke at golang snowflake proxy Android app Help with: agix:2020-03-05 Last week: - Still on #31426 - Fixed #31528 - I have summarized the errors and difficulties I encountered when setting up BridgeDB and noted some small fixes and suggestions for improvement Next week: - Continue on #31426 - Checkout the BridgeDB Port to Python3 Help with: - I changed the references in #31528 but responses like "Uh, oh spaghettios!" have already been translated into different languages. How should I handle that? - Just to clarify, what is the preferred way for you guys to submit a fix? Should I just push my branch or submit a patch as an attachment on the ticket?

1 0

Dip down for 1 hour approx today March 5 1400 UTC
by Hiro 05 Mar '20

05 Mar '20

Hi all, We have to migrate dip to a different VM and the service might be down for about 30/60 minutes today March 5 around 1400 UTC. What will you have to do? Mostly nothing except waiting for the service to come back. If when the service is back you notice some issues please open a ticket on Trac under the git component and assign it to me. We will try to copy everything over, but it might happen that you will have to re-add your ssh key to your profile or your profile pic (if you have changed that in the first place). The "new gitlab" will be available at gitlab.tp.o and dip.tp.o will redirect to it. I will send another email when everything has been migrated. Talk soon, -hiro

1 1

Tor Project will be participating in GSoC under the DIAL Open Source Center
by Pili Guerra 05 Mar '20

05 Mar '20

Hi everyone, I just wanted to let you all know that Tor Project will be participating in Google Summer of Code this year under the DIAL Open Source Center umbrella organization[1]! Please check out our project ideas page[2] and help us spread the word to any interested students. Thank you! Pili [1] https://hub.osc.dial.community/t/google-summer-of-code-2020-open-source-cen… <https://hub.osc.dial.community/t/google-summer-of-code-2020-open-source-cen…> [2] https://community.torproject.org/gsoc/ <https://community.torproject.org/gsoc/> — Project Manager: Tor Browser, UX and Community teams pili at torproject dot org gpg 3E7F A89E 2459 B6CC A62F 56B8 C6CB 772E F096 9C45

1 0

S30 Meeting on Tuesday March 10th at 1800 UTC [Empowering Communities in the Global South to Bypass Censorship ]
by Gaba 03 Mar '20

03 Mar '20

Hi! We are meeting on March 10th for the S30 project [0]. The meeting will be at 1800 UTC in #tor-meeting, irc.oftc.net The agenda and notes will be at https://pad.riseup.net/p/sponsor30-agenda-notes-Y1-keep cheers, gaba [0] https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor30 -- Project Manager: Network, Anti-Censorship and Metrics teams gaba at torproject.org she/her are my pronouns GPG Fingerprint EE3F DF5C AD91 643C 21BE 8370 180D B06C 59CA BD19

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

tor-project