March 2020 - tor-project - lists.torproject.org

February 2020 report for the user experience team
by Antonela Debiasi 11 Mar '20

11 Mar '20

Hello Tor, We started February in the rainy Berlin. We met Firefox's folks at their bi-annual All Hands gathering and we discussed regular topics around our collaboration. After that, many of us met at FOSDEM. We specially enjoyed the talk diogosergio gave at the Open Design Track. You may want to watch it! https://fosdem.org/2020/schedule/event/git_workflow_for_design_in_os_projec… S27 Onion services ================== We spent February on implementing an approach to move forward with the onion names' problem. We found that organically, developers have been approaching this problem in different ways, mostly with solutions tailored for the pleasure of their own service. Given that there is not a complete solution that works perfectly for all our user groups, we also approached this problem with a broad angle. From headers to centralized lists we are trying to allow end-users to have an experience close to the defaults without losing security: type a memorable domain at the URL bar and reach an onionsite, anonymously. As we update the URL bar to show the alias, the circuit display will allow users to check the origin onion address for integrity. Impressive work, acat! You can read more about this process here https://trac.torproject.org/projects/tor/ticket/28005 OTF Usability Lab: Tor Metrics ============================== Last year we applied with Simply Secure for an OTF Usability Labs grant for working on the next iteration of the metrics portal. The proposal went through,and Nina Vizz will lead this project. Gaba and I as well Metrics Team folks, we will be the stakeholders of this project. User Research ============= Nah and Piliattended the Open Design meetup, theusability testing workshop before FOSDEM. They built together a "Download Page" research that S9 partners will be running in March along with trainings. We made progress on setting a workflow in git for our upcoming user research. https://dip.torproject.org/torproject/ux/research/issues/1 We worked on the Onion Services Research, we expect to run it virtually and in person. https://dip.torproject.org/torproject/ux/research/issues/3 Since we may need to localize some of this materials, emmapeel suggested to have it in .md format. We will be working on it during March. https://dip.torproject.org/torproject/ux/research/issues/2 WWW === We finally iterated torproject.org/download. We included a label for each operative system, we improved the discoverability of the signature files and we also made it look better in small screens. https://dip.torproject.org/torproject/web/tpo/issues/59 https://dip.torproject.org/torproject/web/tpo/issues/6 https://trac.torproject.org/projects/tor/ticket/32460 S30 === During February, ICFP fellowship Babatunde has been traveling through Nigeria and Cameroon interviewing users who have been experiencing information controls. He is now digesting all this material and we will work together on expanding Tor personas in order to inform our work on s30. L10n ==== We have updated more websites with new contentstill, no new languages added but soon we will. Also, we helped Snowflake and Tor Browser teams with the translations and reviewsof the next features release. We got some more reviewers but still, more is needed. If you are interested in being a reviewer of a language just ping emmapeel on irc! Fundraising =========== Emmapeel and alsmith helped draft the MOSS proposal for the Linguine project, to prepare an open platform for translation. We submited a joint DRL proposal for continuing our work on user research with communities with Guardian Project and Tails. Thanks Al and Pili for working on this narrative and estimations! Open Team ========= If you missed those, February weekly meetings notes are here: https://lists.torproject.org/pipermail/ux/2020-February/000487.html https://lists.torproject.org/pipermail/ux/2020-February/000488.html https://lists.torproject.org/pipermail/ux/2020-February/000490.html Peace and love, A -- Antonela Debiasi UX Team Lead @antonela E2330A6D1EB5A0C8 https://torproject.org

1 0

minutes from the sysadmin meeting
by Antoine Beaupré 10 Mar '20

10 Mar '20

Hello everyone! here's your monthly minutes digest. :) # Roll call: who's there and emergencies anarcat, gaba, hiro, and linus present. # What has everyone been up to ## hiro - migrate gitlab-01 to a new VM (gitlab-02) and use the omnibus package instead of ansible (#32949) - automate upgrades (#31957 ) - anti-censorship monitoring (external prometheus setup assistance) (#31159) - blog migration planning and setting up expectations ## anarcat <https://trac.torproject.org/projects/tor/query?owner=anarcat&status=closed&…> AKA: Major work: * retire textile [#31686][] * new gnt-fsn node (fsn-node-04) [#33081][] * fsn-node-03 disk problems [#33098][] * fix up /etc/aliases with puppet [#32283][] * decomission storm / bracteata on February 11, 2020 [#32390][] * review the puppet bootstrapping process [#32914][] * ferm: convert BASE_SSH_ALLOWED rules into puppet exported rules [#33143][] * decomission savii [#33441][] * decomission build-x86-07 [#33442][] * adopt puppetlabs apt module [#33277][] * provision a VM for the new exit scanner [#33362][] * started work on unifolium decom [#33085][] * improved installer process (reduced the number of steps by half) * audited nagios puppet module to work towards puppetization ([#32901][]) [#32901]: https://bugs.torproject.org/32901 Routine tasks: * Add aliases to apache config on check-01 [#33536][] * New RT queue and alias iff@tpo [#33138][] * migrate sysadmin roadmap in trac wiki [#33141][] * Please update karsten's new PGP subkey [#33261][] * Please no longer delegate onionperf-dev.torproject.net zone to AWS [#33308][] * Please update GPG key for irl [#33492][] * peer feedback work * taxes form wrangling * puppet patch reviews * znc irc bouncer debugging [#33483][] * CiviCRM mail rate expansion monitoring [#33189][] * mail delivery problems [#33413][] * [meta-policy process][] adopted * package installs ([#33295][]) * RT root noises ([#33314][]) * debian packaging and bugtracking * SVN discussion * contacted various teams to followup on buster upgrades (translation [#33110][] and metrics [#33111][]) - see also [progress followup][] * nc.riseup.net retirement coordination #32391 [progress followup]: https://help.torproject.org/tsa/howto/upgrades/buster/#Per_host_progress [meta-policy process]: https://help.torproject.org/tsa/policy/tpa-rfc-1-policy/ [#33111]: https://bugs.torproject.org/33111 [#33110]: https://bugs.torproject.org/33110 [#33314]: https://bugs.torproject.org/33314 [#33295]: https://bugs.torproject.org/33295 [#33413]: https://bugs.torproject.org/33413 [#33189]: https://bugs.torproject.org/33189 [#33483]: https://bugs.torproject.org/33483 [#33536]: https://bugs.torproject.org/33536 [#31686]: https://bugs.torproject.org/31686 [#33081]: https://bugs.torproject.org/33081 [#33098]: https://bugs.torproject.org/33098 [#33138]: https://bugs.torproject.org/33138 [#33141]: https://bugs.torproject.org/33141 [#32283]: https://bugs.torproject.org/32283 [#32390]: https://bugs.torproject.org/32390 [#32914]: https://bugs.torproject.org/32914 [#33143]: https://bugs.torproject.org/33143 [#33261]: https://bugs.torproject.org/33261 [#33308]: https://bugs.torproject.org/33308 [#33362]: https://bugs.torproject.org/33362 [#33441]: https://bugs.torproject.org/33441 [#33442]: https://bugs.torproject.org/33442 [#33492]: https://bugs.torproject.org/33492 [#33277]: https://bugs.torproject.org/33277 [#33085]: https://bugs.torproject.org/33085 ## qbi - created several new trac components (for new sponsors) - disabled components (moved to archive) - changed mailing list settings on request of moderators # What we're up to next I suggest we move this to the systematic roadmap / ticket review instead in the future, but that can be discussed in the roadmap review section below. For now: ## anarcat * unifolium retirement (cupani, polyanthum, omeiense still to migrate) * chase cymru and replace moly? * retire kvm3 * new ganeti node ## hiro - retire gitlab-01 - TPA-RFC-2: define how users get support, what's an emergency and what is supported (#31243) - Migrating the blog to a static website with lektor. Make a test with discourse as comment platform. # Roadmap review We keep on using this system for march: <https://trac.torproject.org/projects/tor/wiki/org/teams/SysadminTeam> Many things have been rescheduled to march and april because we ran out of time to do what we wanted. In particular, the libvirt/kvm migrations are taking more time than expected. # Policies review TPA-RFC-1: policy; marked as adopted TPA-RFC-2; support; hiro to write up a draft. TPA-RFC-3: tools; to be brainstormed here The goal of the new RFC is to define which *tools* we use in TPA. This does not concern service admins, at least not in the short term, but only sysadmin stuff. "Tools", in this context, are programs we use to implement a "service". For example, the "mailing list" service is being ran by the "mailman" tool (but could be implemented with another). Similarly, the "web cache proxy" service is implemented by varnish and haproxy, but is being phased out in favor of Varnish. Another goal is to *limit* the number of tools team members should know to be functional in the team, and formalize past decisions (like "we use debian"). We particularly discussed the idea of introducing Fabric as an "ad-hoc changes tool" to automate host installation, retirement, and reboots. It's already in use to automate libvirt/ganeti migrations and is serving us well there. # Other discussions A live demo of the Fabric code was performed some time after the meeting and no one raised objections to the new project. # Next meeting No discussed, but should be on april 6th 2020. # Metrics of the month * hosts in Puppet: 77, LDAP: 81, Prometheus exporters: 124 * number of apache servers monitored: 31, hits per second: 148 * number of nginx servers: 2, hits per second: 2, hit ratio: 0.89 * number of self-hosted nameservers: 6, mail servers: 10 * pending upgrades: 174, reboots: 0 * average load: 0.63, memory available: 308.91 GiB/1017.79 GiB, running processes: 411 * bytes sent: 169.04 MB/s, received: 101.53 MB/s * planned buster upgrades completion date: 2020-06-24 -- Antoine Beaupré torproject.org system administration

1 0

cupani AKA git-rw IP address changed
by Antoine Beaupré 10 Mar '20

10 Mar '20

The main git server, cupani, is the machine you connect to when you push or pull git repositories over ssh to git-rw.torproject.org. That machines has been migrated to the new Ganeti cluster. This required an IP address change from: 78.47.38.228 2a01:4f8:211:6e8:0:823:4:1 to: 116.202.120.182 2a01:4f8:fff0:4f:266:37ff:fe32:cfb2 DNS has been updated and preliminary tests show that everything is mostly working. You *will* get a warning about the IP address change when connecting over SSH, which will go away after the first connection. That is normal. The SSH fingerprints of the host did *not* change. Please do report any other anomaly using the normal channels: https://help.torproject.org/tsa/doc/how-to-get-help/ The service was unavailable for about an hour during the migration. Thank you for your patience, A. PS: details of the work are available here: https://trac.torproject.org/projects/tor/ticket/33446 -- Antoine Beaupré torproject.org system administration

1 0

February 2020 report for the Tor Browser team
by Pili Guerra 10 Mar '20

10 Mar '20

Hi everyone, During February the team has been working hard on finishing off work around improving Onion Service usability in Tor Browser as part of our work for Sponsor 27[1]. This work includes some exciting updates to HSv3 client authorization[2], including authorization key management[3] in Tor Browser. We have also been working on improvements to the way connection errors to onion services are presented[4][5] to a user when connection to an onion service fails[6][7]. Related to this, we have also been working on how we present warnings about self-signed certificates for .onion sites[8] and improving the url bar onion indicators[9]. We have also been working on a proof of concept to support human memorable addresses for onion services[10] by supporting .onions in HTTPSEverywhere[11]. We made two releases this past month: 9.0.5[12] and 9.5a5[13]. We have also moved closer to getting automatic nightly updates working and we hope to share some exciting news in this area soon. Finally, we have started working on the transition away from Mozilla's ESR train and onto the Release train. As part of this work we have started to rebase Tor Browser patches onto mozilla-central and reviewing our testsuite. A few of us were at FOSDEM in Brussels where we gave a talk[14] sharing our plans for the year ahead for the Tor Browser Team. The full list of tickets closed by the Tor Browser team in February is accessible using the TorBrowserTeam202020[15] keyword in our bug tracker. All tickets on our radar for March can be seen with the `TorBrowserTeam202003` keyword in our bug tracker.[16] Thanks for reading! Pili [1] https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor27 [2] https://trac.torproject.org/projects/tor/ticket/30237 [3] https://trac.torproject.org/projects/tor/ticket/19757 [4] https://trac.torproject.org/projects/tor/ticket/30022 [5] https://trac.torproject.org/projects/tor/ticket/30025 [6] https://trac.torproject.org/projects/tor/ticket/19251 [7] https://trac.torproject.org/projects/tor/ticket/23545 [8] https://trac.torproject.org/projects/tor/ticket/13410 [9] https://trac.torproject.org/projects/tor/ticket/32645 [10] https://trac.torproject.org/projects/tor/ticket/30029 [11] https://trac.torproject.org/projects/tor/ticket/28005 [12] https://blog.torproject.org/new-release-tor-browser-905 [13] https://blog.torproject.org/new-release-tor-browser-95a5 [14] https://fosdem.org/2020/schedule/event/tor/ [15] https://trac.torproject.org/projects/tor/query?status=closed&keywords=~TorB… [16] https://trac.torproject.org/projects/tor/query?status=accepted&status=assig… <https://trac.torproject.org/projects/tor/query?status=accepted&status=assig…> — Project Manager: Tor Browser, UX and Community teams pili at torproject dot org gpg 3E7F A89E 2459 B6CC A62F 56B8 C6CB 772E F096 9C45

1 0

S27 (Onion Services) Meeting today: Tuesday 10th March @ 15UTC
by Pili Guerra 10 Mar '20

10 Mar '20

Hi everyone, Here’s your usual reminder about this meeting today to discuss the Onion Services project[1]. Please note that, if you recently changed to summer time, this meeting may be at a different time than what you are used to. Thanks! Pili [1] https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor27 <https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor27> — Project Manager: Tor Browser, UX and Community teams pili at torproject dot org gpg 3E7F A89E 2459 B6CC A62F 56B8 C6CB 772E F096 9C45

1 0

Network health meeting notes, 10 Mar 2020
by Georg Koppen 10 Mar '20

10 Mar '20

Hi! Yesterday we had another round of our weekly network health meeting in #tor-meeting. The IRC log was captured, as usual, by meetbot and can be found at: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-03-09-19.00.log… (If you ever asked why there are sometimes pings like "Sebastian, weasel, micah, arma1, stefani: dirauth update. Thanks!" in our #tor-project channel seemingly out of the blue, be sure to read the meeting log above as it contains an explanation of that is going on. :)) The details from our meeting pad are as follows: Discussion: - How to move forward with #32672 (bridges, timeframe, other nagging efforts to upgrade)? [GeKo: phw whill contact bridge operators this week and we'll make some recommendations for the network team as to when to merge the patch in our meeting next Monday] - How do we adjust meeting times due to DST? [GeKo: the plan is to keep the UTC meeting time for now and move to 1800 UTC once the Europeans have switched to DST as well] Statuses: GeKo: Last week: - more sbws work (help with reviewing #30726, started to look into #33009 thinking about how to best test a fix within the integration tests framework) - looking at the bridges situation for #32672 - resume work on #32864 - feedback reviews - look at ggus' Tor legal questions pad (https://pad.riseup.net/p/tor-legal-questions-keep) email draft: () - tracked down a bug in stem (https://github.com/torproject/stem/commit/7a0a8dd8d4218d5dabec3c2e47bebc4d1…) that our doctor checks hit - the usual bad relay activity - sent out monthly team report for February (https://lists.torproject.org/pipermail/tor-project/2020-March/002745.html) - input for DRL proposal This week: - more sbws work - #32864 - getting back to EFF for the legal questions update juga: Last week: - Worked on #33570 Correct the relays to keep after retrieving new consensuses - Worked/revised on #30726 Missing relay keys in bandwidth file spec Next week: - Continue with #30726 Missing relay keys in bandwidth file spec - Continue with #30719 Work out why 90% of sbws measurements fail Gaba: Not much related to network health. Georg

1 1

OONI Monthly Report: February 2020
by Maria Xynou 09 Mar '20

09 Mar '20

Hello, Below is OONI's progress report from February 2020. ## OONI Probe support for running circumvention tool tests During February 2020, we added support to OONI Probe for testing the availability of the Tor, obfs4proxy, and Psiphon circumvention tools. These tests have been integrated into the OONI Probe engine (https://github.com/ooni/probe-engine) and are available to run via the OONI Probe desktop app. Below are the specifications for each test: * Tor test: https://github.com/ooni/spec/blob/master/nettests/ts-023-tor.md * Psiphon test: https://github.com/ooni/spec/blob/master/nettests/ts-015-psiphon.md As part of our work on integrating these tests into OONI Probe, we: * Implemented the Tor test UI in the OONI Probe desktop app: https://github.com/ooni/probe/issues/975 * Improved the UX of the Tor test results in the OONI Probe desktop app: https://github.com/ooni/probe/issues/990 * Created an animation that appears when OONI Probe users run circumvention tool tests: https://github.com/ooni/probe/issues/1029 * Configured the OONI Probe CLI (which the desktop app relies on) to submit measurements, even if an experiment fails: https://github.com/ooni/probe-engine/issues/316 ## Analysis of circumvention tool test data and presentation in OONI Explorer We added support for analyzing the new OONI Probe circumvention tool test results, and we integrated them into OONI Explorer and the OONI API. In February 2020, we worked on presenting the measurements in OONI Explorer for the following new OONI Probe tests: * Psiphon: https://github.com/ooni/explorer/issues/332 * Tor: https://github.com/ooni/explorer/issues/333 We also investigated how the batch pipeline deals with the new OONI Probe experiments for Tor and Psiphon: https://github.com/ooni/backend/issues/301 ## Making OONI Probe reporting logic more resilient to censorship To collect OONI measurements from around the world, OONI Probe clients need to upload measurements to OONI servers. As some censors may seek to block connections to OONI servers (in an attempt to prevent the collection of measurements), we have adopted various strategies over the years (such as using onion services), but they have presented various limitations and challenges. As part of our ongoing efforts to make OONI Probe’s reporting logic more resilient to censorship, we researched whether requesting OONI Probe users in censored environments to submit their measurements via instant messaging app messages would be a feasible and desirable option. This is documented through the following ticket: https://github.com/ooni/probe/issues/976 We also explored the possibility of enabling OONI Probe clients to submit their measurements via email, as documented through the following ticket: https://github.com/ooni/probe/issues/992 ## OONI Probe orchestration logic for circumvention tool testing As part of our work on developing OONI Probe orchestration logic that is specific to circumvention tool testing, we updated the canonical location of Tor test targets: https://github.com/ooni/backend/issues/299 These Tor testing targets are then given out to OONI Probe clients by using the OONI Probe orchestration system. ## OONI Probe desktop app We released the 4th release candidate of the OONI Probe desktop app 3.0.0: https://github.com/ooni/probe-desktop/releases/tag/v3.0.0-rc.4 Throughout February 2020, we researched how to best run OONI Probe automatically on desktop (https://github.com/ooni/probe/issues/1011) and we made a series of improvements to the OONI Probe desktop app. https://github.com/ooni/probe-desktop/pull/97 https://github.com/ooni/probe-desktop/pull/98 https://github.com/ooni/probe-desktop/pull/99 https://github.com/ooni/probe-desktop/pull/100 https://github.com/ooni/probe-desktop/pull/101 https://github.com/ooni/probe-desktop/pull/102 https://github.com/ooni/probe-desktop/pull/103 https://github.com/ooni/probe-desktop/pull/104 https://github.com/ooni/probe-desktop/pull/105 https://github.com/ooni/probe-desktop/pull/106 https://github.com/ooni/probe-desktop/pull/107 We also designed mock-ups for the UI elements of the OONI Probe desktop app: https://github.com/ooni/probe/issues/1009#issuecomment-590459771 https://github.com/ooni/probe/issues/966#issuecomment-585128461 https://github.com/ooni/probe/issues/1006#issuecomment-590354947 ## OONI Probe Command Line Interface (CLI) In February 2020, we made 2 OONI Probe CLI releases: * OONI Probe CLI v3.0.0-rc.9: https://github.com/ooni/probe-cli/releases/tag/v3.0.0-rc.9 * OONI Probe CLI v3.0.0-rc.10: https://github.com/ooni/probe-cli/releases/tag/v3.0.0-rc.10 These releases enable users to stop running experiments, specify test list categories, and they address a critical bug in how measurements are stored on disk. Throughout the month, we made a series of improvements to the OONI Probe CLI: https://github.com/ooni/probe-cli/pull/104 https://github.com/ooni/probe-cli/pull/106 https://github.com/ooni/probe-cli/pull/107 https://github.com/ooni/probe-cli/pull/108 https://github.com/ooni/probe-cli/pull/109 https://github.com/ooni/probe-cli/pull/110 https://github.com/ooni/probe-cli/pull/111 https://github.com/ooni/probe-cli/pull/112 https://github.com/ooni/probe-cli/pull/113 https://github.com/ooni/probe-cli/pull/114 https://github.com/ooni/probe-cli/pull/115 https://github.com/ooni/probe-cli/pull/116 ## OONI Probe mobile app Throughout February 2020, we made a series of improvements to the OONI Probe mobile app. On Android: https://github.com/ooni/probe-android/pull/289 https://github.com/ooni/probe-android/pull/290 https://github.com/ooni/probe-android/pull/291 https://github.com/ooni/probe-android/pull/292 https://github.com/ooni/probe-android/pull/293 https://github.com/ooni/probe-android/pull/295 https://github.com/ooni/probe-android/pull/296 https://github.com/ooni/probe-android/pull/297 On iOS: https://github.com/ooni/probe-ios/pull/336 https://github.com/ooni/probe-ios/pull/337 https://github.com/ooni/probe-ios/pull/338 https://github.com/ooni/probe-ios/pull/339 https://github.com/ooni/probe-ios/pull/340 https://github.com/ooni/probe-ios/pull/341 These improvements were in preparation for the release of OONI Probe 2.3.0, including a fix to the OONI Probe WhatsApp test which was presenting false positives following recent changes to WhatsApp’s infrastructure. ## Research on push notifications We researched how to best acquire push notification permissions from OONI Probe mobile app users (see https://github.com/ooni/probe/issues/813) As part of this, we wrote a document which outlines the problem that we are trying to solve, the core questions that we need to answer, and it proposes a potential implementation solution. This document is available here: https://docs.google.com/document/d/1x1jMjMq4-e8Xb4USegBSUV6V_cpNUWb2GwfBE5j… We also deployed a test backend for countly (https://github.com/ooni/sysadmin/commit/d69f9975a16fbb7c4b7a29ec0a0be28c135…) and did some testing to check how well it could fit our use case as a push notification backend. ## OONI Probe measurement engine We implemented padding for DNS over HTTPS (DoH) and DNS over TLS (DoT) in the OONI Probe measurement engine (https://github.com/ooni/probe-engine/issues/285) We also explored test helper options for the OONI Probe SNI blocking experiment (https://github.com/ooni/probe-engine/issues/303) OONI Probe has relied on the MaxMind database (https://www.maxmind.com/) for IP geolocation, but they recently changed their license. As part of our efforts to manage this change (https://github.com/ooni/probe-engine/issues/269) we explored the possibility of generating our own ASN database that would be compatible with the MaxMindDB (https://github.com/ooni/probe-engine/issues/336) We made a series of improvements to the OONI Probe engine throughout the month, as documented through the following pull requests. https://github.com/ooni/probe-engine/pull/313 https://github.com/ooni/probe-engine/pull/318 https://github.com/ooni/probe-engine/pull/319 https://github.com/ooni/probe-engine/pull/322 https://github.com/ooni/probe-engine/pull/323 https://github.com/ooni/probe-engine/pull/324 https://github.com/ooni/probe-engine/pull/329 https://github.com/ooni/probe-engine/pull/333 https://github.com/ooni/probe-engine/pull/342 https://github.com/ooni/probe-engine/pull/343 https://github.com/ooni/probe-engine/pull/344 https://github.com/ooni/probe-engine/pull/345 https://github.com/ooni/probe-engine/pull/346 https://github.com/ooni/probe-engine/pull/347 https://github.com/ooni/probe-engine/pull/354 https://github.com/ooni/probe-engine/pull/360 https://github.com/ooni/probe-engine/pull/361 https://github.com/ooni/probe-engine/pull/364 https://github.com/ooni/probe-engine/pull/368 https://github.com/ooni/probe-engine/pull/369 https://github.com/ooni/probe-engine/pull/370 https://github.com/ooni/probe-engine/pull/373 https://github.com/ooni/probe-engine/pull/374 https://github.com/ooni/probe-engine/pull/375 ## Making the OONI Probe apps rely entirely on the OONI Probe golang engine We made considerable progress on making the OONI Probe mobile and desktop apps rely entirely on the golang based probe-engine. To this end, we: * Investigated the use of certifi/gocertifi to support shipping SSL certificates using Go: https://github.com/ooni/probe-engine/issues/296 * Replaced most of the HTTP code in measurement-kit with the golang based ooni/netx: https://github.com/ooni/probe-engine/issues/302 * Implemented a golang-based replacement for the measurement-kit method called MKAsyncTask: https://github.com/ooni/probe-engine/issues/339 * Ensured we never emit empty fields in the golang code: https://github.com/ooni/probe-engine/issues/348 ## OONI Explorer Throughout February 2020, we made a series of improvements to OONI Explorer based on GitHub tickets opened by community members and our team over the last months. Our work on improving OONI Explorer is available through the following pull requests. https://github.com/ooni/explorer/pull/387 https://github.com/ooni/explorer/pull/393 https://github.com/ooni/explorer/pull/394 https://github.com/ooni/explorer/pull/395 https://github.com/ooni/explorer/pull/397 https://github.com/ooni/explorer/pull/398 https://github.com/ooni/explorer/pull/401 https://github.com/ooni/explorer/pull/404 https://github.com/ooni/explorer/pull/405 https://github.com/ooni/explorer/pull/408 https://github.com/ooni/explorer/pull/409 https://github.com/ooni/explorer/pull/411 https://github.com/ooni/explorer/pull/412 https://github.com/ooni/explorer/pull/415 We also started implementing a strategy for performing end-to-end testing of OONI Explorer to ensure that everything works as expected. See: https://github.com/ooni/explorer/issues/159 & https://github.com/ooni/explorer/pull/417. This is a big step in the direction of ensuring better quality of the OONI software ecosystem end-to-end. ## OONI API In order to improve the performance of the OONI API and of the services that rely on it (such as OONI Explorer), we investigated how to better log long-running queries (https://github.com/ooni/backend/issues/325) As part of this, we created an internal dashboard that generates metrics on query runtime and enables us to better track heavy queries (https://github.com/ooni/api/pull/162) We also implemented limits to the database queries (https://github.com/ooni/api/pull/165) and made a series of other improvements, as documented through the following pull requests. https://github.com/ooni/api/pull/163 https://github.com/ooni/api/pull/164 https://github.com/ooni/api/pull/166 https://github.com/ooni/api/pull/167 https://github.com/ooni/api/pull/168 https://github.com/ooni/api/pull/169 https://github.com/ooni/api/pull/170 https://github.com/ooni/api/pull/171 https://github.com/ooni/api/pull/172 ## OONI data processing pipeline In order for OONI Explorer to present “confirmed blocked” cases, we need to add the fingerprints of block pages to the OONI database. Thanks to the support of community members -- and, especially, Vasilis Ververis from the Magma project -- many GitHub tickets have been filed providing block page fingerprints. Throughout February 2020, we completed the task of adding all the remaining fingerprints to the OONI database: https://github.com/ooni/backend/issues/281. This improves the analytics needed to extract per-website metrics. We implemented a watchdog system that monitors the amount of rows generated in our fast-path pipeline and compares them with the measurement table. This enables us to see if we’re missing any measurements and to otherwise better understand issues related to OONI data processing. We also implemented an internal dashboard that generates charts and alarms based on deltas (https://github.com/ooni/backend/issues/219) All of this work is available through the following pull request: https://github.com/ooni/pipeline/pull/292 We also made several other improvements to the OONI data processing pipeline throughout February 2020, as documented through the following pull requests. https://github.com/ooni/pipeline/pull/290 https://github.com/ooni/pipeline/pull/291 https://github.com/ooni/pipeline/pull/292 https://github.com/ooni/pipeline/pull/295 https://github.com/ooni/pipeline/pull/296 https://github.com/ooni/pipeline/pull/297 https://github.com/ooni/pipeline/pull/299 ## OONI infrastructure We made several important improvements to the OONI server infrastructure. Specifically, we: * Consolidated the pattern used to deploy web-connectivity test helpers: https://github.com/ooni/backend/issues/300 * Upgraded our monitoring software to the latest version, prometheus 2.15.2: https://github.com/ooni/backend/issues/302 * Dealt with several incidents affecting our infrastructure and discussed how to better document them going forward: https://github.com/ooni/backend/issues/343 * Added support for logging long-running queries and plotted them in an internal dashboard: https://github.com/ooni/backend/issues/325 ## OONI website In collaboration with community members, we: * Improved the SEO of the OONI website: https://github.com/ooni/ooni.org/issues/314 * Replaced the CSS framework of the OONI website: https://github.com/ooni/ooni.org/issues/343 * Fixed the HTML <title> tag for certain pages on our website, improving how our website appears in search engines: https://github.com/ooni/ooni.org/issues/296 ## Google Summer of Code We submitted the following OONI project ideas for the Google Summer of Code (GSoC): * OONI Probe network experiments: https://community.torproject.org/gsoc/ooni-probe-experiments/ * OONI Explorer Advanced Search: https://community.torproject.org/gsoc/ooni-explorer-advanced-search/ * Privacy aware geo-lookup: https://community.torproject.org/gsoc/privacy-aware-geo-lookup/ ## Report on censorship in Togo We published a research report on the blocking of instant messaging apps in Togo amid the 2020 presidential election. Our report is available here: https://ooni.org/post/2020-togo-blocks-instant-messaging-apps/ Thanks to measurements contributed by OONI Probe users in Togo, we found that access to the WhatsApp mobile app, Telegram Web, and Facebook Messenger were blocked on 2 networks (Togo Telecom & Atlantique Telecom) in Togo on election day. They were, however, accessible on the Canalbox network, suggesting that internet censorship varies across ISPs in Togo. VPNCompare published a blog post discussing our report on Togo: https://www.vpncompare.co.uk/internet-censorship-togo/ ## Supported OTF Fellow Over the last year, we have hosted an OTF Information Controls Fellow who has been researching internet censorship in certain states of India. Throughout February 2020, we worked closely with the fellow to analyze relevant OONI measurements and document the findings. ## Collaboration with Netalitica Netalitica researchers continue to do an excellent job updating the Citizen Lab test lists! In February 2020, we reviewed Netalitica updates to the Citizen Lab test lists for Kazakhstan (https://github.com/citizenlab/test-lists/pull/585) Turkey, and Ukraine. In response to a data analysis request from Netalitica, we extracted and analyzed Web Connectivity measurements from 19 countries and generated charts, all of which are available through the following ticket: https://github.com/ooni/ooni.org/issues/333. ## Other test list updates We opened pull requests with several other test list updates as well. More specifically, we: * Updated the global and Chinese test lists to include coronavirus-related websites: https://github.com/citizenlab/test-lists/pull/571 & https://github.com/citizenlab/test-lists/pull/572 * Updated the global test list to add a circumvention tool site: https://github.com/citizenlab/test-lists/pull/569 * Updated the Iranian test list: https://github.com/citizenlab/test-lists/pull/575 & https://github.com/citizenlab/test-lists/pull/574 ## Reviewed RightsCon session proposals This year, OONI’s Maria is a Program Committee Co-Chair for the Network Connectivity and Internet Shutdowns track at RightsCon 2020. As part of this capacity, Maria reviewed 27 RightsCon session proposals submitted under the Network Connectivity and Internet Shutdowns track (excluding session proposals submitted by OONI). ## Organization of IFF Internet Measurement Village Throughout February 2020, we continued to coordinate efforts around the organization of the Internet Measurement Village at the Internet Freedom Festival (IFF). Our coordination efforts involved reaching out to community members to encourage them to submit session proposals, discussing session ideas with community members, collecting session ideas in order to create an agenda, and participating in bi-weekly calls with other village organizers. Unfortunately though the IFF 2020 has been cancelled as a result of the escalating impact of COVID-19 (coronavirus) in Europe and around the world. Nonetheless, we hope to co-organize the Internet Measurement Village at the IFF next year. Meanwhile, we would like to explore opportunities to facilitate measurement sessions remotely online (beyond the online monthly OONI community meetings). ## Community use of OONI data ### Censored Planet & Citizen Lab research paper Censored Planet and the Citizen Lab collaborated on a research paper, titled “Measuring the Deployment of Network Censorship Filters at Global Scale”, which makes use of OONI data. Their research paper (available here: https://censoredplanet.org/assets/filtermap.pdf) will appear in Network and Distributed System Security Symposium (NDSS), 2020. Censored Planet also published a report summarizing the findings: https://censoredplanet.org/filtermap As part of their research, Censored Planet and the Citizen Lab used OONI data (along with Censored Planet data) to gather blockpages from filter deployments around the world. ### Blocking of Tutanota in Russia In mid-February 2020, OONI measurements collected from Russia revealed the blocking of email provider tutanota.com. Once Tutanota was notified of the blocking of their site in Russia, they independently coordinated OONI Probe testing on Twitter to gather more measurements: https://twitter.com/TutanotaTeam/status/1228735217768292352 Roskomsvoboda followed up to share the relevant blocklist (https://reestr.rublacklist.net/search/?q=tutanota) and they published a post on the blocking of tutanota.com in Russia: https://roskomsvoboda.org/55478/ A news article, citing OONI Explorer measurements on the blocking of tutanota.com in Russia, was also published: https://www.clubic.com/pro/legislation-loi-internet/donnees-personnelles/ac… ## Community activities ### FOSDEM OONI’s Federico traveled to Brussels to attend FOSDEM (https://fosdem.org/2020/) on 1st & 2nd February 2020. ### Community meeting We facilitated the monthly OONI Community Meeting on 25th February 2020 on our Slack channel (https://slack.ooni.org/) during which we discussed: 1. Measuring throttling 2. Measuring the blocking of IP addresses 3. Mining OONI data to examine the blocking of the Tor network ## Userbase In February 2020, 7,669,694 OONI Probe measurements were collected from 5,511 networks in 214 countries around the world. This information can also be found through our measurement stats on OONI Explorer (see chart on monthly coverage worldwide): http://explorer.ooni.org/ ~ The OONI team. -- Maria Xynou Research & Partnerships Director Open Observatory of Network Interference (OONI) https://ooni.org/ PGP Key Fingerprint: 2DC8 AFB6 CA11 B552 1081 FBDE 2131 B3BE 70CA 417E

1 0

Network Team Meeting Notes, 24 February and 4 March
by Alexander Færøy 09 Mar '20

09 Mar '20

Hello folks, The meeting we had last Wednesday on the 4th of March had so few participants that we went over the weekly items we always have, but postponed discussions until more members of the team could be present. Logs from the meeting the 4th of March: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-03-04-22.59.html Meeting notes from the meeting on the 24th of February: 1) We started out with roadmap. 2) We had discussions on reviews and in particular how to do reviews of TROVE security issues. 3) We went over our 0.4.3 status page. 4) We discussed how to do the integration with Shadow in Tor. 5) Swati joined our meeting to say bye after she finished her work as a Google Season of Docs student. Swati have done some excellent work on re-structuring our Unix man pages and worked close together with Taylor on this. 6) Nobody had anything else for this meeting and the meeting was ended. --- end of summary --- You can read the network team meeting log at: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-02-24-17.59.html Below are the contents of our meeting pad: Nick: Week of 17 Feb (planned): - Lots of code review - Continue bug retrospective? - Work on relay/dirauth modularization stuff - Any C style work to do? Week of 17 Feb (actual): - Worked on lots of patches to make more relay code disabled when --disable-relay-mode - lots of review and merge - continued bug retrospective - review and merge - far too many meetings - Revised pending patches Week of 24 Feb (planned): - Review and merge - Try to advance TROVE-2020-00{1,2,3} - Prepare for March - Write policy proposals for adopting current acting policies, and adding a "critical" scurity level - C style work as possible - Try to finish bug retrospective? - Try to work on smaller code-cleanup patches and any pending 043 issues - Try to do a personal roadmap for next 3-4 months catalyst: week of 02/17 (2020-W08) (planned): - TPI holiday 02/17 - reviews - more gsod stuff as needed - writing feedback - create follow up ticket(s) for stuff out of scope for #32921 - dig a bit more into #32622 week of 02/17 (2020-W08) (actual): - TPI holiday 02/17 - more gsod review (#33275) - gsod: reviewed TOC and cross-ref linking (#33369) - more review of #32921 week of 02/24 (2020-W09) (planned): - reviews - gsod wrap up (final reports, etc) - writing feedback ahf: Week of 17/2 2020 (planned) - Write email about a discussion I had with Peter Stuge and Linus at FOSDEM. - Hack on 0.4.3 tickets. - If the new GL instance comes up before Friday, try to do a run of ticket migrations to it to see if things are better. Write email to tor-project@ with an update. - Look into crash issue in #32729 Week of 17/2 2020 (actually) - Many meetings and post-meeting conversations/TODO list items. - Tried to reproduce #32340 (I think we have had this ticket on all platforms now at some point) - Hiro said GL instance is progressing. Worked on Trac Wiki to Markdown to GL wiki migration. - Didn't get around to write Linus/Peter email. Week of 24/2 2020 (planned) - Get minimal test case working for TROVE issue. - Do patch for #32165. - Maybe write that Linus/Peter email :-) - Figure out what is up and down with the Outreachy project(s). asn: Week of 17/02 (planned): - Get back to OBv3. - Get #32709 merged. Week of 17/02 (actual): - Got #32709 merged! - Started working on OBv3 documentation. Week of 24/02 (planned): - Merge first round of OBv3 documentation. - Finish OBv3 integration with travis/readthedocs/etc. - Reviews. dgoulet: Week of Feb 17th (actual): - Reviews and merges. - Patch for #33361 - Requested testing on longclaw/moria for #33029. We got it merged finally. We can move on to #33072. - Tracing branch #32910 discussed with nickm. Experimented with various possible fixes to nickm's comment. - Spent a considerable amount of time analyzing Freenode tor's onion service log to understand why the service stopped publishing its descriptors. So I revived an old patch from #24346 and made it to #33400. - Confirming that #32672 will affect bridges. - Paper revisions with Rob for our NetDev 0x14 submissions. Worked on the slide outline as well. Week of Feb 24th (planned): - Work on #33072, #33400 (if need be) - Revised #32910 (tracing branch from nickm's feedback). - Do a little bit of slides for NetDev. - Feedback writing. - Need to think about the next things I'll work on for the next months. Gaba: Last week (actual): - reduce scalabiltiy project to fit new requirements Week of February 24th (planned) - scalabiltiy project risks - run behind trac - s55 follow up teor: (online first meeting of the month, offline at the usual meeting time) Week of 10 February (planned): Important: - Write Draft of Proposal 313: Relay IPv6 Statistics (#33159) - Revise Proposal 313 based on reviews - respond to emails and IRC Roadmap: - open Sponsor 55 tickets for required work - based on proposals - with updated estimates - make existing tickets children of Sponsor 55 parent tickets Other: - Ticket triage, backport deciding, quick code reviews - Python 3 reviews / helping new contributors Week of 10 February (actual): Important: - Write Proposal 313: Relay IPv6 Statistics (#33159) - Revise Proposal 313 based on reviews - Merge Proposal 313 - Create Sponsor 55 tickets, doing estimates, and writing a rough implementation order - respond to emails Roadmap: - made some existing tickets children of Sponsor 55 parent tickets - found a relay bandwidth stats bug that affects our bandwidth and connection stats (#33201), so I guess it's Sponsor 55. - Turns out it's not a bug, it's a misleading comment. - test IPv6 in CI (#33195 and children) - fix protocol version sorting issue (#33285) - help tom with info about IPv6 relays for consensus health (#33266) - Start designing chutney reachability tests (#33232) Other: - Ticket triage, backport deciding, quick code reviews, merges - Python 3 reviews / helping new contributors - Help with manual page rewrite (#33188) - Help with git scripts (#32121, #33284) - Help with 0.4.3 logging fix (#33087) Week of 17 February (planned): Important: Roadmap: - chutney relay IP4 reachability self-tests (#33232) - Add an ipv6 mixed network to chutney and tor - #32588 Setting ORPort [ipv6]:auto mistakenly advertises port 94 - Maybe #33220 Prop 311: 3. Allow Relay IPv6 Extends Other: - Ticket triage, backport deciding, quick code reviews and ticket help - Back to normal code reviews (thanks everyone for letting me focus on Sponsor 55 proposals!) - Python 3 reviews / helping new contributors Week of 17 February (actual): Important: - tor-relays moderation - list moderation was taking a lot of time, and I need to focus on Sponsor 55, so I'm stepping away from list moderation for a while Roadmap: - Make consensus voting more resistant to late votes (#4631) - chutney relay IP4 reachability self-tests (#33232) - Add an ipv6 mixed network to chutney and tor (#33334) - Update the default chutney networks for new tor features (#33376) - Use chutney's diagnostics in Tor's CI (#33353, #32792) - Require all nodes to bootstrap (#33378) - Working on: Require all relays in the consensus - Next up is: Remove AssumeReachable 1 from relays - chutney and tor changes to improve chutney tests Other: - Ticket triage, backport deciding, quick code reviews and ticket help - Back to normal code reviews (thanks everyone for letting me focus on Sponsor 55 proposals!) - Python 3 reviews / helping new contributors - Some quick bug fixes Mike: week of 19 Feb (planned): - Investigate/reproduce circpad shutdown bugs (#30992 and chutney warns); plan some fixes - Mull over Research Janitor responsibilities and priorites - Maybe review metrics-team tickets week of 19 Feb (actual): - Still discussing Reasearch Janitor role priorities - Performance funding proposal work week of 24 Feb (planned): - Cashing in some banked overtime dayz (Monday+Tuesday) - Probably keep discussing Reasearch Janitor role priorities - Performance funding proposal work - Metrics-team flashflow experiment+sbws review - circpad bugs I hope? Coin toss for that vs metrics-team review (though the metrics work is pretty cold now anyway... maybe it can wait some more) jnewsome: week of 02/03 (2020-W06) (actual): - Onboarding logistics - Fleshed out Shadow sponsor page a bit https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor38 - Fixed some Shadow compiler warnings and bugs (https://github.com/shadow/shadow/issues/711 and related issues) week of 02/10 (2020-W07) (planned): - Enable -Werror in Shadow, suppressing existing warnings if necessary (https://github.com/shadow/shadow/issues/711) - Add clang compile to CI - Fix remaining tests that listen on hard-coded ports (https://github.com/shadow/shadow/issues/718) - Start planning Shadow TCP redesign week of 02/10 (2020-W07) (actual): - Fixed some additional warnings in Shadow, but haven't enabled -Werror yet. Now planning to only enable it in CI - Investigated possibility of running Shadow CI inside Docker containers. Ran into issues with elfloader. Deferring for now since elfloader is planned to go away. - Created Github CI action for shadow-plugin-tor (shadow-plugin-tor@head against with fixed versions of shadow and tgen) week of 02/17 (2020-W08) (planned): - Extract shadow-plugin-tor CI core logic into a shell script - Set up shadow repository to also run the shadow-plugin-tor CI (shadow@head against fixed versions of shadow-plugin-tor and tgen) - Suppress remaining compiler warnings in Shadow and enable -Werror in CI - Start looking at TCP module and planning refactor/redesign week of 02/17 (2020-W08) (actual): - TPI holiday on Monday; vacation on Friday - Extracted shadow-plugin-tor CI core logic into a shell script https://github.com/shadow/shadow-plugin-tor/pull/86 - Fixed remaining compiler warnings in Shadow (https://github.com/shadow/shadow/pull/725) and enabled -Werror in CI (https://github.com/shadow/shadow/pull/726) - Started looking at TCP module and planning refactor/redesign, with an eye towards migrating it to Rust week of 02/24 (2020-W08) (planned): - Enable running the shadow-plugin-tor CI from the shadow repo (should be trivial after last week's refactor) - Continue looking at TCP module and planning refactor/redesign, with an eye towards migrating it to Rust - Phantom kickoff meeting this week (migrating Shadow to process-per-simulated-host) - Create a tor trac ticket for "build for shadow", capturing the requirements All the best, Alex. -- Alexander Færøy

1 0

Changes to the Tor Exit List service
by Iain Learmonth 09 Mar '20

09 Mar '20

Hi, We will shortly be decommissioning the server that ran TorDNSEL and Tor Check until now. The TorDNSEL software will be replaced on the new server with software we will be able to more easily maintain going forward. Service operators using the Tor Exit List service may need to take action to avoid service interruption, details are contained later in this post. ## For exit relay operators: The primary change that comes with the new software is that exit policies are no longer considered when deciding if an IP address is to be included in the list. If we have observed an exit relay using an IP address through our active measurements, this will be listed as an exit relay in the new service regardless of the exit policy. For exit relay operators that also route their own traffic via their exit relay's IP address and have exit policies specifically to deny access to services that block Tor traffic, such that those services will allow you to connect without Tor, this will no longer work. We recommend using a dedicated IP address for your exit relay. That allows your ISP to more easily recognize that abuse complaints and DMCA notices can be forwarded to you to be quickly responded to with a boilerplate response, as opposed to cutting off your Internet access or providing your personal information to the copyright cartels. ## For service operators: Depending on how you obtain exit address information, you may need to take action to avoid service interruption. If you are using CollecTor or Onionoo, these interfaces will remain unchanged and are already compatible with the new setup. If you are using the DNS exit list service or the bulk exit list exporter then you may need to make changes. ### DNS exit list The good news is that this service now behaves closer to a typical DNS-based list service and so it may be easier to integrate into your use-case now, and no longer require a custom implementation. The old DNS exit list would have lookups that look like: <reverse client ip>.<server port>.<reverse server ip>.ip-port.exitlist.torproject.org For services that are accessed via multiple IP addresses, e.g. IRC networks with multiple servers or websites behind load balancers, this leads to service operators needing to perform multiple lookups in order to have confidence that an IP address is not an exit relay. Instead, services can now use this simplified service: <reverse client ip>.dnsel.torproject.org, just like other DNS list services (https://en.wikipedia.org/wiki/Domain_Name_System-based_Blackhole_List ). If a client IP address is a Tor exit relay, the service will return with an A record of 127.0.0.2. You'll also be able to look up a TXT record with the fingerprint of the relay to learn more about the individual relay. Note that some IP addresses are shared between multiple exit relays, they will still only have one A record but may have multiple TXT records, one for each fingerprint. If an IP address is not known in the Tor network, the response will contain a NXDOMAIN (no such domain) status. For example: ------------------------------------------------------------------------ $ dig +noall +comments +answer 199.72.247.162.dnsel.torproject.org ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46707 199.72.247.162.dnsel.torproject.org. 1080 IN A 127.0.0.2 ------------------------------------------------------------------------ $ dig +noall +comments +answer 199.72.247.162.dnsel.torproject.org txt ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29451 199.72.247.162.dnsel.torproject.org. 1095 IN TXT "B34CC9056250847D1980F08285B01CF0B718C0B6" ------------------------------------------------------------------------ dig +noall +comments +answer 198.81.40.188.dnsel.torproject.org txt ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 2678 ------------------------------------------------------------------------ *The old DNS exit list service will be turned off on the 1st April 2020. Please ensure you have updated to the new service before this time.* ### Bulk exit list changes The bulk exit list exporter also used to consider exit policies, but will now return all exit relay IP addresses regardless of the query made. You should not need to make any modifications if you are using this service but you may find that you can remove some complexity from your client if you were filtering by service before. You can fetch a list of all exit IP addresses seen from: https://check.torproject.org/torbulkexitlist This list is updated at most once every 40 minutes depending on the number of exit relays in the network at the time, fetching it every hour would be reasonable. Thanks, Iain.

1 0

Network Team Meeting Time Changes
by Alexander Færøy 09 Mar '20

09 Mar '20

Hello all, The US have moved from Standard Time to Daylight Time and we have updated the meeting times for the weekly network team meetings. Please see an updated list at https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam All the best, Alex. -- Alexander Færøy

1 0