tor-project

tor-project@lists.torproject.org

17 participants
2326 discussions

minutes from the sysadmin meeting
by Antoine Beaupré 07 Oct '19

07 Oct '19

Hello! Here are the minutes from this month's sysadmin meeting. # Roll call: who's there and emergencies anarcat, hiro, ln5, qbi and weasel are here. # What has everyone been up to ## anarcat * announced LDAP sudo transition plan ([#6367][]) * finished first phase of the hiera transition ([#30020][]) * deployed trocla in test ([#30009][]) * coordinate textile shutdown ([#31686][]) * announced jabber service shutdown ([#31700][]) * closed snowflake -> TPA transition ticket for now, external monitoring is sufficient ([#31232][]) * improvements on grafana dashboards * gitlab, nextcloud transitions coordination and oversight * ooni.tpo to ooni.io transition coordination ([#31718][]) * bugtracking on networking issues ([#31610][], [#31805][], [#31916][]) * regular janitorial work (security upgrades, reboots, crashes, disk space management, etc) * started needrestart deployment to reduce that work ([#31957][]) * completed the "reports card" questionaire ([#30881][]) * continued work on the upstream prometheus module * tested puppetboard as a Puppet Dashboard ([#31969][]) [#6367]: https://bugs.torproject.org/6367 [#30009]: https://bugs.torproject.org/30009 [#30020]: https://bugs.torproject.org/30020 [#31232]: https://bugs.torproject.org/31232 [#31686]: https://bugs.torproject.org/31686 [#31700]: https://bugs.torproject.org/31700 [#31718]: https://bugs.torproject.org/31718 [#31610]: https://bugs.torproject.org/31610 [#31805]: https://bugs.torproject.org/31805 [#31916]: https://bugs.torproject.org/31916 [#31969]: https://bugs.torproject.org/31969 ## weasel * Started with new onionoo hosts. Currently there's just one backend on fsn, irl is doing the service part (cf. #31659) * puppet cleanup: nameserver/hoster info * new static master on fsn * staticsync and bacula puppet cleanups/major-rework/syncs with debian * new fsn web frontends. only one is currently rotated * retire togashii, started retiring saxatile * moved windows VM away from textile * random updates/reboots/fixes * upgraded polyanthum to Debian 10 ## Hiro * Setup dip so that it can be easily rebased with debian upstream * Migrated gettor from getulum to gettor-01 * Random upgrades and reboots * Moving all my services to ansible or packages (no ad - hoc configuration): - Gettor can be deployed and updated via ansible - Survey should be deployed and updated via ansible - Gitlab (dip) is already on ansible - Schleuder should be maintained via packages * Nagios checks for gettor ## ln5 Didn't do much. :( ## qbi Didn't do volunteering due to private stuff # What we're up to next ## anarcat New: * LDAP sudo transition ([#6367][]) * jabber service shutdown ([#31700][]) * considering unattended-upgrades or at least automated needrestart deployment ([#31957][]) * followup on the various ops report card things ([#30881][]) * maybe deploy puppetboard as a Puppet Dashboard ([#31969][]), possibly moving puppetdb to a separate machine * nbg1/prometheus stability issues, ipsec seems to be the problem ([#31916][]) Continuing/stalled: * director replacement ([#31786][]) * taking a break on hiera refactoring ([#30020][]) * send root@ emails to RT ([#31242][]) * followup with email services improvements ([#30608][]) * continue prometheus module merges * followup on SVN decomissionning ([#17202][]) [#17202]: https://bugs.torproject.org/17202 [#30608]: https://bugs.torproject.org/30608 [#30881]: https://bugs.torproject.org/30881 [#31242]: https://bugs.torproject.org/31242 [#31232]: https://bugs.torproject.org/31232 [#31786]: https://bugs.torproject.org/31786 ## weasel * more VMs should move to gnt-fsn * more VMs should be upgraded * maybe get some of the pg config fu from dsa-puppet since the 3rd party pg module sucks ## Hiro * Nagios checks for bridgedb * decommissioning getulum * ansible recipe to manage survey.tp.o * dev portal coding in lektor * finishing moving gettor to gettor-01 includes gettor-web via lektor * do usual updates and rebots ## ln5 Nextcloud migration. # Other discussions ## configuration management systems We discussed the question of the "double tools problem" that seems to be coming up with the configuration management system: most systems are managed with Puppet, but some services are deployed with Puppet. It was argued it might be preferable to use Puppet everywhere to ease onboarding, since it would be one less tool to learn. But that might require giving people root, or managing services ourselves, which is currently out of the question. So it was agreed it's better to have services managed with ansible than not managed at all... # Next meeting We're changing the time because 1400UTC would be too early for anarcat because of daylight savings. We're pushing to 1500UTC, which is 1600CET and 1000EST. # Metrics of the month Access and transfer rates are an average over the last 15 days. * hosts in Puppet: 79, LDAP: 82, Prometheus exporters: 106 * number of apache servers monitored: 26, hits per second: 177 * number of self-hosted nameservers: 4, mail servers: 10 * pending upgrades: 0, reboots: 0 * average load: 0.51, memory available: 318.82 GiB/871.81 GiB, running processes: 379 * bytes sent: 134.28 MB/s, received: 94.38 MB/s Now also available as the main Grafana dashboard. Head to <https://grafana.torproject.org/>, change the time period to 7 days, and wait a while for results to render. Note that the retention period of the Prometheus server has been reduced from 30 to 15 days to address stability issues with the server (ticket #31916), so far without luck. -- Antoine Beaupré torproject.org system administration

1 0

Tor Browser Release Meeting this week: Wednesday 9th October @ 18UTC
by Pili Guerra 07 Oct '19

07 Oct '19

Hi everyone, We’ll be having our usual Tor Browser Release meeting this Wednesday 9th October @ 18UTC on #tor-meeting. We’ll be discussing the upcoming 9.0 stable release and this will be the very last chance to request any small additions. Thanks! Pili — Project Manager: Tor Browser, UX and Community teams pili at torproject dot org gpg 3E7F A89E 2459 B6CC A62F 56B8 C6CB 772E F096 9C45

1 0

S27 Meeting: Tuesday 8th October @ 16UTC
by Pili Guerra 07 Oct '19

07 Oct '19

Hi everyone, We’ll be having an un-scheduled meeting to discuss S27 and in particular Objective 2, Activity 1: Integrating client-side authorization to onion services v3[1]. The main agenda topic will be how to resolve incompatibilities between Tor Proposal 309: Optimistic SOCKS[2] and Tor Proposal 304: Extending SOCKS5 Onion Service Error Codes[3]. This meeting will take place 1hour later than usual at 1600UTC. Please join us if you think you have some input you can give to help us figure this one out. Thanks! Pili [1] https://trac.torproject.org/projects/tor/ticket/30000 <https://trac.torproject.org/projects/tor/ticket/30000> [2] https://gitweb.torproject.org/torspec.git/tree/proposals/309-optimistic-soc… <https://gitweb.torproject.org/torspec.git/tree/proposals/309-optimistic-soc…> [3] https://gitweb.torproject.org/torspec.git/tree/proposals/304-socks5-extendi… <https://gitweb.torproject.org/torspec.git/tree/proposals/304-socks5-extendi…> — Project Manager: Tor Browser, UX and Community teams pili at torproject dot org gpg 3E7F A89E 2459 B6CC A62F 56B8 C6CB 772E F096 9C45

1 0

[meeting] Sponsor 30: Empowering Communities in the Global South to Bypass Censorship
by Gaba 04 Oct '19

04 Oct '19

Hi! On Monday October 7th at 15UTC we will meet in #tor-meeting in irc to coordinate work on the project "Empowering Communities in the Global South to Bypass Censorship" [0] cheers, gaba [0]https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor30 -- Project Manager: Network, Anti-Censorship and Metrics teams gaba at torproject.org she/her are my pronouns GPG Fingerprint EE3F DF5C AD91 643C 21BE 8370 180D B06C 59CA BD19

1 0

Notes from Vegas Meeting OCTOBER 3 2019
by Erin Wyatt 04 Oct '19

04 Oct '19

Notes from Vegas Meeting OCTOBER 3 2019 ITEMS DISCUSSED: + Gaba • - Topics for Tor at Mozilla All Hands are being written down in a pad to be submitted ASAP. • * Discussion: We need to strategize on who we send and what we want to accomplish to make the most of the opportunity; priorities: browser and scalability people, and someone who can tie it all together. + Georg - Blog: a) Where are we with fixing our blog comment issue? b) A gentle reminder: please deal with the comments of your respective blog post. If you think comments should not get published, please remove them as this gets them out of the approval queue and it helps others dealing with comments via https://blog.torproject.org/admin/content/comment/approval. If you think you can't handle the comments yourself, please find someone in your team or somewhere else who can or close comments on the blog post otherwise. We have established a policy regarding comments on our blog a while back which can be found at: https://trac.torproject.org/projects/tor/wiki/doc/community/blog-comment-po…. * Discussion: Our blog hosting company increased our bill to ~$800/month (from ~$280) because of increased traffic, people will discuss outside this meeting on how to address the issue. + Pili - What is our policy on people printing Tor swag for themselves, their friends, etc...? i.e. not to make a profit or sell. We have different opinions from different people. * Discussion: We should agree on a policy; for now Isa will talk to Jon about when to send cease/desist letters. --------------->8 --------------->8 ———————>8 GENERAL NOTES: Steph 1) Press inquiries. 2) Community portal reviews. 3) Newsletter out this week. 4) Sent initial task to prospective Outreachy interns and getting them started. 5) Preparing for EOY campaign. 6) Signed us on to a letter to end Amazon Ring-police surveillance partnerships Philipp 1) Determined ten winners for our bridge campaign. We're counting 95 new bridges so far. 2) Snowflake made it into the most recent Tor Browser alpha for Windows 3) Progress on BridgeDB (updated requirements, prototype of language switcher, updated translations 4) Progress on obfs4 (polished flow obfuscator) Georg 1) The second Tor Browser alpha version based on Firefox 68 ESR got out (including snowflake support for Windows, the latest Tor alpha, x86_64 support for Android and other neat things) 2) 20 days left to Tor Browser 9.0 and a ton of work to do :) 3) Work on the OTF proposal for ESR migration Nick 1) 0.4.2.2-alpha comes out in the next couple of working days. This should fix all the known release-blockers; remaining issues are should-fix, not must-fix. 2) Working on team policy proposals. We adopted a meta-policy for deciding whether to adopt team policies: see https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/MetaPol… in case you want to use it too. 3) Hoping to open 0.4.3.x early, if we can make enough progress on 042-should issues. Alex: 1) Had 1:1 with everybody from the network-team by now together with gaba and will continue from now on with just me and the team member. 2) Work on 0.4.2 and S29 tasks. 3) Continuing with the Gitlab migration work together with a lot of people from different teams. Erin 1) Working on handbook clarifications 2) General HR stuff Gus 1) Getting in touch with Outreachy applicants. If you see someone asking on IRC about more information, point them to this wiki page: https://trac.torproject.org/projects/tor/wiki/Outreachy 2) Sponsor9: partners are signing the Partnership Agreement. We have some training in Global South in 2 weeks. 3) Answering Frontdesk/RT. 4) Working on Community (training resources) and Support portal content (old FAQ migration). 5) Submitted Freedom not fear travel allowance 6) Pili and brabo are organizing our activities in FOSDEM, help them. Please, update this wiki page: https://trac.torproject.org/projects/tor/wiki/org/meetings/2020Brussels 7) We should somehow formalize our process for now to submit people's about/people page to www folks. 8) CCC: are we going to organize activities there this year? Matt 1) Continuing dev work 2) Continuing roles/responsibility transition Mike 1) Firefox esr networking code review Roger 0) Steph, let's be sure not to drop the "connecting EFF to relay operator who might need legal advice" thread. Let me know how I can help there. [in motion -steph] 1) Pili, where are we on the trademark swag policy discussion? I saw a thread with Jon. 2) I heard a rumor we might be scheduling the next in-person dev meeting for February. We should start collecting date conflicts asap, and start a preliminary invite list, in order to send out save-the-dates asap. 3) Fyi, the network health team is bumping out a pile of obsolete relays real soon now. Pili 1) Helping out with OTF Browser proposal 2) Replying to Outreachy applicants 3) S27 report 4) Organising for FOSDEM 5) October roadmap reviews 6) Starting to think about the developer portal sitemap and information architecture with Antonela Arturo 1) Moving forward with OONI Probe Desktop and CLI apps 2) Deploying fastpath for near realtime publication of OONI measurements: https://github.com/ooni/pipeline/pull/213 & event detection engine: https://github.com/ooni/pipeline/pull/228 3) Working on Oonifying our probe services, see: https://github.com/ooni/sysadmin/issues/359 & cleaning up our infrastructure to ensure things are redeployable: https://github.com/ooni/sysadmin/issues/356 4) Published a post on blocking in media websites in Egypt: https://ooni.org/post/2019-egypt-blocks-bbc-and-alhurra/ 5) Hacktoberfest! https://twitter.com/OpenObservatory/status/1179033711557263361 6) Working towards releasing OONI Probe mobile 2.2 and ensuring all strings are translated Antonela 1) September end of the month duties 2) TB9 Release: UX improvements: new identity, network settings, dark theme support, onboarding update. 3) EOY: Working with Alon on the art 4) WWW: some progress on the tickets with my name isabela 1) Presented at Semcomp22 at USP São Carlos/Brazil 2) Working on YE campaign stuff that Sarah needs from me 3) Editing Major Donors folder text and selecting different metrics we could use for a visualization to go w/ the folder 4) catching up w/ email backlog 5) helping out with OTF Browser proposal Sarah 1) EOY campaign planning 2) Helping with OTF proposal 3) Updating major donors with latest vision 4) Planning Boston get together 5) Sourcing new foundation/grant funding Antoine 1) continued coordination on retirement of old services and machines 2) more puppet training, brought up the idea of using puppet for hosting services too instead of forcing people to use ansible 3) announced LDAP password change 4) started experimenting with more "standard" (as in: packaged in debian) tools different things in Puppet and debian (needrestart, trocla, etc) 5) dealt with much maintenance churn (disk space issue, security upgrades, reboots, etc), trying to automate Gaba 1) Tor/Mozilla meeting is next October 8th. Topics: https://pad.riseup.net/p/DYCTCA7mcI1bqS5B9RfR 2) Sponsor 30 preparation for next weeks's meeting 3) Catching up with trac tickets 4) A few 1:1:1s this week 5) Gitlab migration (right now the blocker is how we resolve registration for users and resolve moderation and spam) 6) Other random stuff

1 0

Network Team Meeting Notes, 2 October 2019
by Alexander Færøy 03 Oct '19

03 Oct '19

Hello, Here is a short summary of the network team meeting from yesterday: 1) We went over our Kanban board to ensure that it is up-to-date. We forgot to look at weekly review distribution. 2) We looked at 0.4.2 ticket status. It looks like most people are currently focused on 0.4.2 tickets. We should ensure in the future that we have space in our schedule for doing release tickets around the time of feature freeze. Some tickets were distributed around amongst the team members. Priority should be on the 042-must tickets for now. Once all 042-must tickets are done, Nick said he is happy to open up for the 043 merge window ahead of schedule. 3) During the network team retrospective meeting that happened Tuesday, the team agreed to move forward with a subset of the PR policy that have recently been discussed at some of the network team meetings. Alex is doing the work to turn this into a policy we can vote on. 4) Nick asked some questions about an upcoming poll about C coding style in tor.git. The poll is going to be sent out to the team soon. 5) We discussed the currently active proposal discussions that are happening on the internal network-team mailing list: The policies discussed: 1. "Opting-in to faster reviews and merges". 2. "Commit Bits For CI And Review Roles". 3. "Draft merge policy for discussion and vote". The discussion on these items will continue on the network-team mailing list and during upcoming meetings. 6) Alex asked a question on what we should do with these policies that we are now discussing in terms of announcing them to the rest of the organisation since a lot of the debate around them is happening on the internal network team mailing list. We decided that Alex puts these items onto the pad we are going to use for doing monthly network team reports to tor-project(a)lists.torproject.org. 7) Nobody had anything else to talk about so the meeting was over. Please remember that our next meeting is on Monday the 7th of October at 17:00 UTC. --- end of summary --- You can read today's network team meeting log at: http://meetbot.debian.net/tor-meeting/2019/tor-meeting.2019-10-02-22.59.html Below are the contents of our meeting pad: gaba: (updated on september 30th) Last week (actual): . s30 . s31 . 1:1:1s . get up to date with trac tickets This week (planned): . s30 coordination with all teams involved . roadmap update (bring october tickets) Help with: teor: (online first week of the month, offline at the usual meeting time) Week of 23 September (planned): Take Time for: - gitlab signup / config - team policies: - faster reviews - commit bit and roles? - finish add tor controller trace logging to diagnose stem hangs (#30901) - try nickm's suggested a simpler implementation - split off bugs I found while writing the control trace code Roadmap: - Sponsor 31 outstanding bugs / tasks - Sponsor 31 possible coding tasks for me - follow up email Other: - finish off bug smash logging bugs Week of 23 September (actual): Take Time for: - gitlab signup / config - team policies: - faster reviews - commit bit and roles - kept working on add tor controller trace logging to diagnose stem hangs (#30901) - split off bugs I found while writing the control trace code - did some preparation work Roadmap: - Sponsor 27 - Add IPv6 HSv3 chutney tests to CI - Sponsor 31 reviews - Sponsor 31 modularisation: disable relay mode when compiling - Initial design and strategy to move forward - Preparation work - speed up CI Other: - Finish off bug smash logging bugs - Quick bug fixes x3 Week of 30 September (planned): Take Time for: - team policy discussions - meetings x3 Roadmap: - add tor controller trace logging to diagnose stem hangs (#30901) - try nickm's suggested a simpler implementation - Sponsor 31 modularisation: disable relay mode when compiling - First cut: disable DirCache, DirPort, ORPort, and sets --disable-module-dirauth - If I stall on coding, finish sponsor 31 documentation that's assigned to me Other: - Reviews and revisions - Check CI & backports Week of 30 September (actual): Urgent: - Take Time for: - team policy discussions - meetings x3 Roadmap: - Sponsor 31 config refactor - control trace logging #30901 - preparation work - fix conflicting bugs - started work - Sponsor 31 modularisation: disable relay mode when compiling #31851 - Initial design and strategy to move forward - Preparation work - speed up CI Other: - Revisions - Check CI & backports - Quick bug fixes x2 - tor-relays moderation Nick: Week of 23 September (planned): - Keep an eye on blog comments - Finish edding circuit-padding document - Start a C style poll for the team - Meet about sponsor 31 on tuesday - attend ED tea time later Monday - Respond if needed to expected reviews on paper with Ian and Chelsea - Work on 042-must/should items - Review and merge Week of 23 September (actual): - Response to review feedback on paper with Ian and Chelsea - Handle blog spam on Tor posts - Reviews and merges - Sponsor-31 meeting - 31372: use "make -k" on CI - 31759: Improve "annotate_ifdef_directives" - 31849: move tor-guts into tor. Week of 30 September (planned): - One more PETS review. - Work on 042-must/should items - Start on another alpha, if all the -must items are done? - Kick off a C style poll - More review and feedback on pending refactoring - Review and merge stuff - Finish circuit-padding editing Mike: Week of 9/23 (planned): - Research discussion - Work with Nick to clean up final XXX's in circpad doc and merge/cite the quickstart guide - Cloudflare bot fight? - Maybe have time to start on Firefox review? <-- gaba: please or let pili know that you can't do it this week --> Week of 9/23 (actual): - Research discussion + emails - Wrapped up circpad doc (for now) - Wrote plan for circpad simulator (https://trac.torproject.org/projects/tor/ticket/31788#comment:2) - Attempted #31653; ran into unit test issues and also libevent perf issues - Started Firefox networking review (probably 75% done) Week of 10/02 (planned): - Finish Firefox networking review - Check in with Tobias re circpad perf issues and simulator plan - Perhaps update circpad doc with perf notes Need help with/at risk of dropping this month: <-- do you still need help with this? - Not doing any code reviews this month, unless on circpad patches. Got enough for Tor Browser - Deep-thought-required research project followup - (Google masque, BGP, ECN, Rob's bw experiments, Dennis's Mozilla video, etc etc...) - Relay community drive/mgmt (and related LTS herding) catalyst: week of 09/23 (2019-W39) (planned): - reviews - provide feedback about C style - more feedback to review guidelines thread as needed - more feedback re gitlab migration as needed - check in with swati on Summer of Docs project - #30984 week of 09/23 (2019-W39) (actual): - reviews - chatted with some people re PR review guidelines - reviewed some policy proposals - wrote some feedback about C style week of 09/30 (2019-W40) (planned): - reviews - #30984 - review policy proposals asn: Week of 23/09 (planned): - Figure out how to do key blinding in onionbalance. - Forward progress in #31369 now that atagar has given me feedback Week of 23/09 (actual): - Significant progress on OB key blinding (#31777). - Continue work on stem descriptor encoding (#31823). - Started thread to discuss onion service errors and how they should be handled: https://lists.torproject.org/pipermail/tor-dev/2019-September/014046.html - Wrote documentation for client auth: #28966 Week of 02/10 (planned): - Bring OB hsv3 descriptor encoding closer to completion. - Attend s27 meeting about error passing over SOCKS vs HTTP vs control port - Handle remaining 042 bugfixes. ahf: Week of 23rd of September (planned): - Do first round of 1:1:1 with Gaba and other people on the team. - Work on #31810 - Continue to work on #28930 - Friday is Gitlab day where I will continue with the migration project. Week of 23rd of September (actually): - Did first rounds of 1:1:1 with Gaba. - Productive Gitlab meeting with people from different teams. - Proposed patches for #31091 and #31810 Week of 30. September (planned): - Finish 0.4.2 tickets - Get back to S29 tickets - Work on migration scripts for Gitlab transition. - Take the good parts of our PR policy discussion and turn into a policy. - Network team retrospective. dgoulet (offline): Week of 02/10 (planned): - Go through my 042 tickets. - Assess s27 situation and timeline and prioritize work that need to happen soon. - Attend s27 meeting about the SOCKS error situation. - Bad relay world. swati (offline): Week of 02/10: - Registered and created an account for myself on trac. - Created a ticket https://trac.torproject.org/projects/tor/ticket/31943 for reviewing changes in the tor Manual. - Forked the tor repository and cloned the copy of the repository to my local filesystem - Pushed changes to the local repository and created a pull request. - The pull request is https://github.com/torproject/tor/pull/1391. - Can someone assign reviewers for this pull request? I don't seem to have write access for this repository. All the best, Alex. -- Alexander Færøy

1 0

Metrics team meeting notes, 3 Oct 2019
by Gaba 03 Oct '19

03 Oct '19

Meeting Log: http://meetbot.debian.net/tor-meeting/2019/tor-meeting.2019-10-03-15.01.html Meeting pad's content: Next meeting is on October 14th at 15UTC * Debian Buster and Java 11 (irl) karsten looks into #31953 and #31952 on Friday, and if he doesn't succeed, irl takes over next week * AWS + CloudFormation (irl) karsten tries out lightsail next time he creates an EC2 instance * Roadmap - how are we doing? (gaba) https://trello.com/b/Mu5fYg53/tor-metrics-roadmap -- Project Manager: Network, Anti-Censorship and Metrics teams gaba at torproject.org she/her are my pronouns GPG Fingerprint EE3F DF5C AD91 643C 21BE 8370 180D B06C 59CA BD19

1 0

Anti-censorship meeting notes, 3 Oct 2019
by Philipp Winter 03 Oct '19

03 Oct '19

Here's our meeting log: http://meetbot.debian.net/tor-meeting/2019/tor-meeting.2019-10-03-17.00.html And here's our meeting pad: Anti-censorship work meeting pad -------------------------------- Next meeting: Thursday October 3rd 17:00 UTC Weekly meetings, every Thursday at 17:00 UTC, in #tor-meeting at OFTC (channel is logged while meetings are in progress). == Goal of this meeting == Weekly checkin about the status of anti-censorship work at Tor. Coordinate collaboration between people/teams on anti-censorship at Tor. == Links to Useful documents == * Our anti-censorship roadmap: https://dip.torproject.org/torproject/anti-censorship/roadmap/boards * Our roadmap consists of a subset of trac tickets. * The anti-censorship team's wiki page: https://trac.torproject.org/projects/tor/wiki/org/teams/AntiCensorshipTeam * GetTor's roadmap: https://dip.torproject.org/torproject/anti-censorship/gettor/boards * Tickets that need reviews: https://trac.torproject.org/projects/tor/query?status=needs_review&componen… * Projects from sponsors we are working on: * https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor30 * https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor28 --------------------------- --- 3rd October 2019 --- --------------------------- == Announcements == * Looks like our new default bridge (just started shipping in the new Tor Browser alpha) is already blocked by the GFW * Do we know whether it happened before or after the release? Before = source code inspection (and they know about the moved tor-browser-build.git repo); after could be source code inspection or package inspection or black-box testing. * Unfortunately not; I only tested it just now * Add the date and time you tested to MetricsTimeline, if you would. * I will! * Also most of our new bridgeDB bridges seem to be blocked: https://trac.torproject.org/projects/tor/raw-attachment/ticket/31701/obfs4-… * Tor Browser files are appearing now at https://archive.org/details/@gettor for gettor purposes. * And also at https://drive.google.com/drive/folders/13CADQTsCwrGsIID09YQbNz2DfRMUoxUU == Discussion == * What research questions can we think of that we should pitch to students? * Study BridgeDB crawling by setting up a variety of bridges (use tor's BridgeDistribution option) and check if they end up getting blocked. * Understand how and whether benign third parties are using/scraping Tor bridges. Research project sketch here: https://bugs.torproject.org/30636#comment:33 * Measure reachability of our snowflake broker (and bridge?) * Reverse-engineer/study a DPI system https://github.com/net4people/bbs/issues/15 * Understand what's happening with reported Shadowsocks blocking in China since mid September. Active probing? Large-scale flow analysis? * Understand the WebRTC fingerprint of pion-webrtc, in the style of https://trac.torproject.org/projects/tor/wiki/doc/Snowflake/Fingerprinting. * Evaluate effectiveness of obfs4's simplistic "slap some padding onto application data" approach * How does sharknado's packet-burst-breaking approach compare? * What is the the "right" packet sending schedule (assuming no restrictions on efficiency)? Where a schedule is an algorithm that outputs a sequence like "wait 50 ms, send 200 bytes, wait 4100 ms, send 1460 bytes, ..." Start by proposing a strawman schedule to give us something to work with. Ideally I would like to see an adversarial analysis: let one team develop a scheduler (using their own collected traffic), and a different team evaluate it (using their own collected traffic). * torproject.org mirrors - what is the right strategy for accepting them? (conversation will continue in a ticket) == Actions == * == Interesting links == * == Updates == FORMAT! Name: This week: - What you worked on this week. Next week: - What you are planning to work on next week (related to anti-censorship work). Help with: - Something you may need help with. hiro: (2019-09-09)(gettor days are Thursday - snippets https://dip.torproject.org/snippets) - gettor was down due to a VM reboot. Phw added a systemd script to reboot the service. - add archive.org - add gdrive - edit ansible scripts - edit scripts to upload files to various distribution endpoints Next week - reach out to irl about sending gettor stats to metrics - include reviews from code and website changes - review specs: are specs up-to-date? should we change something in the specs? - review docs: write documentation for web site and ansible playbooks. Help with: - probably more reviews. hiro: (2019-09-02)(gettor days are Thursday - snippets https://dip.torproject.org/snippets) - Coded ansible recipes for gettor so that the service can be easily maintained by more people: https://dip.torproject.org/torproject/anti-censorship/gettor-project/gettor… - Fixing some issues about git history taking too much space quota on gitlab and github Next week - use archive.org as new distribution endpoint: upload files to archive.org - reach out to irl about sending gettor stats to metrics - review specs: are specs up-to-date? should we change something in the specs? - review docs: write documentation for web site and ansible playbooks. Help with: - waiting to be told that's fine to upload files to archive.org? Can we start? - review new website. New website should be reviewed. https://dip.torproject.org/torproject/anti-censorship/gettor-project/gettor… phw: This week (2019-10-03): * Created missing tickets for sponsor 30 tasks * Took 10 random bridges from all three BridgeDB pools and sent them to cohosh for testing * More work on obfs4 improvements * Made obfs4 save client state (i.e., random value), and derive probability distributions from it * Interacted with new obfs4 bridge operators * Finished updating BridgeDB's requirements.txt and documented a process for keeping it up-to-date. * Wrote monthly anti-censorship team report. * Filed and implemented https://bugs.torproject.org/31903 to request new BridgeDB translations * Implemented first version of BridgeDB language switcher: https://bugs.torproject.org/26543 * Updated existing BridgeDB metrics files to match our updated metrics spec. Next week: * Finish prototype for obfs4 flow obfuscator * Figure out plan for how to reduce per-packet entropy Help with: * Gaba: (updated October 3rd) Last week (): * sponsor 30 coordination This week (planned): * sponsor 30 triage of tickets * sponsor 30 meeting ahf Last week: - Worked on #28930 This week: - Finished refactoring parts of #28930. Trying to figure out if we should begin the discussion on how PT's can report back on bootstrap info. - Continued to work on a tool to convert Trac tickets into Gitlab tickets. cecylia (cohosh): last updated 2019-10-03 Last week: - grant outline for meeting and set up gitlab repo for it - started new obfs4 tests - displayed preliminary results for obfs4: https://trac.torproject.org/projects/tor/attachment/ticket/31701/obfs4-reac… - made major changes and squashed commits for sequencing layer (#29206) - helped sah with #31794 - started work on #31384 - so many reviews: #31391, #31685, #31537, #31780, #29484, #31794 - merged #31830, #31843 - talked with serna about #25598 This week: - make a patch for the proxy---broker communication (#29207) - more and better tests for #29206 (also related to #29259) - snowflake dogfood and think about how to address bad snowflake health - refactor proxy-pair state machine (#31310) - finish up #31384 - clear out any other september backlog - grant writing Help with: - review of #28942 and #29206 from dcf - feedback on whether the direction in #31384 is reasonable catalyst: week of 09/19 (planned): - reviews - sponsor31 planning - coding style discussion - comment on draft network team review guidelines - #30984 week of 09/19 (actual): - reviews - sponsor31 planning - talking with people about proposed network team review processes week of 09/26 (planned): - reviews - sponsor31 doc coordination - checking in on Season of Docs work - #30984 arlolra: 2019-09-26 Last week: - mia Next week: - add a build step / documentation for code reuse in cupcake - pick up another ticket (looking at #31497, #31685, #31537, #31765, #31028, #31310) Help with: - review of #31391 dcf: 2019-10-03 Last week: - posted Turbo Tunnel candidate protocol evaluation https://github.com/net4people/bbs/issues/14 - posted summary of FOCI paper on meek traffic analysis https://github.com/net4people/bbs/issues/13 - filed tickets for Golang security upgrade (#31889, #31890) Next week: - catch up on Snowflake tickets - review #29206 (sequencing protocol for Snowflake) - review #29206 (pion WebRTC) - archive test pion builds from #28942 - migrate Snowflake broker to a datacenter with IPv6 (#29258) Help with: - Need BridgeDB and meek-azure to redeploy meek-server for #31890

1 0

OONI Monthly Report: September 2019
by Maria Xynou 02 Oct '19

02 Oct '19

Hello, Below we share what the OONI team worked on in September 2019. ## Launch of revamped OONI Explorer We launched the revamped version of OONI Explorer! Access the new OONI Explorer here: https://explorer.ooni.org/ Along with the launch, we published a post that explains in detail what's new about OONI Explorer 2.0: https://ooni.org/post/next-generation-ooni-explorer/ We were interviewed by the Committee to Protect Journalists (CPJ), who published a Q&A where we discuss how journalists can use the new OONI Explorer: https://cpj.org/blog/2019/09/qa-ooni-explorer-team-unlock-the-webs-untold-c… VPNCompare also published a blog post about the new OONI Explorer: https://www.vpncompare.co.uk/ooni-explorer-uncovering-global-web-censorship/ ## Report on blocking of BBC and Alhurra in Egypt In collaboration with Egyptian community members, we published a research report documenting the blocking of BBC and Alhurra in Egypt. Our report is available here: https://ooni.org/post/2019-egypt-blocks-bbc-and-alhurra/ Mohamed El-Taher translated the report to Arabic: https://moeltaher.net/2019/09/26/مصر-تحجب-بي-بي-سي-والحرة-التوسع-في-الرق/ In the report, we share network measurement data on: * Blocking of BBC: https://explorer.ooni.org/search?until=2019-09-26&domain=www.bbc.com&probe_… * Blocking of Alhurra: https://explorer.ooni.org/search?until=2019-09-26&domain=www.alhurra.com&pr… We corroborated OONI measurements by performing manual curl tests with the help of our local partners, the findings of which we share in the report. We found that Egyptian ISPs are using Deep Packet Inspection (DPI) technology that is aware of TLS and which is most likely fingerprinting the SNI field of the TLS handshake. VPNCompare published a blog post covering our report: https://www.vpncompare.co.uk/ooni-bbc-blocked-egypt/ ## Updated test lists To help improve the quality of measurements, we updated the following test lists: 1. Algeria: https://github.com/citizenlab/test-lists/pull/517 2. Cameroon: https://github.com/citizenlab/test-lists/pull/508 3. Tanzania: https://github.com/citizenlab/test-lists/pull/520 4. Indonesia: https://github.com/citizenlab/test-lists/pull/512 5. Vietnam: https://github.com/citizenlab/test-lists/pull/516 As part of the process of updating the above test lists, we reviewed all of the URLs (i.e. deleted defunct URLs, replaced old URLs with new ones, added HTTPS where appropriate, etc.), fixed the category codes (i.e. changed category codes where required), and added new URLs based on relevant desktop research. We also made contributions to other test lists: https://github.com/citizenlab/test-lists/pull/514 https://github.com/citizenlab/test-lists/pull/511 https://github.com/citizenlab/test-lists/pull/510 https://github.com/citizenlab/test-lists/pull/509 https://github.com/citizenlab/test-lists/pull/506 https://github.com/citizenlab/test-lists/pull/502 https://github.com/citizenlab/test-lists/pull/501 https://github.com/citizenlab/test-lists/pull/500 ## Progress on implementing new OONI Probe nettests We made significant progress (80%) towards completing the implementation of new OONI Probe nettests. Specifically, we: * Added functionality to measure SNI-based blocking: https://github.com/ooni/netx/pull/11 * Completed the ndt7 client for OONI Probe: https://github.com/m-lab/ndt7-client-go/pull/29 * Re-wrote DASH to use HTTPS: https://github.com/neubot/dash * Extracted timing of network, TLS, HTTP events: https://github.com/ooni/netx/blob/111e220d770f289108af03fc3ee5fdb526e4906a/…, https://github.com/ooni/netx/blob/111e220d770f289108af03fc3ee5fdb526e4906a/…, and https://github.com/ooni/netx/blob/111e220d770f289108af03fc3ee5fdb526e4906a/… * Implemented DoT and DoH: https://github.com/ooni/netx/tree/master/internal/dnstransport/dnsovertcp and https://github.com/ooni/netx/tree/master/internal/dnstransport/dnsoverhttps * Got the content of DNS packets (including the DoT and DoH cases): https://github.com/ooni/netx/blob/111e220d770f289108af03fc3ee5fdb526e4906a/… ## Progress towards stable release of OONI Probe desktop app for Windows and macOS We made progress on getting the OONI Probe desktop app ready for the stable release. Specifically, we: * Completed the implementation of the Web Connectivity measurement screen for the OONI Probe desktop app: https://github.com/ooni/probe-desktop/pull/44 * Added support for dev tools to aid debugging: https://github.com/ooni/probe-desktop/pull/43 * Wrapped strings (that are to be translated) in Formatted message to support translations: https://github.com/ooni/probe-desktop/pull/41 * Upgraded the dependencies of the app to the latest version: https://github.com/ooni/probe-desktop/pull/40 * Fixed some bugs related to upgrade depedencies: https://github.com/ooni/probe-desktop/pull/39 & https://github.com/ooni/probe-desktop/pull/42 The overall progress and next steps towards the stable release of the OONI Probe desktop app can be tracked through our project board: https://github.com/orgs/ooni/projects/4 ## Progress towards stable release of OONI Probe Command Line Interface (CLI) As part of our work towards the stable release of the OONI Probe ClI, we: * Prepared the v0.1.0 release of probe-engine: https://github.com/ooni/probe-engine/pull/62 * Documented the milestones for the stable probe-cli: https://github.com/ooni/probe-cli/milestone/3 ## Progress on OONI Probe mobile apps We continued to make progress towards the public release of OONI Probe mobile v.2.2.0. Android: https://github.com/ooni/probe-android/pull/263 https://github.com/ooni/probe-android/pull/262 https://github.com/ooni/probe-android/pull/258 https://github.com/ooni/probe-android/pull/256 iOS: https://github.com/ooni/probe-ios/pull/308 https://github.com/ooni/probe-ios/pull/307 ## Progress towards reducing the latency of publication of measurements We continued to make progress towards reducing the latency of the publication of uploaded measurements. This can be viewed through our work on the fast-path: https://github.com/ooni/pipeline/pull/213 The fast-path code has been reviewed and it is currently deployed against active OONI Probe collectors. The remaining next step is to add support for returning the realtime measurements via OONI API queries, and to support the searching and listing of these measurements in OONI Explorer. This is work in progress: https://github.com/ooni/api/pull/115 ## Progress on data processing pipeline heuristics for flagging relevant blocking events for the RSS feed To flag relevant blocking events via the RSS feed in OONI Explorer, we created a new data pipeline with an internal web app and API for the publication of charts and we worked on integrating the blocking events into the database. The component implements a new heuristic to analyze past events and infer when URLs become blocked or unblocked with granularity of a country. Current branch and pull request: https://github.com/ooni/pipeline/pull/228 We expect to have the event engine running soon and we'll then work on publishing it on OONI Explorer. ## Ensuring that our infrastructure is re-deployable As part of our efforts to ensure that all our infrastructure is re-deployable, we made progress on the following: * The dom0 bootstrap script has been updated to work with Debian buster: https://github.com/ooni/sysadmin/pull/353 * We dropped the legacy UID scheme to make it easier to deploy machines: https://github.com/ooni/sysadmin/issues/289 * We upgraded matomo to a latest machine, redeploying it: https://github.com/ooni/sysadmin/pull/355 (fixes: https://github.com/ooni/sysadmin/issues/225 & https://github.com/ooni/sysadmin/issues/149) * Grafana was updated to the latest version: https://github.com/ooni/sysadmin/pull/360 The master ticket with all the hosts that need to be re-deployed is available here: https://github.com/ooni/sysadmin/issues/356 ## Improving the availability of our infrastructure We made progress on improving the availability of our infrastructure. Our plan on how to go forward has been documented in this ticket: https://github.com/ooni/sysadmin/issues/359 Our work on oonifying the probe services is available in this pull request: https://github.com/ooni/sysadmin/pull/361 So far, we have deployed a new web-connectivity test helper and we will oonify the bouncer, collector, and orchestra backends into the same host. ## Community use of OONI data ### OONI data supports court case in Pakistan Back in November 2017, we reported on the blocking of several media websites, Facebook (facebook.com and Facebook Messenger), YouTube, Twitter, Instagram, WhatsApp, and Telegram in Pakistan amid protests: https://ooni.io/post/how-pakistan-blocked-social-media/ Our report -- which shared OONI network measurement data collected from Pakistan -- was used as evidence in a petition filed in the Islamabad High Court by Pakistan's Media Matters for Democracy (http://mediamatters.pk/), challenging the blocking of media websites and popular social media platforms during the protests. The petition challenged the Pakistan Telecommunication Authority's (PTA) authority to block websites without due process. Details are available here: http://digitalrightsmonitor.pk/islamabad-high-court-takes-up-petition-filed… The petition was filed through Charahgar (http://mediamatters.pk/charahgar/), a legal aid centre that Media Matters for Democracy established with the support of Media Legal Defense Initiative (MLDI). As an outcome, the Islamabad High Court declared (in September 2019) that the Pakistan Telecommunications Authority's interpretation of Section 37 of PECA is against the Constitution, and that they cannot regulate content without following the due process, the principles of transparency, and oversight. Details are available here: https://tribune.com.pk/story/2061933/1-pta-not-empowered-block-website-ihc/ This is a landmark victory for internet freedom in Pakistan. ### Blocking of change.org in Cuba Through the use of OONI Probe and OONI data, YucaByte (a Cuban non-profit organization) published a report on the blocking of change.org in Cuba: http://www.yucabyte.org/confirman-bloqueo-changeorg-en-cuba/ Their report (including the use of OONI Probe) received the following press coverage: https://adncuba.com/noticias-de-cuba/tecnologia/confirman-bloqueo-de-la-pla… ### Venezuelan partners nominated for 2019 Gabo Award Through the use of OONI Probe and OONI data, IPYS Venezuela (along with Venezuela Inteligente / VEsinFiltro) has been monitoring internet censorship in Venezuela since 2014. They have created a network of volunteer OONI Probe users across Venezuela, who coordinate on measuring and reporting on emergent censorship events. Through the use of OONI data, they have published numerous research reports, documenting internet censorship events in Venezuela on an ongoing basis. Their Intercortados project (https://ipysvenezuela.org/intercortados/) has been nominated for the 2019 Gabo Award, among the 10 best innovation projects. Further information: https://premioggm.org/premio-gabo/edicion/2019/nominados/ https://premioggm.org/noticias/2019/09/conoce-a-los-40-nominados-al-premio-… https://twitter.com/Mariengracia/status/1172198403478040580 ## Community activities ### Community meeting We held our monthly community meeting on Slack (https://slack.openobservatory.org/) on Tuesday, 24th September 2019. During the meeting, we discussed: 1. Launch of revamped OONI Explorer: Update and questions 2. Query speed problem in API 3. OONI Probe testing in Egypt ## Userbase In September 2019, OONI Probe was run 417,684 times from 5,407 different vantage points in 218 countries around the world. This information can also be found through our stats: https://api.ooni.io/stats ~ The OONI team. -- Maria Xynou Research & Partnerships Director Open Observatory of Network Interference (OONI) https://ooni.torproject.org/ PGP Key Fingerprint: 2DC8 AFB6 CA11 B552 1081 FBDE 2131 B3BE 70CA 417E

1 0

trac into gitlab journey [3rd meeting]
by Gaba 01 Oct '19

01 Oct '19

Hey! The journey continues :) You can find agenda and notes in https://pad.riseup.net/p/e-q1GP43W4gsY_tYUNxf Logs are here: http://meetbot.debian.net/tor-meeting/2019/tor-meeting.2019-10-01-18.00.html The next meeting is October 15th at 1700 UTC Contents of the pad for the meeting of today: SPONSOR 30 MONTHLY MEETINGS References: mail with context: https://lists.torproject.org/pipermail/tor-project/2019-July/002407.html planning document: https://nc.riseup.net/s/SnQy3yMJewRBwA7 migration code & issues: https://dip.torproject.org/ahf/trac-migration ticket: https://trac.torproject.org/projects/tor/ticket/30857 Agenda October 1st - Review actions from last meeting experiment with moving tickets from legacy project into its own (ahf) rename torproject to tpo (gaba) Research on how to deal with spam in gitlab. (gaba) Write down a list of things that you will not see in gitlab from trac (ahf) Write down a more concrete plan for migration. What do we test for? When we do it? Who is helping? - (gaba) <-- it is in https://nc.riseup.net/s/SnQy3yMJewRBwA7 - What decision is missing to move forward? IRC ticket number bot redirection user registration - Next steps when do we do this move? Notes 1. experiment with moving tickets from legacy project into its own ahf experimented with it and "anarcat's solution" works NEXT: ahf will copy all tickets from trac into tpo/legacy project in gitlab. 2. rename torproject to tpo -> not renaming it but using tpo directly 3. spam in gitlab AHF's proposal: https://pad.riseup.net/p/MMEeAUdiUQ-XwhDV-Eww Many more proposals (trying to resolve spam issue, user registration, and anonymous users issue): debian salsa on top (for user registration) cerberus (ahf's proposal) a forum for people to send reports https://discourse.org Next steps: AHF does a trial run of a full migration of ticket (on Friday) Look at bugs.torproject.org for irc bot integration. (arma & ahf & anarcat) Write down proposal to have an instance of discourse (anarcat, gaba) -- Project Manager: Network, Anti-Censorship and Metrics teams gaba at torproject.org she/her are my pronouns GPG Fingerprint EE3F DF5C AD91 643C 21BE 8370 180D B06C 59CA BD19

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

tor-project