August 2017 - tor-project - lists.torproject.org

GSoC 2017 Ahmia Status Update #6
by Pushkar Pathak 22 Aug '17

22 Aug '17

Hello everyone, This is the bi-weekly status report for Ahmia - Hidden Service Search. ## Updated documentation and removed expired pages and links The documentation of Ahmia is now fully complete and available at ahmia.fi/documentation/. [1] ## Updates to 'about' page and other minor fixes in ahmia-site [2][3] ## Django 1.11 upgrade for Ahmia Currently Ahmia supports Django1.10. >> Future tasks - I am working on to add support for advanced search features but since final evaluation deadline is on 29 Aug, I have put this on hold to review the changes I already made and prepare a report I will get back to implement Thanks, Pushkar [1]: https://github.com/ahmia/ahmia-site/commit/d3af0b72d9b175c7d3d269119f4dceeb… [2]: https://github.com/ahmia/ahmia-site/commit/6ee614ad617b0f6202e20f139e6dded4… [3]:https://github.com/ahmia/ahmia-site

1 0

Network team meeting notes from 21 August
by Nick Mathewson 22 Aug '17

22 Aug '17

Hello, all! We had a meeting yesterday. Logs at http://meetbot.debian.net/tor-dev/2017/tor-dev.2017-08-21-16.59.html Notes below: ==== Network team meeting, 21 August 2017 (1700 UTC) Announcements and requests: - review-group-22 !!!!!1! - Out this week: mikeperry, isis, ahf, maybe isabela will miss the meeting - Less than one month in 0.3.2.x merge window - Maybe review the Montreal meeting schedule I just did this morning? [dgoulet] teor: - Last week: * Holidays in sunny northern Australia * endosome can do bidirectional circuit crypt and digest, next stop, streams! [asn:https://github.com/teor2345/endosome] - This week: * Catch up on emails * Catch up on PrivCount in Tor * Re-plan PrivCount Experimental for the next few months, as the last release went over time asn Last week: - We got #20657 merged upstream! That's a huge part of prop224!!! - Fixed a few bugs in the client-side branch (#17242) and it's now in needs_review for Nick. See #17242 for more details. - Found a few more #20657 bugs and fixed them in branch `ticket20657_032_05`. Also addressed #23110 as part of it. See last comment of #17242 for more details. - Pushed fix for #23233 -- bug found by haxxpop. - Updated the prop224 upstreaming pad with the latest tickets/branches: https://pad.riseup.net/p/LPVY6AL_prop224_upstreaming This week: - This week will be more prop224 testing, debugging and bug fixing. We know there are a few more bugs on our prop224 codebase that we need to fix and this is a good time to hunt them down. - Address any review points by Nick on #17242 etc. - I will be AFK tomorrow and most of Wednesday. Will be back on Thursday. Nick: Last week: - Away on vacation - Hacked up HTTP CONNECT support #22407 - Tried to say constructive things about censorship, speech, and technology. This week: - Review lots of things. - Another 0.3.1.x release? rc? - Catch up on emails - Write proposals about authority refactoring - Maybe, write a proposal about TLS 1.3 - Try to schedule a privcount-in-tor meeting. (first, get privcount designers + implementors. Then invite everyone else.) - Finish up event and error list for torlauncher improvements (w catalyst) dgoulet: Last week: - Vacation - I found ourselves an Ubuntu packager! :D [progress]: https://bugs.launchpad.net/ubuntu/+source/tor/+bug/1710753 This week: - Catch up on all the things. - Review #17242 - Do some more prop224 testing and bug fixing - Give some love to 031/032 bugs isabela: last week: -vacation this week: - I wont be at the meeting because I am still on vacation but back tomorrow (tues 22) - I have a question to the team regarding mcs/brade wish list for Tor Launcher progres bar: https://pad.riseup.net/p/tor-client-progress / TB team needs to start building the new experience: https://marvelapp.com/3f6102d so we need to know: 1. What the network team can do on this list? and 2. how long it will take? - I also have a question to isis (but I know they won't be online till after the 22nd)re:https://trac.torproject.org/projects/tor/ticket/15967 which is another dependency the TB team has in order to build that experience. (obs: their deadline to build it is EOM November). - Once I am back from vacation I plan on picking up some sponsor4 reports of completed deliverables - Got an answer from Nick re:PT community meeting organized by Internews in Boston. I will ask them what days are more important for us to participate to see how we can do it. catalyst: Last week (2017-W33): - bug triage - HR stuff - digging into bootstrap progress reporting - blog post fallout (#23270) - wrote some hopefully helpful things on membership proposal thread (+1 --dgoulet) This week (2017-W34): - more work on bootstrap/PT progress reporting pastly: - Porting forward cell tracing code dgoulet originally wrote in order to rerun KIST experiments with the KIST we intend to merge - I now want to take up dgoulet on his offer to let me unload KIST review responsibility. I'll always be around for questions, of course. - KIST ticket is #12541

1 0

oniongit update
by Silvia [Hiro] 21 Aug '17

21 Aug '17

Hey everyone, I just wanted to send a quick update regarding oniongit and gitlab in general. So a main issue when teams started testing oniongit was the possibility to automatically sync repos on oniongit w/ repos on torgit. The idea is always that torgit is pulled by oniongit. This way oniongit doesn't write to our git repositories, but we can run code reviews with updated branches. There are mainly two ways how this can be achieved. The first possibility is that torgit push to oniongit whenever master is updated (with a post-receive hook maybe). The other possibility is that a pipeline (gitlab CI feature) is set up on oniongit to pull periodically from torgit and update the master branch. Note that if you wanted the pipeline could also run a set of tests on the branch since this is basically a docker runner that does this on a separated server. I am in the process of documenting how this can be done. In the meantime please let me know if you'd like me to set it up for you. Another update that you might find interesting is the possibility to set up confidential issues (although some have rightly said that they do not feel comfortable having a web up handling confidential bugs). [Please see https://docs.gitlab.com/ce/user/project/issues/confidential_issues.html] An issue that was also raised since the beginning regarding Trac vs Gitlab is the possibility to have components vs keywords vs labels. In gitlab everything is done w/ labels, so keep that in mind if you are thinking to test issues tracking in oniongit. [More on: https://docs.gitlab.com/ce/user/project/labels.html] [More on gitlab flow features here: https://docs.gitlab.com/ce/workflow/README.html] Please let me know if you'd like to have an account on oniongit or if you'd like to report any problems that you have found. Talk soon, -hiro

6 9

GSoC 2017 - Crash Reporter for Tor Browser: Report #6
by Nur-Magomed 21 Aug '17

21 Aug '17

Hi everyone! This is my bi-week status report for GSoC project "Crash Reporter for Tor Browser". # Crash reporter client: - Cleaned repo, code changes are collected to few patches [1]; - Working on reproduible build of Tor browser with crash reporter: I spent much time setting up rbm, finally I've managed to fix problems and started building with enabled crash reporter. # Next steps: - Test crash reporter when server side is up - Add needed crash reporter fields to white list (that will be sending) - Make TB reproducible build with crashreporter * - Consider and make some options proposed in crash reporter wiki [2] * * maybe these tasks I'll tackle after gsoc Kind regards, Nur-Magomed (nmago) [1] https://github.com/nmago/tor-browser/commits/crdevc [2] https://trac.torproject.org/projects/tor/wiki/doc/crashreporter

1 0

GSoC 2017 - unMessage: Report #6
by Felipe Dau 19 Aug '17

19 Aug '17

Hi everyone! This is my report #6 for the unMessage GSoC project. # Updates Since the last report I continued working on #21 [0] and #33 [1]: - Finished asynchronous changes for element parsing in incoming transmissions [2] and closed #21 - Refactored some parts that deal with elements and added unit tests to check that unMessage creates the right elements for its different features [3] # What's next As the final submission starts soon, I am going to review what we did and prepare the summary for the project. Meanwhile, I am going to continue working on #33, writing more unit tests. Thanks, -Felipe [0]: https://github.com/AnemoneLabs/unmessage/issues/21 [1]: https://github.com/AnemoneLabs/unmessage/issues/33 [2]: https://github.com/AnemoneLabs/unmessage/issues/77 [3]: https://github.com/AnemoneLabs/unmessage/issues/76

1 0

Notes from August 17 2017 Vegas team meeting
by Karsten Loesing 17 Aug '17

17 Aug '17

Notes for August 17 2017 meeting: Alison: 1) support wiki sprint happened and new edits are up! next we will go through the old FAQ and pull what should go into the wiki, make a "top issues" section (eg "your clock is wrong), and write a more extensive "my antivirus is blocking Tor" article 2) Montreal meeting planning: scheduling emails have gone out. should we talk about Mike's meeting taxonomy ideas? (it can probably wait if it needs to) 3) Colin went on "The Complete Privacy and Security Podcast" (episode 40) and did great! 4) Will participate in fundraising sprint next week Steph: 1) Looking for local speaking trainers for a half day session in Montreal 2) Working on end of year fundraising campaign. Filling out prep survey with GR. Sprints start next week. 3) Worked w Philipp to publish onion survey blog post. Planning upcoming posts with Tommy including post on browser downloads over the past 6 months, a story on ISPs 4) Social media. 5) A few press inquiries. 6) Linda will be speaking at Codemasters in Bulgaria. 7) Multiple templates in multiple formats are done, will be uploaded to media.torproject.org https://trac.torproject.org/projects/tor/ticket/23248 8) Signup form w double opt in email confirmation are ready for donor focused newsletter. Now working with Isa, hiro, and GR to figure out how we can have HTML versions and an archive of sent mailings. Shari: 1) working with Giant Rabbit on getting end-of-year campaign started 2) talking with auditor, who is in the office this week 3) scheduled a trip to San Francisco for late September; gonna see Mitchell Baker Arturo: 1) Our OTF proposal got approved! 2) We released ooniprobe 1.1.5 on mobile with push notification support and a new net neutrality test called DASH. It didn't catch fire, but we still have to send the first notification to the 6000+ users that have notifications enabled. 3) Maria attended a Ford Foundation retreat in Maine this week 4) Made an iconset for the citizenlab category codes of URLs: https://github.com/citizenlab/test-lists/issues/202#issue-250349502 5) Leonid did some very important (and I should say also of excellent quality) work on re-organising and improving our dev ops tooling Karsten: 1) Made some progress with integrating sanitized web server logs into CollecTor and the Tor Metrics Library. 2) Cleaning up ExoneraTor a bit towards resolving database performance problems and integration into Tor Metrics website. 3) Hot-fixed Collector after temporary bridge authority outage.

1 0

GSoC 2017 - anon-connection-wizard: Report #4
by iry 17 Aug '17

17 Aug '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Tor people! The following is my bi-week status report for the anon-connection-wizard project. # anon-connection-wizard development ## features implemented - - default settings will be adjusted according to .torrc (anon-connection-wizard will "remember" user's last choices) - - Use Tor ControlSocket and cookie file authentication ## features improved - - switched from /usr/lib/python3.4 to /usr/lib/python3/dist-packages - - several bugs fixed - - switch from self-made .tmp file to python tmpfile - - the only torrc anon-connection-wizard will be used is /etc/torrc.d/anon-connection-wizard.torrc (all other .orig etc files are not needed anymore) ## links - - detailed discussion[0] [1] - - commits on Github [2] # related work - - repair_torrc() is created and serving as a standalone module to repair /etc/tor/torrc in Whonix # Next step - - integrate anon-connection-wizard with whonix-setup-wizard [3] - - a lot of GUI changes for better UX under the guidance of Tor UX team's paper[4] Thank you! Best, iry [0]: http://forums.kkkkkkkkkk63ava6.onion/t/graphical-gui-whonix-setup-wizard - -anon-connection-wizard-technical-discussion/650/280 [1]: http://phabricator.kkkkkkkkkk63ava6.onion/T699 [2]: https://github.com/irykoon/anon-connection-wizard/commits/master [3]: http://forums.kkkkkkkkkk63ava6.onion/t/upgrading-from-local-repository-f or-whonix-14-wiki-page/4121/12 [4]: http://forums.kkkkkkkkkk63ava6.onion/t/graphical-gui-whonix-setup-wizard - -anon-connection-wizard-technical-discussion/650/285 -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZcOOqAAoJEKFLTbxtzdU8894P+wbwsfwyItcjiPyULx58IdgN 393z3nW+JfWXRUWcd4LyLns8fR9FufrlNfccqxMCBAivYvDqKiBhFnfpxKbzMp1o rk6mbd+MK/LLLhRVpIvT+TSTspuO2VpY2ErcY8TBpvHzdyRdQNxeiIP8TqNImFHv 0Dw6lnJwRgOiZys076mpuBNg64l6Cn3QpnY7wObB+eVhaNaqORd3WC1dTVCj92Hv NhxbR0vsoYgCShbGG1+wml0DGgNqu3KIY1tr+ZwGCaJhOOVqGNVvZxtlNf+7ExQA OxNO2HtVZe/bMzXoq3ERBuahNqWPJygkAJrrikXp4sbKQOSCGSdTFLvg9qETXyKV Pn8ASAI+j6qFyugNYlD7yAvH3Hp+ZnyfNj2mygYwRWqAPSh4F5rd7PaSoatWTpY8 Yw2seR+tRVXZ1lQ3wj5SuPioQRKUYMkA9OPO6tgKy3EShX1MiWLqmOFIujkbUWM5 4/QMDnkDnW6vjC2uBtk2KZ5ZvPEX0jBiQOfTnKAqGzMFAyC4PeWfyqFReIT75Qac JMkMMQKrYz2a6EjvDixI5NWoe82JwtDdl5BN1l3oBKurmpzx1Svkai/YRB5+c34u 2KKZS2Z0dc5emob92wsTUjc417evvOvWUkfwINh3dcGc+e3ANSXdxojP3gd17Bbd mQKPVXDOO1aLklp9ZZ9A =+yel -----END PGP SIGNATURE-----

1 2

Tails report for July 2017
by sajolida 16 Aug '17

16 Aug '17

https://tails.boum.org/news/report_2017_07/

1 0

OONI Monthly Report: July 2017
by Arturo Filastò 15 Aug '17

15 Aug '17

Hi Tor! This is what OONI has been up to in July 2017. # OONI Monthly Report: July 2017 The OONI team had an exciting month, having had the opportunity to host the first OONI Partner Gathering and to attend the Citizen Lab Summer Institute (CLSI). We made a new release of our mobile apps, including push notifications to remind users to repeat tests regularly, and the integration of the DASH streaming test which is designed to measure the quality of tested networks by emulating a video streaming. We made significant progress on the re-engineering of our data processing pipeline and on probe orchestration. We also updated various test lists in collaboration with our partners, and we hosted our monthly community meeting. ## Organized and hosted the first OONI Partner Gathering Thanks to support from the OTF Community Lab, the Ford Foundation, and the Media Democracy Fund (MDF), we were able to host our first OONI Partner Gathering! The event was hosted at the University of Toronto on 10th & 11th July 2017. The aim of this two-day event was to bring our international partners together to share skills, knowledge, and research findings. We reflected on our collaboration over the last year, and we brainstormed on future goals and priorities to improve our collaboration on the study of internet censorship. We published a report which provides an overview of the event, its sessions, partner challenges and needs, future goals and priorities, and event outcomes. The report can be found here: https://ooni.torproject.org/post/ooni-partner-gathering-2017/ The event's agenda and session notes can be found here: https://github.com/OpenObservatory/gatherings/tree/master/partner-gathering… As part of the event, we interviewed some of our partners. Their interviews will be published on our website within the next month(s). ## New release: Push notifications and integration of the DASH streaming test into the ooniprobe mobile apps During July we made a new release of our mobile apps, including the following two exciting new features: * Push notifications * DASH streaming test Push notifications will allow us to send messages and alerts to OONI probes all around the world. This is a particularly important feature as it will allow us to communicate directly with probe users, provide instructions, and to encourage users to run specific tests when emergent censorship events are reported by local communities. We implemented a first type of notification that sends a text message and an array of URLs, when this notification is opened in the app, a custom web browser opens the first URL and if there is some connectivity error, the user can cycle to the next (to avoid possible censorship). This first push notification support for apps are only informative: it allows us to send messages to ooniprobe users and engage them to run the app more, in the next release there will be a support to run tests remotely. OONI's new DASH streaming test is designed to measure video streaming performance. DASH, in particular, is designed to measure the quality of tested networks by emulating a video streaming. This test can be useful for examining cases of throttling and potential net neutrality violations. We published a detailed description of the DASH test on our website, which can be found here: https://ooni.torproject.org/nettest/dash/ ## Progress on the re-engineering of our data processing pipeline This month we made a lot of progress on the data processing pipeline. * We migrated blockpages fingerprints to the new pipeline database format to make migration of OONI Explorer to the news DB format possible. * We tackled some differences in old and new pipeline results while running two pipelines side-by-side. * We started hardening distributed pipeline infrastructure with firewalls and better monitoring to prevent incidents. * We started the process of freeing unused computing resources (virtual machines) allocated to OONI infrastructure in OTF cloud. ## Progress on probe orchestration In the area of probe orchestration we made a series of very important improvements, namely: * We deployed to production probe orchestration with message alert push notifications * Released a new version of the ooniprobe mobile apps that supports receiving push notifications * Made various improvements and bugfixes to the orchestration system Learn more by checking out the Changelog: https://github.com/TheTorProject/proteus/blob/master/ChangeLog.md ## Updated test lists During July we created a new test list for Mali, in collaboration with our local partners. This test list can be found here: https://github.com/citizenlab/test-lists/pull/192 We also updated the Egyptian test list to include URLs that were recently reported to be blocked by locals. The updates can be found here: https://github.com/citizenlab/test-lists/pull/194 Furthermore, we added BBC domains to multiple country-specific test lists: https://github.com/citizenlab/test-lists/pull/193 ## Attended the Citizen Lab Summer Institute (CLSI) The OONI team had the opportunity to attend the Citizen Lab Summer Institute (CLSI) in Toronto from 12th-14th July 2017 (https://citizenlab.ca/summerinstitute/2017.html) As part of the event, we facilitated the following sessions: * Presentation of OONI as part of the "Measurement Methods and Tool Showcase" session (Day 2) * Introduction to Network Measurement Research (Day 2) * OONI Partnership Program (Day 2) * Internet Blackout Detection (Day 3) * Contextualizing Censorship Measurement Research (Day 3) As an outcome of our participation at the CLSI, we expect to form new partnerships, to better inform the development of our methodologies (particularly those related to measuring internet blackouts), and to create a community-driven database that contextualizes network measurement data. ### Community meeting We hosted our monthly community meeting on https://slack.openobservatory.org/ on 25th July 2017. As part of the meeting, we discussed the next steps for the development of a methodology for the automatic detection and examination of internet blackouts in regions of India. ## Userbase In July 2017 ooniprobe was run 65,331 times from 1,606 different vantage points across 158 countries around the world. This information can also be found through our stats here: https://measurements.ooni.torproject.org/stats ~ The OONI team.

1 0

Network team meeting, 14 August 2017
by Alexander Færøy 14 Aug '17

14 Aug '17

Hello! It's vacation season so a lot of people are out this week, but the remaining people in the network team had a short meeting today on IRC. The minutes from the IRC meeting can be found at http://meetbot.debian.net/tor-dev/2017/tor-dev.2017-08-14-17.00.html The pad content can be found here: --- snip --- Network team meeting, 14 August 2017 Announcements and requests: - Nick has opened review-group-22. Have a look! Please review things! - Nick's out of town this week. - After the meeting, somebody please send out a copy of this pad and a link to the logs for tor-project@ - How early should people arrive for Montreal anyway? [to team mailing list] Nick: * Last week * Hacks and merges on stuff for R, Q, V, etc. * Numerous bugfixes and cleanups * Grabbed lots of small tickets * Sent out proposal 280 (privcount in tor) * Wrote and sent out proposal 281 (downloading microdescriptors in smarter ways) * Collected lists of things we did for NSF and sponsor4. (ahf, isabela, I sent you email about the sponsor4 status stuff I did) * Wrapped up one last sponsor4 task (#23220) * Started writing proposals on authority refactoring, TLS 1.3 migration * This week * On vacation! (I'll see personal mail, and mostly ignore lists) * probably not hacking, unless I get bored in the car or something ;) * The week after * I'm planning to release 0.3.1.6-rc. Please help with bugfixing! haxxpop: * Last week - run a fuzzing test on an outer layer of hsv3 desc and found a bug (#23233) * This week - write an inner layer fuzzing of hsv3 desc and run it to find some bugs ahf Last week: Sponsor 8: - Went out and got a spare nexus 5 to test on. - Started collecting metrics via the dump system subsystem on Android (on device). - Gave up on the simulator: too slow and querky to do anything useful with except looking at the apps themselves. Misc: - Misc. reviews. - Went to SHA2017 - met a lot of nice Tor users, did a talk. - Tried to keep the next-gen HS running as david+george's prop 244 work. This week: Sponsor 8: - Start looking into Nick's list of timed events, start trying to minimize them, check if it has a direct effect on the measurements. Sponsor 4: - Go over Nick's comments to the prop 278 merge, send new one to review for when he's back from vacation. Misc: - Figure out how to get to montreal: is _all_ network team meetings set in stone now? Next week: - I have vacation to do BornHack! Mike Last week: - Recovering from back injury. This week: - Still recovering, and preparing for vacation - Going to try to get patches ready for #23077 and #23097 before I go. isis: Last week: - Reviewed Chelsea's protover - Wrote up more Ristretto specs - Spent ~1 day total mentoring the intern for the bridge bandwidth project This week: - Mostly AFK to go speak at RustConf catalyst: Last week (2017-W32): - chased down Roger's complaint about variable shadowing warnings from gcc post-hs-merge. Turns out there's an old gcc bug that made it complain about shadowing functions with variables. We should probably ignore those warnings because modern compilers are more discerning - more investigation about bridge bypass (#20532); blocked by some weird hard-to-reproduce issues on Tor Browser - some Montreal meeting logistics. Turns out air travel for me tends to require awkward routings now that I'm not in something that pretends to be an airline hub - met with Nick, Roger, and a few others (tbb-dev types?) about https://pad.riseup.net/p/tor-client-progress - community team sprint. Tried out a few GetTor things and opened tickets suggesting improvements This week (2017-W33): - make more progress on #20532 - make more progress on improving bootstrap progress/error reporting (some good ideas already noted in the pad) komlo: - Last week: - Came up with a rust kata for Montreal, will send that out soon if people want to start beforehand - This week: - Fixing up rust protover from Isis's review --- snip --- Cheers, Alex.

1 0