- tor-project - lists.torproject.org

Core Tor Team work for sponsor 4 in April
by isabela 16 May '17

16 May '17

Core Tor Team April 2017 Report: We finished the remaining tasks related to the implemention of Proposal 140 [1], "Provide diffs between consensuses". * #21646 - Prop140/compression: Refactor "directory request" code [2] * #21647 - Prop140: directory caches cache multiple past diffs or consensuses [3] * #21869 - Labeled storage backend for consensus docs and diffs [4] * #21648 - Prop140: Caches generate diffs as appropriate [5] * #22096 - Increase MALLOC_MP_LIM in sandbox.c [6] We continue to work on Proposal 278 [7], "Directory Compression Scheme Negotiation", finishing the following tasks: * #21664 - Prop278: Make the current 'torgzip' module a submodule of a new 'compression' module [8] * #21662 - Prop278: Add support for LZMA2 and/or Zstandard [9] * #22066 - Add memory measurement code in LZMA and Zstandard compression backends [10] * #21665 - Prop278: Establish an upper-bound for LZMA2 memory usage [11] * #22051 - Use streaming compression API in the non-streaming compression API [12] * #21663 - Prop278: Refactor the torgzip module to support additional compression schemes [13] * #22065 - Prop278: Parse the Accept-Encoding header and pass it to "get" handlers [14] * #21667 - Prop278: Handle new headers in directory.c [15] * #22085 - Refactor and simplify compression tests [16] We also hosted a follow-up meeting with Metrics Team regarding the impact of these efforts on the their work. Notes from this meeting can be found at: https://pad.riseup.net/p/chikeuBN1OUh [1] https://gitweb.torproject.org/torspec.git/tree/proposals/140-consensus-diff… [2] https://trac.torproject.org/projects/tor/ticket/21646 [3] https://trac.torproject.org/projects/tor/ticket/21647 [4] https://trac.torproject.org/projects/tor/ticket/21869 [5] https://trac.torproject.org/projects/tor/ticket/21648 [6] https://trac.torproject.org/projects/tor/ticket/22096 [7] https://gitweb.torproject.org/torspec.git/tree/proposals/278-directory-comp… [8] https://trac.torproject.org/projects/tor/ticket/21664 [9] https://trac.torproject.org/projects/tor/ticket/21662 [10] https://trac.torproject.org/projects/tor/ticket/22066 [11] https://trac.torproject.org/projects/tor/ticket/21665 [12] https://trac.torproject.org/projects/tor/ticket/22051 [13] https://trac.torproject.org/projects/tor/ticket/21663 [14] https://trac.torproject.org/projects/tor/ticket/22065 [15] https://trac.torproject.org/projects/tor/ticket/21667 [16] https://trac.torproject.org/projects/tor/ticket/22085

1 0

Accidental email triggered during drupal migration
by Silvia [Hiro] 16 May '17

16 May '17

Hi everyone, Unfortunately during a migration, some emails were triggered asking old tor-blog users to activate their accounts. Please ignore that. And sorry for the inconvenience. -silvia/hiro

1 0

Notes from weekly network team meeting, 15 May 2017
by Nick Mathewson 15 May '17

15 May '17

Hello! Our weekly meeting logs are available here: http://meetbot.debian.net/tor-dev/2017/tor-dev.2017-05-15-16.59.html Below is a copy of the pad where we wrote our status reports: ===== Network team status updates, 2017 May 15 FEATURE FREEZE TODAY* Discussion topics: - How have the stabilization periods worked historically ? [AHF] - And how should they work this time? [nm] - #21969 seems important! What to do?[asn] - upcoming 0.3.0 and 0.3.1 releases [nm] - hackfest date is June 12 through June 16 -- we might miss isis and aaron j. with those should we try for 26-30 week? - Bootstrap doc? #22265 ======== Updates ======== dgoulet is out this week. nickm: Last week - worked on reviewing merging a whole lot of tickets - worked on consensus compression and storage - today: just review and merge. - Need somebody to review my pending 0.3.1 patches, today if possible? This week - new 0.3.0 and 0.3.1 releases ahf: Last week: Sponsor4: - Prop#278 changes, testing and finishing. - Reviewed various patches and some spec changes. - Helped out with an issue in compression issue that affected moria1 This week: - Get things in for 0.3.1 freeze today :-) - Stabilization period? - Look into next Sponsor4 tasks? pastly: vacation Sebastian: Doing some light rust stuff. Would still appreciate review on the autoconf stuff asn: This week: - Prepared skeleton implementation of prop224 client-side desc fetching #21403. Code needs more tests, docs, beauty. - Put #22006 in needs_review after addressing Yawning's review. - Revised the e2e #21859 branch based on David's comments. It's now ready to be integrated to the larger service-side branch. - Tested the TROVE-2017-002 patch and confirmed it fixes the bug. - Posted a plausible theory in #21969 about what the bug could be (see last comment). - Did some SponsorR bookkeeping Next week: - More work on client-side descriptor fetching/caching #21403. - More work on #21969. - More work on guard discovery stuff. komlo: - Last week - Finished protover_compute_vote, getting closer to a full protover Rust implementation - Helped out with #21846 - This week - Continue protover Rust implementation (hopefully should be done soon) - Look into overriding CString to use malloc and free Mike: Last week: - Reviewed on netflow testing ticket issues - Worked with Isabela and Isis on Bridge discovery plans + UX - Concerned about Guard discovery hackfest This Week: - Reviewing bootstrap doc - Planning lots of travel - Padding catalyst: - Last week * [Coverity] #22221 (CID 1405983) fixed dead code in tests by swapping two loops * [Coverity] #22231 (CID 1397192) an old issue that i figured out the underlying problem for * found that a Linux VM with 2GB RAM seems to make --enable-fragile-hardening behave better in the test suite; was previously getting OOM-killed in slow/crypto/s2k_pbkdf2 * [sponsorM] my obfs4 boostrap failures probably related to #21969 and directory guard stuff. added some comments and uploaded a log. * [sponsorM] reviewing nickm's writeup about bootstrap state modeling; also correspondence with Linda draft at https://pad.riseup.net/p/OroadsO1qBgA * [sponsorM] reproduced jump-to-80% on Linux with obfs4 and existing state. also some 120 second timeout before giving up on a down bridge? maybe this should be shorter? * [sponsorM] opened #22232 to gather info about how Tor Launcher uses the control channel; also read some of the Tor Launcher implementation and have some confirmation that it uses the raw progress/phase numbers as percent completion on the progress bar - This week * draft a wiki page with a high-level overview of the bootstrapping process (#22265) * outline some incremental changes to bootstrap progress reporting that can make a large UX improvement * look some more into #21969 * bug triage

1 0

Linda in Stockholm this Tues-Friday
by Linda Naeun Lee 15 May '17

15 May '17

I am going to the Stockholm internet forum. I will be checking email periodically but may miss meetings.

1 0

We are upgrading our blog!
by Silvia [Hiro] 15 May '17

15 May '17

Hi all, In the next few days we will be in the process of upgrading blog.torproject.org. While we get prepared to refurbish the space, the current blog will be frozen for new content, starting now. This includes posts and comments and, if everything goes as planned, it should only a few days. Thanks in advance for your patience. See you on the other side. -hiro/silvia

1 0

Notes from May 11 2017 Vegas team meeting
by Karsten Loesing 15 May '17

15 May '17

Notes for May 11 2017 meeting: Nick: 1) Wrapping up 0.3.1.x features; on to bugfixes next. 2) Progress wrt personnel Alison: 1) Lots of grantwriting 2) Trying to make it to Seattle at the end of this month/beginning of next. 3) LFP just initiated a big survey to many of our participating libraries to see what they've done wrt privacy practices and policies. 4) Community team continues to work on support portal content. 5) The trainers we're working with in Pakistan just wrapped up a big youth summit in Peshawar. Waiting for the report back on that! 6) Community team members will be helping with the mirror list redesign (#22178) Arturo: 1) Releasing tomorrow a new version of ooniprobe mobile with 4 additional languages: Arabic, Farsi, Russian, Greek 2) Wrote/published OONI team status report for April 3) We now have much better monitoring of OONI infrastructure based on prometheus with alerting to slack, IRC and email when pieces of the infrastructure go down 4) On Sunday Maria and Arturo are going to be speaking at Open Source Conference Albania (https://oscal.openlabs.cc/) Shari: 1) Reviewing applications for Communications Director. Setting up initial interviews for next week. 2) Preparing for board meeting. 3) Begun researching strategic planning firms. 4) Edited grant proposals. 5) Getting ready for Tommy (writer) to start. Lining up things for him to do. 6) Preparing for a bunch of people coming to Seattle in a couple of weeks. Mike: 1) Met with Isa an Isis about bridge distribution 2) Discussed new hires for DRL with Isa; we still need a little budget clarity 3) Working on padding branches Isabela: 1) sent reviewed DRL proposal to Laura - sent out deliverables to stakeholders involved in the proposal 2) working on organize network team hackfest 3) met with mike and isis on building the experience for tor launcher automation (notes to come) 4) writing sponsor4 april report (little late with this one) 5) hiro should freeze the blog today and we will start the migration to the new blog - we will have a blog post up and email tor-project about that 6) linda is working on wireframes of the new portals with antonella and will be at SIF next week 7) my article on coding rights (anti-vigilancia) newsletter was published, is about how to use orbot and orfox. Lots of ppl at cryptorave liked it a lot because they heard all day about tor and appreciate to learn how to use it on mobile. (will tweet it from torproject is in PT and ES) 8) synced with Brad and others on things we can invoice for sponsorR and sponsor4 9) still on prep-work to get the job posts out - hopefully pads for job descriptions will be send out by EOW Georg: 1) we are about to release another alpha (probably the last one before Tor Browser 7.0) to test our latest fixes 2) preparations for SIF 3) thought a bit more about our upcoming work regarding the bridge distribution problem (Sponsor4, obj 2.2) Karsten: 1) We'll likely complete fourth of six Sponsor X deliverables by the end of this week. 2) We're going to have iwakeh as backup operator for CollecTor really soon. 3) We're planning to put out metrics-lib 1.7.0 soon and 2.0.0 in June.

1 0

GSoC17 Student Introduction
by Pushkar Pathak 13 May '17

13 May '17

Hello everyone! I am Pushkar, one of GSoC17 accepted students. I am a third year undergraduate at International Institute of Information Technology, Hyderabad (India). I have been working on 'Ahmia - Hidden Service Search' [0][1] for some time now and will be extending my contribution through GSoC this summer. I am being mentored by Juha Nurmi (numes) and George (asn). Ahmia is a search engine that indexes, searches, and catalogs content published on Tor Hidden Services. Furthermore, it is a medium to share meaningful insights, statistics, and news about the Tor network itself. There are several improvements and upgrades required in Ahmia. *>>Tasks * ● Automate Blacklisting Fetch a list of child abuse media sites and remove these sites from Elasticsearch. Also add MD5 checksums of child abuse websites to banned database for others to check. ● Add Hidden Services page Improve the existing Add page so that adding a website stores the data to SQL Database under '/onionsadded'. From there crawler can crawl these websites once a day. Remove the entries after 1 week so that the list is fresh . ● Data visualization Graphs need to plotted for various statistics in the Statistics page. Some examples include: ○ Linking structure between sites and keyword based labeling for onions in the graph - ○ Popularity of domains according to backlinks and search clicks. I plan to use either Google Charts or D3.js to plot these graphs. ● Replace Polipo with Tor Socks5 proxy in ahmia-crawler As of now Ahmia crawler uses Polipo as an HTTP proxy to direct tor traffic. But since Polipo is now no longer maintained and torsocks can provide better functionality, the crawler code needs to be updated to use torsocks. Modules like socksipy can be used to connect crawler to torsocks. ● Upgrade support from Elastic 2.4.0 to 5.X Ahmia settings should be adjusted accordingly to support Elastic 5.X. It will require a full cluster restart since rolling upgrades are not supported in major version upgrade. Upgrading includes replacing Groovy scripts with Painless. Painless is sandboxed and a Elasticsearch targeted scripting language which replaced Groovy in Elastic 5.0.0. ● Detailed Documentation and update software dependencies A detailed documentation at ahmia.fi[0] as well as on the Github page[1]. ● Advance search options - Advance Search options as mentioned below can be incorporated in search bar to allow better customisable searches. ○ Double quotes(""): Returns pages that contain exactly "term"(case sensitive). ○ AND operator(&&): Logical AND gate i.e. it returns all the pages that contain all queries separated by ‘&&’. ○ OR operator(||): Logical OR gate i.e it returns all the pages that contain queries separated by ‘||’. This is one of the optional tasks I have included. If any of the features mentioned above is not completed in the given timeline, this feature will be dropped and priority will be given to the uncompleted task. >>Timeline Week 1 - Automating blacklisting of onions with child abuse content Week 2 - Tweaking 'Add' page to save the added onion under '/onionsadded' Week 3 - Replace Polipo with Torsocks5 in ahmia-crawler Week 4 - 1st Evaluation Week 5+6 - Data visualization of statistics Week 7 - Upgrade support from Elastic2.4.0 to Elastic5.X Week 8 - 2nd Evaluation Week 9 - Updating dependancies and documentation Week 10 - Adding advanced search options like "",|| and && Week 11 - Catch up and bug fixes I will be mailing biweekly status report to this list. Feel free to contact me if you have any suggestions or doubts. IRC: mdhash I would like to thank Juha and the Tor team for their constant support and guidance. It has been a great experience for me to contribute to TorProject and I look forward to be a core member of the community. Thanks, Pushkar Pathak [0]: https://ahmia.fi [1]: https://github.com/ahmia

2 1

Crash Reporter for Tor Browser - GSoC 2017 Intro
by Nur-Magomed 12 May '17

12 May '17

* Didn't change the title in previous letter, sorry :/ Hi everyone! My name is Nur-Magomed, I’m 4th year student from North-Caucasus Federal University, Russia. This summer, I will be working on Crash Reporter for Tor Browser as a part of GSoC 2017. My mentors are: Tom Ritter - Primary Mentor, Georg (gk) - Backup Mentor. At the beginning I would like to thank all Tor dev community and my mentors that helped me and accepted my proposal, I’m glad to be a part of Tor dev community! About project Crash Reporter will be helping developers to improve Tor Browser, find bugs and crash reasons easier, that would make Tor Browser more stable and user-friendly. Mozilla Firefox has the crash reporter based on Google BreakPad with server side - Mozilla Socorro. My summer work is focused at adapting Firefox Crash Reporter for Tor Browser (Linux version). And also adapting Socorro for changed Crash Reporter and run it as “.onion” service. Full project proposal can be found at blog [1] or in PDF file [2]. Project timeline GSoC coding period starts on May 30 and ends on August 21 comprising a total of 12 weeks: week 1 - 5 : Crash Reporter client side | +--- week 1+2 : Build Tor Browser with Crash reporter. Improving crash data minidumps generator (don’t save privacy data in reports) | +--- week 3 : Redesign report’s client UI form, making functionality to delete data after sent, removing not necessary UI functions | +--- week 4 : Adapting report’s sender for Tor network (sending through Tor network) | +--- week 5 : Testing and review code week 6 - 11 : Crash Reporter server side based on Socorro | +---week 6+7 : working on Collector, adapting it according to changed minidumps | +---week 8+9 : updating DB structure, adapting Processor (for DB and for Collector’s reports) | +---week 9+10 : Working on statistic site - updating design templates, ORM classes for DB, and etc. | +---week 11 : Testing and review code. Setting up service on Tor network. week 12: Final testing, documentation Before coding will start there are some tasks to do (I'm starting work): > be sure which ones of fields or of their combinations could be privacy-sensitive > find the best way that explains to user what happened and what he can do after crash (in UI design) > try build Tor Browser with Crash Reporter from Firefox [for Linux] I will send bi-weekly status reports to this list, and also public it on my blog [1]. Please don’t hesitate to contact me if you have any suggestions. E-Mail: nmagoru [at] gmail.com IRC (OFTC): nmago Time Zone: UTC+03:00 Kind regards, Nur-Magomed Links: [1] https://torcrashreporter.wordpress.com/2017/04/03/google-summer-of-code- proposal-crash-reporter-for-tor-browser/ [2] https://torcrashreporter.files.wordpress.com/2017/05/final_proposal.pdf

2 1

Re: [tor-project] Welcome Nur-Magomed!
by Nur-Magomed 12 May '17

12 May '17

Hi everyone! My name is Nur-Magomed, I’m 4th year student from North-Caucasus Federal University, Russia. This summer, I will be working on Crash Reporter for Tor Browser as a part of GSoC 2017. My mentors are: Tom Ritter - Primary Mentor, Georg (gk) - Backup Mentor. At the beginning I would like to thank all Tor dev community and my mentors that helped me and accepted my proposal, I’m glad to be a part of Tor dev community! About project Crash Reporter will be helping developers to improve Tor Browser, find bugs and crash reasons easier, that would make Tor Browser more stable and user-friendly. Mozilla Firefox has the crash reporter based on Google BreakPad with server side - Mozilla Socorro. My summer work is focused at adapting Firefox Crash Reporter for Tor Browser (Linux version). And also adapting Socorro for changed Crash Reporter and run it as “.onion” service. Full project proposal can be found at blog [1] or in PDF file [2]. Project timeline GSoC coding period starts on May 30 and ends on August 21 comprising a total of 12 weeks: week 1 - 5 : Crash Reporter client side | +--- week 1+2 : Build Tor Browser with Crash reporter. Improving crash data minidumps generator (don’t save privacy data in reports) | +--- week 3 : Redesign report’s client UI form, making functionality to delete data after sent, removing not necessary UI functions | +--- week 4 : Adapting report’s sender for Tor network (sending through Tor network) | +--- week 5 : Testing and review code week 6 - 11 : Crash Reporter server side based on Socorro | +---week 6+7 : working on Collector, adapting it according to changed minidumps | +---week 8+9 : updating DB structure, adapting Processor (for DB and for Collector’s reports) | +---week 9+10 : Working on statistic site - updating design templates, ORM classes for DB, and etc. | +---week 11 : Testing and review code. Setting up service on Tor network. week 12: Final testing, documentation Before coding will start there are some tasks to do (I'm starting work): > be sure which ones of fields or of their combinations could be privacy-sensitive > find the best way that explains to user what happened and what he can do after crash (in UI design) > try build Tor Browser with Crash Reporter from Firefox [for Linux] I will send bi-weekly status reports to this list, and also public it on my blog [1]. Please don’t hesitate to contact me if you have any suggestions. E-Mail: nmagoru [at] gmail.com IRC (OFTC): nmago Time Zone: UTC+03:00 Kind regards, Nur-Magomed Links: [1] https://torcrashreporter.wordpress.com/2017/04/03/google-summer-of-code-pro… [2] https://torcrashreporter.files.wordpress.com/2017/05/final_proposal.pdf

1 0

GSoC 2017 - Introduction: unMessage
by Felipe Dau 11 May '17

11 May '17

Hi everyone! I am Felipe, one of the students accepted into GSoC this year. I am a Computer Engineering student at the Federal Technological University of Parana (UTFPR) in Brazil, studying Anonymous Communication on a research project lead by David R. Andersen [0], who is also my advisor. Since I joined the project three years ago, we have mostly worked on pyaxo [1], nymserv [2], nymphemeral [3], and recently, unMessage [4] - check them out! unMessage is a peer-to-peer instant messenger that provides privacy and anonymity to its users. In order to achieve that, it uses the Double Ratchet Algorithm [5] for encryption, Onion Services [6] for transport metadata minimization and its own protocol for application metadata minimization. unMessage is still in alpha stage, there are many things to explore, research and develop, and improving it is the GSoC project I am going to work on while mentored by dawuud [7] and meejah [8]. Specifically, we are going to: - Improve its setup script [9] - Use attrs [10] - Support file transfer [11] - Add a logger [12] - Make functions/methods asynchronous [13] - Add a test suite [14] Depending on the development of each task, it is possible that there will be more time available to: - Use automat [15] - Support offline messages [16] Although we selected a couple of user features, most of the tasks are architecture centered to increase its quality and maintainability. unMessage has potential to become a great app the community can benefit from, providing useful features and a good user experience for private and anonymous communication - this project is a big step in this direction. More information can be found in the GSoC proposal [17], documentation [18], GitHub page [4] and issues tracker [19]. Feel free to contact me as well: OFTC: dau Email: dau(a)riseup.net PGP: 0xC5A49047572A0D47 Having this opportunity means a lot to me. I am looking forward to start working with the Tor community and continue to do so after the summer. Thank you very much! I would like to thank David R. Andersen, who has been my advisor since we started working together, developed unMessage with me and has always been supportive. I also would like to thank Patrick Schleizer, HulaHoop and meejah for the support and contributions since the first unMessage release. This project has only made possible because of you! Thanks, -Felipe [0]: https://github.com/rxcomm [1]: https://github.com/rxcomm/pyaxo [2]: https://github.com/rxcomm/nymserv [3]: https://github.com/felipedau/nymphemeral [4]: https://github.com/AnemoneLabs/unmessage [5]: https://whispersystems.org/docs/specifications/doubleratchet [6]: https://www.torproject.org/docs/hidden-services.html [7]: https://github.com/david415 [8]: https://github.com/meejah [9]: https://github.com/AnemoneLabs/unmessage/issues/35 [10]: https://github.com/AnemoneLabs/unmessage/issues/34 [11]: https://github.com/AnemoneLabs/unmessage/issues/12 [12]: https://github.com/AnemoneLabs/unmessage/issues/30 [13]: https://github.com/AnemoneLabs/unmessage/issues/21 [14]: https://github.com/AnemoneLabs/unmessage/issues/33 [15]: https://github.com/AnemoneLabs/unmessage/issues/36 [16]: https://github.com/AnemoneLabs/unmessage/issues/32 [17]: https://gist.github.com/felipedau/8b48c6fde875e673f62d7569a27f254a [18]: https://unmessage.readthedocs.io [19]: https://github.com/AnemoneLabs/unmessage/issues

1 0