- tor-project - lists.torproject.org

GSoC 2017 - unMessage: Report #5
by Felipe Dau 04 Aug '17

04 Aug '17

Hi everyone! This is my report #5 for the unMessage GSoC project. # Updates Since the last report I continued working on #21 [0] and #33 [1]: - Stopped using queues and threads to check for incoming transmissions (that's when the other user starts an action) - Removed disk dependency to run tests without saving any files - Added integration tests to make sure things do not break when making low level changes for the unit tests # What's next Now that I have more confidence to make changes, I am going to start working on the unit tests. I also expect to review with my mentors what has been done for #21 and declare it done. More details on these tasks can be followed in the respective tickets [0, 1], branches [2, 3] and the latest PR [4]. Thanks, -Felipe [0]: https://github.com/AnemoneLabs/unmessage/issues/21 [1]: https://github.com/AnemoneLabs/unmessage/issues/33 [2]: https://github.com/felipedau/unmessage/tree/21/deferreds [3]: https://github.com/felipedau/unmessage/tree/33/feature/test-suite [4]: https://github.com/AnemoneLabs/unmessage/pull/72

1 0

Core Tor team sponsor4 July report
by isabela 04 Aug '17

04 Aug '17

Core Tor Team July 2017 Report In July we reviewed, triaged and fixed misc bugs related to releases 0.3.0 and 0.3.1: * Never send a consensus which the downloader already has. [1] This was an issue we identified that make the most trouble in test networks, where the consensus update rate is so fast that relays frequently try to download a consensus. We investigated and identified what caused the issue and fixed it. * LZMA coder causes crash when the sandbox is enabled. [2] While doing measurements for this project we noticed that Tor instances running as relays or authorities would sometimes crash when the sandbox is enabled. This is due to the MALLOC_MP_LIM value in sandbox.c, which is currently set to 16 MB, being too low. We limit our LZMA coder to only use 16 MB, but the coder allocates some additional data other than its internal buffer, which leads to the crash. * Investigated: Assertion failure in consensus_cache_entry_handle_get on Windows.[3] We created a mitigation and diagnostic branch. It doesn't fix this, but it makes it nonfatal and tries to get more useful info if it happens again. And receive a report that is still present in 0.3.1.4-alpha. Will continue investigation in August. * Bug: src/common/compress.c:576: tor_compress_process: [4] We fixed a bug found 0.3.1 at a stable Gentoo Linux server. * Bridge unavailable during differential consensus update [5] Bug reported for Raspberry Pi running Raspbian. we revised our threadpool implementation to prevent this background work from affecting other background work. There are 3 changelog entries for it in 0.3.1.5-alpha. ** Major bugfixes (relay, performance): Perform circuit handshake operations at a higher priority than we use for consensus diff creation and compression. This should prevent circuits from starving when a relay or bridge receives a new consensus, especially on lower-powered machines. Fixes bug 22883; bugfix on 0.3.1.1-alpha. ** Minor features (directory cache, consensus diff): Add a new MaxConsensusAgeForDiffs option to allow directory cache operators with low-resource environments to adjust the number of consensuses they'll store and generate diffs from. Most cache operators should leave it unchanged. Helps to work around bug 22883. ** Minor features (relay, performance): Always start relays with at least two worker threads, to prevent priority inversion on slow tasks. Part of the fix for bug 22883. Allow background work to be queued with different priorities, so that a big pile of slow low-priority jobs will not starve out higher priority jobs.This lays the groundwork for a fix for bug 22883. * zstd tests fail with libzstd 1.3.0 [6] Write zstd epilogues correctly when the epilogue requires reallocation of the output buffer, even with zstd 1.3.0. (Previously, we worked on 1.2.0 and failed with 1.3.0). * consdiff test fails on OS X [7] test_consdiff_base64cmp would fail on OS X because while OS X follows the standard of (less than zero/zero/greater than zero), it doesn't follow the convention of (-1/0/+1). Make the test comply with the standard. We started to land updates to dir-spec.txt for prop #278 in GL/ahf/torspec/tree/prop278-in-dir-spec [8] And concluded our review of lz4 compressor [9]. For that we added a simple LZ4 test to our own little benchmarking tool [10]. Our results [11] (results) showed that LZ4 is a lot faster than anything else we have (even beats gzip at compression level 1), but the compression ratio is also worse than anything else we have for the cached-consensusdocument (~2.2 MB of structured text). Since we are currently batch processing the compression of our "larger" files in the background we decided we wouldn't win much by including LZ4 here. For August we feel that we are pretty close from finishing this deliverable and we plan to publish a blog post about compression + consensus diff work done with this project. Hope to share that soon. [1] https://trac.torproject.org/projects/tor/ticket/22702 [2] https://trac.torproject.org/projects/tor/ticket/22751 [3] https://trac.torproject.org/projects/tor/ticket/22752 [4] https://trac.torproject.org/projects/tor/ticket/22719 [5] https://trac.torproject.org/projects/tor/ticket/22883 [6] https://trac.torproject.org/projects/tor/ticket/22927 [7] https://trac.torproject.org/projects/tor/ticket/22870 [8] https://trac.torproject.org/projects/tor/ticket/22275 [9] https://trac.torproject.org/projects/tor/ticket/22819 [10] https://gitlab.com/ahf/tor-sponsor4-compression/tree/master/bench [11] https://docs.google.com/spreadsheets/d/1WzFXQGfH8yI4WCCRAEQBt1Z_sC-3hMkmE6H…

1 0

July 2017 Report for the Tor Browser Team
by Georg Koppen 04 Aug '17

04 Aug '17

Hi! In July the Tor Browser team made 4 releases, Tor Browser 7.0.2[1], Tor Browser 7.0.3[2], Tor Browser 7.5a2[3], and Tor Browser 7.5a3[4]. All four releases were out-of-bound releases fixing security issues. Tor Browser 7.0.2 and Tor Browser 7.5a2 included updated Tor versions (0.3.0.9 and 0.3.1.4-alpha respectively) and Tor Browser 7.0.3 and Tor Browser 7.5a3 fixed a potential proxy bypass bug.[5] The latter two were Linux-only releases as no other platform was affected by the problem, which got reported in our newly launched bug bounty program provided by HackerOne.[6] Non-release work focused on both regressions due to our transition to Firefox 52 ESR[7][8] and on Sponsor4 work: we got Selfrando ready to be shipped in 32bit Linux alpha bundles[9] and are about to switch to our new reproducible build system. The upcoming alpha series should be the last one that is built with Gitian, which is exciting news. The full list of tickets closed by the Tor Browser team in June is accessible using the `TorBrowserTeam201707` keyword in our bug tracker.[10] In August we plan to finally switch to rbm as our new tool for our reproducible builds which then allows us to start working on Tor Browser for 64bit Windows systems, which is due later this year[11]. Then we hope to get our work on our Panopticlick system done so that we can make use of it for fingerprinting statistics. Finally, we'll put some energy on redesigning Tor Launcher with help from the UX team and network team mainly to make it easier for censored users to get the bridges they need to reach the Internet.[12] All tickets on our radar for this month can be seen with the `TorBrowserTeam201708` keyword in our bug tracker.[13] Georg [1] https://blog.torproject.org/blog/tor-browser-702-released [2] https://blog.torproject.org/blog/tor-browser-703-released [3] https://blog.torproject.org/blog/tor-browser-75a2-released [4] https://blog.torproject.org/blog/tor-browser-75a3-released [5] https://trac.torproject.org/projects/tor/ticket/23044 [6] https://hackerone.com/torproject [7] https://trac.torproject.org/projects/tor/ticket/22343 [8] https://trac.torproject.org/projects/tor/ticket/22618 [9] https://trac.torproject.org/projects/tor/ticket/20848 [10] https://trac.torproject.org/projects/tor/query?status=closed&keywords=~TorB… [11] https://trac.torproject.org/projects/tor/ticket/20636 [12] https://trac.torproject.org/projects/tor/ticket/21951 [13] https://trac.torproject.org/projects/tor/query?status=accepted&status=assig…

1 0

Notes from August 3 2017 Vegas team meeting
by Karsten Loesing 04 Aug '17

04 Aug '17

Notes for August 3 2017 meeting: Alison: 1) Worked on further refining support wiki. Wrote a blog post about the support wiki that Steph will publish sometime next week. The post makes it clear that the support wiki is a dynamic document that will get better with more feedback, and I'm hoping to get feedback in the post comments that we can incorporate into the wiki during our support wiki sprint on August 11. 2) Onboarded Steph about Tor/FOSS culture and history. When I get some time I'm going to clean up my notes and turn that document into an actual onboarding doc. Feedback would be great! 3) Worked on the code of conduct draft some. Now wondering what's up with the membership guidelines? 4) Spoke about onion services for archives at the Society of American Archivists Conference last week. 5) Began organizing the Tor Speakers Bureau. Almost 30 people have signed up! 6) Worked on some grant stuff with the comms team. 7) Worked on some Tor Meeting planning stuff. Next week's Vegas meeting will be all about planning the Tor Meeting schedule. Georg: 1) Who should go to the OTF Summit? (Discussed for a while. Isabela will reach out to teams and individuals, aiming to reply to OTF RSN) Mike: 0) Working to get Prop247 (hidden service guard selection) performance experiments running. Will need to coordinate with Karsten for running a few onionperf instances. 1) Goal is to get both Prop247 and Prop254 implementations working well enough for experiments to run while I am on vacation. Going to be ignoring most other non-critical things. Nick: 0) Nothing seems to be on fire with us. 1) Didn't get hs-ng merged for defcon, but it *is* working. Code still under review. Hoping we'll merge by the 0.3.2 deadline (15 Sep.) 2) Had to push back 0.3.1 stable, since it wasn't ready for the start of august. Hoping for start of september. Not currently planning corresponding delays in 0.3.2 stable deadlines. 3) I'm trying to push the privcount work forward. 0.3.2 merge for some in-tor trial component seems doable, pending prompt replies from nrl folks. 4) Got useful feedback from mcs et al on usability features in progress reporting for Tor Launcher. Karsten: 1) Double-checked Tor Browser initial download numbers without finding anything unusual. iwakeh might continue this analysis next week. Isabela: 0) worked with Tommy and Metrics folks on OTF response 1) Worked with Erin, Tommy and Steph on 'onboarding wiki' - https://trac.torproject.org/projects/tor/wiki/org/onboarding (we organized the information people brainstormed into sections and started populating those but there are still lots of work to do on this page) 2) Got deliverables reports out - getting july reports ready to be send before i go on vacation 3) great progress with tor launcher ui changes - now working on dependencies and copy / also lots of progress overall with ux work, did an update no all our projects at this week meeting (you can see on meetings notes to ux list) 4) did some prep work for sponsor8 5) worked on a proposal for a collaborative way to build tor meeting schedule Steph: 1) Still working with Giant Rabbit on getting the newsletter setup. Slowly making process. Also working with them to move press contacts/orgs into Civi. Starting to move contacts in there now 2) Maintaining blog calendar and working with authors + Tommy on posts - lined up: MOSS award, Canada events, Orfox, Support Wiki, Onion Survey, Browser download metrics 3) Getting in more of a flow with press inquiries. We answered a couple questions for TheQuestion.ru, they'll be on the English site as well. Connected Arthur with a presentation. Will coordinate promotion of Colin's podcast 4) Have made a little headway with templates. Need to make odf versions. We'll have as a resource in media.tp

1 0

Plan to double-check Tor Browser initial download numbers
by Karsten Loesing 02 Aug '17

02 Aug '17

Hello list, it's been almost two years since we started collecting sanitized Apache web server logs. During this time the number of Tor Browser initial downloads rarely went below 70,000 per day. https://metrics.torproject.org/webstats-tb.html Either there must be a steady demand for fresh binaries, or there is a non-zero number of bots downloading the Tor Browser binary several times per day. I already double-checked our aggregation code that takes sanitized web server logs as input and produces daily totals as output. It looks okay to me. I'd also like to double-check whether there's anything unexpected happening before the sanitizing step. For example, could it be that there are a few IP addresses making hundreds or thousands of requests? Or are there lots of requests with same referrers or common user agents indicating bots? My plan is to ask our admins to temporarily add a second Apache log file on one of the dist.torproject.org hosts with the default Apache log file format without the sanitizing that is usually applied. A snapshot of 15 or 30 minutes would likely be sufficient as sample. I'd analyze this log file on the server, delete it, and report my findings here. This message has two purposes: 1. Is this approach acceptable? If not, are there more acceptable approaches yielding similar results? 2. Are there any theories what might keep the numbers from dropping below those 70,000 requests per day? What should I be looking for? Thanks! All the best, Karsten

5 10

June 2017 grants report
by Tommy Collison 01 Aug '17

01 Aug '17

Hello Tor! I call this a grants report, but I do other writing-y stuff as well. ### July 2017 - Wrote and submitted three new grants, two to MDF and one to DRL. - Edited the final report for the MOSS grant, which will also go out as a blogpost in the next week or two. - Wrote two blogposts. [1] [2] - Researched a bunch of new grants. - Worked on the onboarding wiki. - Worked on UX/UI stuff. - Worked on a grant submission system, which'll make it easier to apply for grants. [1] https://blog.torproject.org/blog/tor-joins-online-day-action-net-neutrality [2] https://blog.torproject.org/blog/de-anonymization-smart-homes-and-erlang-at… As always, you can email me if you have questions or comments. -TC p.s. I'll be in San Francisco August 16-30; hope to see some of you!

1 0

Sukhbir's July 2017
by Sukhbir Singh 01 Aug '17

01 Aug '17

Hi, In July, I worked on the following: * Tor Messenger - Completed the transition to ESR52 including switching to the new build system that uses runc, and rebasing the patches. We are now building on Tor Browser tag tor-browser-52.2.0esr-7.0-1-build1. https://gitweb.torproject.org/tor-messenger-build.git/log/?h=esr52 We are now reusing most of the build components from Tor Browser thanks to the power of rbm! - This month, the focus is to pick tickets that we think are critical and then close them before the end of the month, working towards the 0.5.0b1 release. * TorBirdy Prepared and tested TorBirdy release 0.2.3 that will be released in the coming week. -- Sukhbir

1 0

UX team summary for July 2017
by Linda Naeun Lee 01 Aug '17

01 Aug '17

Hi all! July was another good month for UX. :) I gave a talk to PETS 2017 about my paper ( https://petsymposium.org/2017/papers/issue3/paper2-2017-3-source.pdf) which evaluated the usability of Tor Launcher. I got a great reaction from the audience, most of which seemed to buy the "usability is important!" message. Some OTF people were in the audience, and they seemed to understand how critical usability work is! We were encouraged to apply for a grant to get some funding for UX work, so Isa and Tommy are following through on that in August. Isa and I worked in person while at PETS to information architect what will go on the home portal for torproject.org. We know it's taking a while, and people are wary of attempts to improve the site. But we're quite serious and persistent! It's just not funded at the moment and getting our spare cycles. We finished redesigning Tor launcher (https://marvelapp.com/3f6102d/) to better guide users through configuring their network settings. Some of the changes made were: 1) combining all the configuration options into a single screen, as inspired by the in-browser network settings menu 2) clarifying which countries will need to configure bridges on the first screen 3) incorporating the bridgeDB service into the interface. In addition we started other work, such as: - potential changes to the tor browser toolbar to give easier access to security controls - messages and indicators for different states when visiting a .onion site (http + .onion, for instance) - user testing for onionbrowser to validate recent changes to their onboarding sequence Cheers, Linda N. Lee Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2

2 1

Reviving the 'Trac discussion' - next meeting on July 11th
by isabela 01 Aug '17

01 Aug '17

Hi everyone, It has been a while since we started evaluating if we want to find alternatives to Trac. You might remember we sent a survey out to collect more info, and based on the answers we believe we should figure out a better solution. https://lists.torproject.org/pipermail/tor-project/2017-March/000975.html https://lists.torproject.org/pipermail/tor-project/2017-March/000978.html In Amsterdam meeting we hosted a discussion on the results of our survey and up to this moment we have been evaluating gitlab as a possible alternative to Trac or eventually as a possible code review tool we can use. But this has been a little 'loose' and we wanted to organize things better in order to be able to make decisions and move forward. Therefore we are thinking of breaking this into 3 'tasks' we want to cover for development: * continue integration - jenkins is doing this now for us (only used for internal contributors tho, would be nice to think of a way external contributors could use it / maybe setting up travis for them) * code review - We have a test gitlab instance running at https://oniongit.eu [or emuo4mf6vwghcaqn.onion]. Network team has accounts on it and we would like to have more people testing it. Please bear in mind that this is a test machine, it is not backed up, and can be slow on occasions. * issue tracker - could be gitlab or another solution, we are still looking into how to solve this one (yes, the wiki is not on this list for now) So! We would like to propose the following moving forward: 1. Meet on irc to answer any questions on this new approach and get more people trying our gitlab testing installation [MEETING IS ON JULY 11TH TUESDAY AT 1400 UTC ON #tor-project channel] 2. Set up a 'end date' for our gitlab testing phase - that is why we would like more folks trying it out 3. Look into a plan to provide CI for external contributors (using travis maybe?) 4. Create a list of requests for what we need for issue tracker and see what to do about that We believe that covering the points above will help us move forward with this project. Please reach out to hiro if you'd like to have access to the gitlab test set up. Any questions/feedback is welcome as always. thanks! isabela and hiro o/

3 5

Notes from 31 July Network Team Meeting
by Nick Mathewson 31 Jul '17

31 Jul '17

Hi, all! We had another network team meeting today. The log is at http://meetbot.debian.net/tor-dev/2017/tor-dev.2017-07-31-17.00.html Notes from the pad are below. Network team pad, for 31 July Meeting (or 1 August, for those in UTC+5 or later) Notes from last week's meeting: * https://lists.torproject.org/pipermail/tor-project/2017-July/001323.html (Did we do what we had planned?) Announcements and reminders: * Montréal hackfest pad is at Things we should talk about: * Has there been progress on review-group-21 ? * I saw some coverity issues last week but no coverity master. Did they get handled? * We should assign roles for August: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/TeamRot… * Isa wonders if we should pursue gitonion because of it not working for folks who are using TB at high security level * What is blocking 0.3.1.x-stable? Tasks for after the meeting: * nickm: fix tor_parse_long regression * everyone: take roles for august * everyone: answer on oniongit email thread! teor (not online): Last week: * Worked with Nick on a spec and draft C code for PrivCount blinding in production tor * Made crypto_rand_double() randomise all the mantissa bits, and remove a slight bias #23061 * Deploy experimental PrivCount for Single Onion Service counts * Start work on a circuit sampling feature for experimental PrivCount, because python can be slow This week: * More PrivCount testing * Maybe re-deploy some additional experimental PrivCount features * Maybe get cell crypto working in endosome Nick: Last week: * Worked with Teor on getting the privcount blinding and tabulation system into Tor. This simplifies privcount a lot because events no longer need to get marshalled and unmarshalled, and makes Tor expose much less information that it does now (or did, under previous privcount branch) * Lots of code review on prop224 stuff. * Misc review and fixes * Merge workaround for some coverity madness; get rewarded with 14 coverity issues; resolve 10 of them. * Delayed 0.3.1.x-stable by a month. This week: * More work on privcount: get spec draft finished with Teor; share with other privcount people and with sponsor Q folks. * Try to finish review on 20657 (prop224 service-side stuff) * Release an 0.3.1.x-alpha * Other hacking TBD dgoulet: Last week: * Service implementation under review in #20657. * Client implementation++ in #17242. I've actually almost exclusively only worked on that. * Prop224 upstream bug opened: #23056 This week: * Finalizing and hopefully putting #17242 in review for upstream. * Assisting asn as needed on nickm's review of #20657. komlo (offline): This week: * Refactoring and getting ready for code review for protover rust * Thinking abuot creating a "intro to rust" coding exercise for the rust hackday ahf Last week: Sponsor 8: - Went over Catalyst + Nick's control port proposal and looked into the control port protocol. - Created a wiki page for network team sponsor 8 planning/notes: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Sponsor8 - Wrapped my head around the Android emulator, but I doubt we will be able to use that for much metrics collections. The android-x86 project might be more relevant here. - Got a test apk to run where I could test Android's "Doze" feature. Misc: - Reviewed #23030, #22883, #22927, #22915, and #20247 - Got a bit excited about asn+dgoulet's prop224 work and got a service running with an ircd to test it out. This week: Sponsor 4: - Fix remaining issues that is missing for Sponsor 4 and 0.3.1. - Write blog post about compression + consensus diff. Sponsor 8: - Look into Android-x86 for a "fast" non-device environment. - Start collecting "Doze" metrics from Orbot to see how we do. - (ahf, please document hwo to do the above stuff as you go along, so that other folks can try it too? -nickm) Misc: - Traveling to SHA2017 at the end of the week. asn: Last week: - Reviewed and tested more #20657 code. - Started fixing up prop224 service-side #20657 based on Nick's review (about halfway there) - Started reviewing and testing David's client-side #17242 branch. - Some hackerone triaging. This week: - Finish up fixing #20657 branch. - Review and test the client-side branch. - Fix more prop224 bugs - Write some unittests for client-side prop224. Mike: Last week: - Finished tor patches for pinning layer2 and layer3 guards (#13837) - Started work on stem-based prop247 prototype/performance simulator This week: - Finish prop247 prototype; Start onionperf testing catalyst: Last 2 weeks (2017-W29, 2017-W30): - moved house - looked into error reporting in test suite in #22636 - started catching up on email and Trac - Tor launcher automation meeting This week (2017-W31): - await arrival of household furniture, etc. - write up some notes to follow up on Tor launcher stuff - file ticket for test suite error reporting issues i noticed in #22636 and write up some of my observations about them - resume attempting to get chutney to do something useful to simulate #20532 isis: last week: - reviewed #22885 - revised my patch for showing info about cert expirations #17639 and wrote a test for it - revised travis configs and got them merged #22636 - fixed a whole lot of brokenness resulting from upgrading the BridgeDB machine to Debian 9 (#22998 #23032 #23033 #23034) - this week: - revising the new captcha server for moat #15967 - working on standardisation and finishing up the crypto needed for hyphae #22775 - updating #16562 with notes from meeting with trevor and reading the generalised edDSA specs/conversations Isabela: Last week: - finished deliverables report for Sue - worked with Erin on onboarding wiki page (thanks for the brainstorming) - submitted modularization proposal and got contract for sponsor8 proposal! - had meeting about tor launcher new design - trying to make sure all teams/people related to it, coordinates between themselves for networkers, TB team will give a wish list for part of the experience that they have to build, so you can review and tell what is possible what is not possible) This week: - july report for sponsor4 - prep work for sponsor8 - prep work for trac/git meeting (which got moved to August7!) - reminder about vacation time (august 7 till 14 - work on 15 - vacation again from august 16 till 21)

1 0