August 2017 - tor-project - lists.torproject.org

GSoC 2017 - Crash Reporter for Tor Browser: Report #5
by Nur-Magomed 05 Aug '17

05 Aug '17

Hi everyone! This is my bi-week status report for GSoC project "Crash Reporter for Tor Browser". # Crash reporter client: - We considered carefully crash report fields and made decision to reinterpret blacklist feature as allow list [1]; - removed crash event files (some kind of log files that are keeping a few locally crash data even if we sent them - are not the same as crash report files) keeping functional[2] Next steps: # Using new RBM[3] we trying to build Tor browser alpha with crashreporter and make it reproducible # After that we need to test it with server side Kind regards, Nur-Magomed (nmago) [1] https://github.com/nmago/tor-browser/commit/1b3bf799aec4327e99eb0fe0365a637… [2] https://github.com/nmago/tor-browser/commit/0bb8072c03cbf311db2f71d59fdbb6c… [3] https://gitweb.torproject.org/builders/tor-browser-build.git/

1 0

New Website for Nyx (aka arm)
by Damian Johnson 04 Aug '17

04 Aug '17

Hi all, I'm delighted to announce Nyx has a new home! https://nyx.torproject.org/ For those who remember Tor Cloud this is stylistically based on it. Feedback welcome! Tested with Firefox and Chromium. Passes W3C validation and works without JS (though no FAQ). I'll replace arm's old page (https://www.atagar.com/arm/) with a redirect after a while unless folks can think of a reason to keep the old site around. Cheers! -Damian

1 0

GSoC 2017 - unMessage: Report #5
by Felipe Dau 04 Aug '17

04 Aug '17

Hi everyone! This is my report #5 for the unMessage GSoC project. # Updates Since the last report I continued working on #21 [0] and #33 [1]: - Stopped using queues and threads to check for incoming transmissions (that's when the other user starts an action) - Removed disk dependency to run tests without saving any files - Added integration tests to make sure things do not break when making low level changes for the unit tests # What's next Now that I have more confidence to make changes, I am going to start working on the unit tests. I also expect to review with my mentors what has been done for #21 and declare it done. More details on these tasks can be followed in the respective tickets [0, 1], branches [2, 3] and the latest PR [4]. Thanks, -Felipe [0]: https://github.com/AnemoneLabs/unmessage/issues/21 [1]: https://github.com/AnemoneLabs/unmessage/issues/33 [2]: https://github.com/felipedau/unmessage/tree/21/deferreds [3]: https://github.com/felipedau/unmessage/tree/33/feature/test-suite [4]: https://github.com/AnemoneLabs/unmessage/pull/72

1 0

Core Tor team sponsor4 July report
by isabela 04 Aug '17

04 Aug '17

Core Tor Team July 2017 Report In July we reviewed, triaged and fixed misc bugs related to releases 0.3.0 and 0.3.1: * Never send a consensus which the downloader already has. [1] This was an issue we identified that make the most trouble in test networks, where the consensus update rate is so fast that relays frequently try to download a consensus. We investigated and identified what caused the issue and fixed it. * LZMA coder causes crash when the sandbox is enabled. [2] While doing measurements for this project we noticed that Tor instances running as relays or authorities would sometimes crash when the sandbox is enabled. This is due to the MALLOC_MP_LIM value in sandbox.c, which is currently set to 16 MB, being too low. We limit our LZMA coder to only use 16 MB, but the coder allocates some additional data other than its internal buffer, which leads to the crash. * Investigated: Assertion failure in consensus_cache_entry_handle_get on Windows.[3] We created a mitigation and diagnostic branch. It doesn't fix this, but it makes it nonfatal and tries to get more useful info if it happens again. And receive a report that is still present in 0.3.1.4-alpha. Will continue investigation in August. * Bug: src/common/compress.c:576: tor_compress_process: [4] We fixed a bug found 0.3.1 at a stable Gentoo Linux server. * Bridge unavailable during differential consensus update [5] Bug reported for Raspberry Pi running Raspbian. we revised our threadpool implementation to prevent this background work from affecting other background work. There are 3 changelog entries for it in 0.3.1.5-alpha. ** Major bugfixes (relay, performance): Perform circuit handshake operations at a higher priority than we use for consensus diff creation and compression. This should prevent circuits from starving when a relay or bridge receives a new consensus, especially on lower-powered machines. Fixes bug 22883; bugfix on 0.3.1.1-alpha. ** Minor features (directory cache, consensus diff): Add a new MaxConsensusAgeForDiffs option to allow directory cache operators with low-resource environments to adjust the number of consensuses they'll store and generate diffs from. Most cache operators should leave it unchanged. Helps to work around bug 22883. ** Minor features (relay, performance): Always start relays with at least two worker threads, to prevent priority inversion on slow tasks. Part of the fix for bug 22883. Allow background work to be queued with different priorities, so that a big pile of slow low-priority jobs will not starve out higher priority jobs.This lays the groundwork for a fix for bug 22883. * zstd tests fail with libzstd 1.3.0 [6] Write zstd epilogues correctly when the epilogue requires reallocation of the output buffer, even with zstd 1.3.0. (Previously, we worked on 1.2.0 and failed with 1.3.0). * consdiff test fails on OS X [7] test_consdiff_base64cmp would fail on OS X because while OS X follows the standard of (less than zero/zero/greater than zero), it doesn't follow the convention of (-1/0/+1). Make the test comply with the standard. We started to land updates to dir-spec.txt for prop #278 in GL/ahf/torspec/tree/prop278-in-dir-spec [8] And concluded our review of lz4 compressor [9]. For that we added a simple LZ4 test to our own little benchmarking tool [10]. Our results [11] (results) showed that LZ4 is a lot faster than anything else we have (even beats gzip at compression level 1), but the compression ratio is also worse than anything else we have for the cached-consensusdocument (~2.2 MB of structured text). Since we are currently batch processing the compression of our "larger" files in the background we decided we wouldn't win much by including LZ4 here. For August we feel that we are pretty close from finishing this deliverable and we plan to publish a blog post about compression + consensus diff work done with this project. Hope to share that soon. [1] https://trac.torproject.org/projects/tor/ticket/22702 [2] https://trac.torproject.org/projects/tor/ticket/22751 [3] https://trac.torproject.org/projects/tor/ticket/22752 [4] https://trac.torproject.org/projects/tor/ticket/22719 [5] https://trac.torproject.org/projects/tor/ticket/22883 [6] https://trac.torproject.org/projects/tor/ticket/22927 [7] https://trac.torproject.org/projects/tor/ticket/22870 [8] https://trac.torproject.org/projects/tor/ticket/22275 [9] https://trac.torproject.org/projects/tor/ticket/22819 [10] https://gitlab.com/ahf/tor-sponsor4-compression/tree/master/bench [11] https://docs.google.com/spreadsheets/d/1WzFXQGfH8yI4WCCRAEQBt1Z_sC-3hMkmE6H…

1 0

July 2017 Report for the Tor Browser Team
by Georg Koppen 04 Aug '17

04 Aug '17

Hi! In July the Tor Browser team made 4 releases, Tor Browser 7.0.2[1], Tor Browser 7.0.3[2], Tor Browser 7.5a2[3], and Tor Browser 7.5a3[4]. All four releases were out-of-bound releases fixing security issues. Tor Browser 7.0.2 and Tor Browser 7.5a2 included updated Tor versions (0.3.0.9 and 0.3.1.4-alpha respectively) and Tor Browser 7.0.3 and Tor Browser 7.5a3 fixed a potential proxy bypass bug.[5] The latter two were Linux-only releases as no other platform was affected by the problem, which got reported in our newly launched bug bounty program provided by HackerOne.[6] Non-release work focused on both regressions due to our transition to Firefox 52 ESR[7][8] and on Sponsor4 work: we got Selfrando ready to be shipped in 32bit Linux alpha bundles[9] and are about to switch to our new reproducible build system. The upcoming alpha series should be the last one that is built with Gitian, which is exciting news. The full list of tickets closed by the Tor Browser team in June is accessible using the `TorBrowserTeam201707` keyword in our bug tracker.[10] In August we plan to finally switch to rbm as our new tool for our reproducible builds which then allows us to start working on Tor Browser for 64bit Windows systems, which is due later this year[11]. Then we hope to get our work on our Panopticlick system done so that we can make use of it for fingerprinting statistics. Finally, we'll put some energy on redesigning Tor Launcher with help from the UX team and network team mainly to make it easier for censored users to get the bridges they need to reach the Internet.[12] All tickets on our radar for this month can be seen with the `TorBrowserTeam201708` keyword in our bug tracker.[13] Georg [1] https://blog.torproject.org/blog/tor-browser-702-released [2] https://blog.torproject.org/blog/tor-browser-703-released [3] https://blog.torproject.org/blog/tor-browser-75a2-released [4] https://blog.torproject.org/blog/tor-browser-75a3-released [5] https://trac.torproject.org/projects/tor/ticket/23044 [6] https://hackerone.com/torproject [7] https://trac.torproject.org/projects/tor/ticket/22343 [8] https://trac.torproject.org/projects/tor/ticket/22618 [9] https://trac.torproject.org/projects/tor/ticket/20848 [10] https://trac.torproject.org/projects/tor/query?status=closed&keywords=~TorB… [11] https://trac.torproject.org/projects/tor/ticket/20636 [12] https://trac.torproject.org/projects/tor/ticket/21951 [13] https://trac.torproject.org/projects/tor/query?status=accepted&status=assig…

1 0

Notes from August 3 2017 Vegas team meeting
by Karsten Loesing 04 Aug '17

04 Aug '17

Notes for August 3 2017 meeting: Alison: 1) Worked on further refining support wiki. Wrote a blog post about the support wiki that Steph will publish sometime next week. The post makes it clear that the support wiki is a dynamic document that will get better with more feedback, and I'm hoping to get feedback in the post comments that we can incorporate into the wiki during our support wiki sprint on August 11. 2) Onboarded Steph about Tor/FOSS culture and history. When I get some time I'm going to clean up my notes and turn that document into an actual onboarding doc. Feedback would be great! 3) Worked on the code of conduct draft some. Now wondering what's up with the membership guidelines? 4) Spoke about onion services for archives at the Society of American Archivists Conference last week. 5) Began organizing the Tor Speakers Bureau. Almost 30 people have signed up! 6) Worked on some grant stuff with the comms team. 7) Worked on some Tor Meeting planning stuff. Next week's Vegas meeting will be all about planning the Tor Meeting schedule. Georg: 1) Who should go to the OTF Summit? (Discussed for a while. Isabela will reach out to teams and individuals, aiming to reply to OTF RSN) Mike: 0) Working to get Prop247 (hidden service guard selection) performance experiments running. Will need to coordinate with Karsten for running a few onionperf instances. 1) Goal is to get both Prop247 and Prop254 implementations working well enough for experiments to run while I am on vacation. Going to be ignoring most other non-critical things. Nick: 0) Nothing seems to be on fire with us. 1) Didn't get hs-ng merged for defcon, but it *is* working. Code still under review. Hoping we'll merge by the 0.3.2 deadline (15 Sep.) 2) Had to push back 0.3.1 stable, since it wasn't ready for the start of august. Hoping for start of september. Not currently planning corresponding delays in 0.3.2 stable deadlines. 3) I'm trying to push the privcount work forward. 0.3.2 merge for some in-tor trial component seems doable, pending prompt replies from nrl folks. 4) Got useful feedback from mcs et al on usability features in progress reporting for Tor Launcher. Karsten: 1) Double-checked Tor Browser initial download numbers without finding anything unusual. iwakeh might continue this analysis next week. Isabela: 0) worked with Tommy and Metrics folks on OTF response 1) Worked with Erin, Tommy and Steph on 'onboarding wiki' - https://trac.torproject.org/projects/tor/wiki/org/onboarding (we organized the information people brainstormed into sections and started populating those but there are still lots of work to do on this page) 2) Got deliverables reports out - getting july reports ready to be send before i go on vacation 3) great progress with tor launcher ui changes - now working on dependencies and copy / also lots of progress overall with ux work, did an update no all our projects at this week meeting (you can see on meetings notes to ux list) 4) did some prep work for sponsor8 5) worked on a proposal for a collaborative way to build tor meeting schedule Steph: 1) Still working with Giant Rabbit on getting the newsletter setup. Slowly making process. Also working with them to move press contacts/orgs into Civi. Starting to move contacts in there now 2) Maintaining blog calendar and working with authors + Tommy on posts - lined up: MOSS award, Canada events, Orfox, Support Wiki, Onion Survey, Browser download metrics 3) Getting in more of a flow with press inquiries. We answered a couple questions for TheQuestion.ru, they'll be on the English site as well. Connected Arthur with a presentation. Will coordinate promotion of Colin's podcast 4) Have made a little headway with templates. Need to make odf versions. We'll have as a resource in media.tp

1 0

Plan to double-check Tor Browser initial download numbers
by Karsten Loesing 02 Aug '17

02 Aug '17

Hello list, it's been almost two years since we started collecting sanitized Apache web server logs. During this time the number of Tor Browser initial downloads rarely went below 70,000 per day. https://metrics.torproject.org/webstats-tb.html Either there must be a steady demand for fresh binaries, or there is a non-zero number of bots downloading the Tor Browser binary several times per day. I already double-checked our aggregation code that takes sanitized web server logs as input and produces daily totals as output. It looks okay to me. I'd also like to double-check whether there's anything unexpected happening before the sanitizing step. For example, could it be that there are a few IP addresses making hundreds or thousands of requests? Or are there lots of requests with same referrers or common user agents indicating bots? My plan is to ask our admins to temporarily add a second Apache log file on one of the dist.torproject.org hosts with the default Apache log file format without the sanitizing that is usually applied. A snapshot of 15 or 30 minutes would likely be sufficient as sample. I'd analyze this log file on the server, delete it, and report my findings here. This message has two purposes: 1. Is this approach acceptable? If not, are there more acceptable approaches yielding similar results? 2. Are there any theories what might keep the numbers from dropping below those 70,000 requests per day? What should I be looking for? Thanks! All the best, Karsten

5 10

June 2017 grants report
by Tommy Collison 01 Aug '17

01 Aug '17

Hello Tor! I call this a grants report, but I do other writing-y stuff as well. ### July 2017 - Wrote and submitted three new grants, two to MDF and one to DRL. - Edited the final report for the MOSS grant, which will also go out as a blogpost in the next week or two. - Wrote two blogposts. [1] [2] - Researched a bunch of new grants. - Worked on the onboarding wiki. - Worked on UX/UI stuff. - Worked on a grant submission system, which'll make it easier to apply for grants. [1] https://blog.torproject.org/blog/tor-joins-online-day-action-net-neutrality [2] https://blog.torproject.org/blog/de-anonymization-smart-homes-and-erlang-at… As always, you can email me if you have questions or comments. -TC p.s. I'll be in San Francisco August 16-30; hope to see some of you!

1 0

Sukhbir's July 2017
by Sukhbir Singh 01 Aug '17

01 Aug '17

Hi, In July, I worked on the following: * Tor Messenger - Completed the transition to ESR52 including switching to the new build system that uses runc, and rebasing the patches. We are now building on Tor Browser tag tor-browser-52.2.0esr-7.0-1-build1. https://gitweb.torproject.org/tor-messenger-build.git/log/?h=esr52 We are now reusing most of the build components from Tor Browser thanks to the power of rbm! - This month, the focus is to pick tickets that we think are critical and then close them before the end of the month, working towards the 0.5.0b1 release. * TorBirdy Prepared and tested TorBirdy release 0.2.3 that will be released in the coming week. -- Sukhbir

1 0

UX team summary for July 2017
by Linda Naeun Lee 01 Aug '17

01 Aug '17

Hi all! July was another good month for UX. :) I gave a talk to PETS 2017 about my paper ( https://petsymposium.org/2017/papers/issue3/paper2-2017-3-source.pdf) which evaluated the usability of Tor Launcher. I got a great reaction from the audience, most of which seemed to buy the "usability is important!" message. Some OTF people were in the audience, and they seemed to understand how critical usability work is! We were encouraged to apply for a grant to get some funding for UX work, so Isa and Tommy are following through on that in August. Isa and I worked in person while at PETS to information architect what will go on the home portal for torproject.org. We know it's taking a while, and people are wary of attempts to improve the site. But we're quite serious and persistent! It's just not funded at the moment and getting our spare cycles. We finished redesigning Tor launcher (https://marvelapp.com/3f6102d/) to better guide users through configuring their network settings. Some of the changes made were: 1) combining all the configuration options into a single screen, as inspired by the in-browser network settings menu 2) clarifying which countries will need to configure bridges on the first screen 3) incorporating the bridgeDB service into the interface. In addition we started other work, such as: - potential changes to the tor browser toolbar to give easier access to security controls - messages and indicators for different states when visiting a .onion site (http + .onion, for instance) - user testing for onionbrowser to validate recent changes to their onboarding sequence Cheers, Linda N. Lee Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2

2 1