- tor-project - lists.torproject.org

GSoC 2017 Ahmia Status Update #5
by Pushkar Pathak 06 Aug '17

06 Aug '17

Hello everyone, This is the bi-weekly status report for Ahmia - Hidden Service Search ## Added support for subdomains Until now it was not possible for operators to add their sub domains to Ahmia database. Now we have made required modifications in our 'Add' service to support sub domains. [0] ## Documentation I am working on to update all the documentation and software dependencies across Ahmia. [1][2] If time permits, I will try to add advanced search queries using operators like "",|| and &&. Thanks, Pushkar [0]: https://github.com/mdhash/ahmia-site/commits/master [1]: https://ahmia.fi/documentation/ [2]: https://github.com/mdhash/ahmia-site/commit/d3af0b72d9b175c7d3d269119f4dcee…

1 0

SponsorR Status Report July 2017
by George Kadianakis 06 Aug '17

06 Aug '17

Hello, here is the July 2017 report for SponsorR: - Our development team has entered the final stages of coding prop224! We successfuly demoed a proof-of-concept client connection to a prop224 hidden service website on the real network with great results! The remaining prop224 work is testing, bugfixing, stabilization, and improvements. Also, we need to merge all this code upstream in tor-0.3.2 which means a serious review process! - To mark the beginning of this merge procedure, we *merged* upstream the first service-side part of prop224!!! That's a pretty big piece of work: https://trac.torproject.org/projects/tor/ticket/21979 - We also merged upstream some groundwork prop224 tasks: https://trac.torproject.org/projects/tor/ticket/22727 https://trac.torproject.org/projects/tor/ticket/22726 - The final service-side part of prop224 passed its first review rounds, and we have now moved to final review and fixes: https://trac.torproject.org/projects/tor/ticket/20657 https://oniongit.eu/dgoulet/tor/merge_requests/6 - We also finished the client-side branch and we are now starting the review: https://trac.torproject.org/projects/tor/ticket/17242 https://oniongit.eu/asn/tor/merge_requests/2/commits - Fixed an annoying prop224 warning message that was forgotten in 0.3.0: https://lists.torproject.org/pipermail/tor-relays/2017-August/012689.html

1 0

GSoC 2017 - Crash Reporter for Tor Browser: Report #5
by Nur-Magomed 05 Aug '17

05 Aug '17

Hi everyone! This is my bi-week status report for GSoC project "Crash Reporter for Tor Browser". # Crash reporter client: - We considered carefully crash report fields and made decision to reinterpret blacklist feature as allow list [1]; - removed crash event files (some kind of log files that are keeping a few locally crash data even if we sent them - are not the same as crash report files) keeping functional[2] Next steps: # Using new RBM[3] we trying to build Tor browser alpha with crashreporter and make it reproducible # After that we need to test it with server side Kind regards, Nur-Magomed (nmago) [1] https://github.com/nmago/tor-browser/commit/1b3bf799aec4327e99eb0fe0365a637… [2] https://github.com/nmago/tor-browser/commit/0bb8072c03cbf311db2f71d59fdbb6c… [3] https://gitweb.torproject.org/builders/tor-browser-build.git/

1 0

New Website for Nyx (aka arm)
by Damian Johnson 05 Aug '17

05 Aug '17

Hi all, I'm delighted to announce Nyx has a new home! https://nyx.torproject.org/ For those who remember Tor Cloud this is stylistically based on it. Feedback welcome! Tested with Firefox and Chromium. Passes W3C validation and works without JS (though no FAQ). I'll replace arm's old page (https://www.atagar.com/arm/) with a redirect after a while unless folks can think of a reason to keep the old site around. Cheers! -Damian

1 0

GSoC 2017 - unMessage: Report #5
by Felipe Dau 04 Aug '17

04 Aug '17

Hi everyone! This is my report #5 for the unMessage GSoC project. # Updates Since the last report I continued working on #21 [0] and #33 [1]: - Stopped using queues and threads to check for incoming transmissions (that's when the other user starts an action) - Removed disk dependency to run tests without saving any files - Added integration tests to make sure things do not break when making low level changes for the unit tests # What's next Now that I have more confidence to make changes, I am going to start working on the unit tests. I also expect to review with my mentors what has been done for #21 and declare it done. More details on these tasks can be followed in the respective tickets [0, 1], branches [2, 3] and the latest PR [4]. Thanks, -Felipe [0]: https://github.com/AnemoneLabs/unmessage/issues/21 [1]: https://github.com/AnemoneLabs/unmessage/issues/33 [2]: https://github.com/felipedau/unmessage/tree/21/deferreds [3]: https://github.com/felipedau/unmessage/tree/33/feature/test-suite [4]: https://github.com/AnemoneLabs/unmessage/pull/72

1 0

Core Tor team sponsor4 July report
by isabela 04 Aug '17

04 Aug '17

Core Tor Team July 2017 Report In July we reviewed, triaged and fixed misc bugs related to releases 0.3.0 and 0.3.1: * Never send a consensus which the downloader already has. [1] This was an issue we identified that make the most trouble in test networks, where the consensus update rate is so fast that relays frequently try to download a consensus. We investigated and identified what caused the issue and fixed it. * LZMA coder causes crash when the sandbox is enabled. [2] While doing measurements for this project we noticed that Tor instances running as relays or authorities would sometimes crash when the sandbox is enabled. This is due to the MALLOC_MP_LIM value in sandbox.c, which is currently set to 16 MB, being too low. We limit our LZMA coder to only use 16 MB, but the coder allocates some additional data other than its internal buffer, which leads to the crash. * Investigated: Assertion failure in consensus_cache_entry_handle_get on Windows.[3] We created a mitigation and diagnostic branch. It doesn't fix this, but it makes it nonfatal and tries to get more useful info if it happens again. And receive a report that is still present in 0.3.1.4-alpha. Will continue investigation in August. * Bug: src/common/compress.c:576: tor_compress_process: [4] We fixed a bug found 0.3.1 at a stable Gentoo Linux server. * Bridge unavailable during differential consensus update [5] Bug reported for Raspberry Pi running Raspbian. we revised our threadpool implementation to prevent this background work from affecting other background work. There are 3 changelog entries for it in 0.3.1.5-alpha. ** Major bugfixes (relay, performance): Perform circuit handshake operations at a higher priority than we use for consensus diff creation and compression. This should prevent circuits from starving when a relay or bridge receives a new consensus, especially on lower-powered machines. Fixes bug 22883; bugfix on 0.3.1.1-alpha. ** Minor features (directory cache, consensus diff): Add a new MaxConsensusAgeForDiffs option to allow directory cache operators with low-resource environments to adjust the number of consensuses they'll store and generate diffs from. Most cache operators should leave it unchanged. Helps to work around bug 22883. ** Minor features (relay, performance): Always start relays with at least two worker threads, to prevent priority inversion on slow tasks. Part of the fix for bug 22883. Allow background work to be queued with different priorities, so that a big pile of slow low-priority jobs will not starve out higher priority jobs.This lays the groundwork for a fix for bug 22883. * zstd tests fail with libzstd 1.3.0 [6] Write zstd epilogues correctly when the epilogue requires reallocation of the output buffer, even with zstd 1.3.0. (Previously, we worked on 1.2.0 and failed with 1.3.0). * consdiff test fails on OS X [7] test_consdiff_base64cmp would fail on OS X because while OS X follows the standard of (less than zero/zero/greater than zero), it doesn't follow the convention of (-1/0/+1). Make the test comply with the standard. We started to land updates to dir-spec.txt for prop #278 in GL/ahf/torspec/tree/prop278-in-dir-spec [8] And concluded our review of lz4 compressor [9]. For that we added a simple LZ4 test to our own little benchmarking tool [10]. Our results [11] (results) showed that LZ4 is a lot faster than anything else we have (even beats gzip at compression level 1), but the compression ratio is also worse than anything else we have for the cached-consensusdocument (~2.2 MB of structured text). Since we are currently batch processing the compression of our "larger" files in the background we decided we wouldn't win much by including LZ4 here. For August we feel that we are pretty close from finishing this deliverable and we plan to publish a blog post about compression + consensus diff work done with this project. Hope to share that soon. [1] https://trac.torproject.org/projects/tor/ticket/22702 [2] https://trac.torproject.org/projects/tor/ticket/22751 [3] https://trac.torproject.org/projects/tor/ticket/22752 [4] https://trac.torproject.org/projects/tor/ticket/22719 [5] https://trac.torproject.org/projects/tor/ticket/22883 [6] https://trac.torproject.org/projects/tor/ticket/22927 [7] https://trac.torproject.org/projects/tor/ticket/22870 [8] https://trac.torproject.org/projects/tor/ticket/22275 [9] https://trac.torproject.org/projects/tor/ticket/22819 [10] https://gitlab.com/ahf/tor-sponsor4-compression/tree/master/bench [11] https://docs.google.com/spreadsheets/d/1WzFXQGfH8yI4WCCRAEQBt1Z_sC-3hMkmE6H…

1 0

July 2017 Report for the Tor Browser Team
by Georg Koppen 04 Aug '17

04 Aug '17

Hi! In July the Tor Browser team made 4 releases, Tor Browser 7.0.2[1], Tor Browser 7.0.3[2], Tor Browser 7.5a2[3], and Tor Browser 7.5a3[4]. All four releases were out-of-bound releases fixing security issues. Tor Browser 7.0.2 and Tor Browser 7.5a2 included updated Tor versions (0.3.0.9 and 0.3.1.4-alpha respectively) and Tor Browser 7.0.3 and Tor Browser 7.5a3 fixed a potential proxy bypass bug.[5] The latter two were Linux-only releases as no other platform was affected by the problem, which got reported in our newly launched bug bounty program provided by HackerOne.[6] Non-release work focused on both regressions due to our transition to Firefox 52 ESR[7][8] and on Sponsor4 work: we got Selfrando ready to be shipped in 32bit Linux alpha bundles[9] and are about to switch to our new reproducible build system. The upcoming alpha series should be the last one that is built with Gitian, which is exciting news. The full list of tickets closed by the Tor Browser team in June is accessible using the `TorBrowserTeam201707` keyword in our bug tracker.[10] In August we plan to finally switch to rbm as our new tool for our reproducible builds which then allows us to start working on Tor Browser for 64bit Windows systems, which is due later this year[11]. Then we hope to get our work on our Panopticlick system done so that we can make use of it for fingerprinting statistics. Finally, we'll put some energy on redesigning Tor Launcher with help from the UX team and network team mainly to make it easier for censored users to get the bridges they need to reach the Internet.[12] All tickets on our radar for this month can be seen with the `TorBrowserTeam201708` keyword in our bug tracker.[13] Georg [1] https://blog.torproject.org/blog/tor-browser-702-released [2] https://blog.torproject.org/blog/tor-browser-703-released [3] https://blog.torproject.org/blog/tor-browser-75a2-released [4] https://blog.torproject.org/blog/tor-browser-75a3-released [5] https://trac.torproject.org/projects/tor/ticket/23044 [6] https://hackerone.com/torproject [7] https://trac.torproject.org/projects/tor/ticket/22343 [8] https://trac.torproject.org/projects/tor/ticket/22618 [9] https://trac.torproject.org/projects/tor/ticket/20848 [10] https://trac.torproject.org/projects/tor/query?status=closed&keywords=~TorB… [11] https://trac.torproject.org/projects/tor/ticket/20636 [12] https://trac.torproject.org/projects/tor/ticket/21951 [13] https://trac.torproject.org/projects/tor/query?status=accepted&status=assig…

1 0

Notes from August 3 2017 Vegas team meeting
by Karsten Loesing 04 Aug '17

04 Aug '17

Notes for August 3 2017 meeting: Alison: 1) Worked on further refining support wiki. Wrote a blog post about the support wiki that Steph will publish sometime next week. The post makes it clear that the support wiki is a dynamic document that will get better with more feedback, and I'm hoping to get feedback in the post comments that we can incorporate into the wiki during our support wiki sprint on August 11. 2) Onboarded Steph about Tor/FOSS culture and history. When I get some time I'm going to clean up my notes and turn that document into an actual onboarding doc. Feedback would be great! 3) Worked on the code of conduct draft some. Now wondering what's up with the membership guidelines? 4) Spoke about onion services for archives at the Society of American Archivists Conference last week. 5) Began organizing the Tor Speakers Bureau. Almost 30 people have signed up! 6) Worked on some grant stuff with the comms team. 7) Worked on some Tor Meeting planning stuff. Next week's Vegas meeting will be all about planning the Tor Meeting schedule. Georg: 1) Who should go to the OTF Summit? (Discussed for a while. Isabela will reach out to teams and individuals, aiming to reply to OTF RSN) Mike: 0) Working to get Prop247 (hidden service guard selection) performance experiments running. Will need to coordinate with Karsten for running a few onionperf instances. 1) Goal is to get both Prop247 and Prop254 implementations working well enough for experiments to run while I am on vacation. Going to be ignoring most other non-critical things. Nick: 0) Nothing seems to be on fire with us. 1) Didn't get hs-ng merged for defcon, but it *is* working. Code still under review. Hoping we'll merge by the 0.3.2 deadline (15 Sep.) 2) Had to push back 0.3.1 stable, since it wasn't ready for the start of august. Hoping for start of september. Not currently planning corresponding delays in 0.3.2 stable deadlines. 3) I'm trying to push the privcount work forward. 0.3.2 merge for some in-tor trial component seems doable, pending prompt replies from nrl folks. 4) Got useful feedback from mcs et al on usability features in progress reporting for Tor Launcher. Karsten: 1) Double-checked Tor Browser initial download numbers without finding anything unusual. iwakeh might continue this analysis next week. Isabela: 0) worked with Tommy and Metrics folks on OTF response 1) Worked with Erin, Tommy and Steph on 'onboarding wiki' - https://trac.torproject.org/projects/tor/wiki/org/onboarding (we organized the information people brainstormed into sections and started populating those but there are still lots of work to do on this page) 2) Got deliverables reports out - getting july reports ready to be send before i go on vacation 3) great progress with tor launcher ui changes - now working on dependencies and copy / also lots of progress overall with ux work, did an update no all our projects at this week meeting (you can see on meetings notes to ux list) 4) did some prep work for sponsor8 5) worked on a proposal for a collaborative way to build tor meeting schedule Steph: 1) Still working with Giant Rabbit on getting the newsletter setup. Slowly making process. Also working with them to move press contacts/orgs into Civi. Starting to move contacts in there now 2) Maintaining blog calendar and working with authors + Tommy on posts - lined up: MOSS award, Canada events, Orfox, Support Wiki, Onion Survey, Browser download metrics 3) Getting in more of a flow with press inquiries. We answered a couple questions for TheQuestion.ru, they'll be on the English site as well. Connected Arthur with a presentation. Will coordinate promotion of Colin's podcast 4) Have made a little headway with templates. Need to make odf versions. We'll have as a resource in media.tp

1 0

Plan to double-check Tor Browser initial download numbers
by Karsten Loesing 02 Aug '17

02 Aug '17

Hello list, it's been almost two years since we started collecting sanitized Apache web server logs. During this time the number of Tor Browser initial downloads rarely went below 70,000 per day. https://metrics.torproject.org/webstats-tb.html Either there must be a steady demand for fresh binaries, or there is a non-zero number of bots downloading the Tor Browser binary several times per day. I already double-checked our aggregation code that takes sanitized web server logs as input and produces daily totals as output. It looks okay to me. I'd also like to double-check whether there's anything unexpected happening before the sanitizing step. For example, could it be that there are a few IP addresses making hundreds or thousands of requests? Or are there lots of requests with same referrers or common user agents indicating bots? My plan is to ask our admins to temporarily add a second Apache log file on one of the dist.torproject.org hosts with the default Apache log file format without the sanitizing that is usually applied. A snapshot of 15 or 30 minutes would likely be sufficient as sample. I'd analyze this log file on the server, delete it, and report my findings here. This message has two purposes: 1. Is this approach acceptable? If not, are there more acceptable approaches yielding similar results? 2. Are there any theories what might keep the numbers from dropping below those 70,000 requests per day? What should I be looking for? Thanks! All the best, Karsten

5 10

June 2017 grants report
by Tommy Collison 01 Aug '17

01 Aug '17

Hello Tor! I call this a grants report, but I do other writing-y stuff as well. ### July 2017 - Wrote and submitted three new grants, two to MDF and one to DRL. - Edited the final report for the MOSS grant, which will also go out as a blogpost in the next week or two. - Wrote two blogposts. [1] [2] - Researched a bunch of new grants. - Worked on the onboarding wiki. - Worked on UX/UI stuff. - Worked on a grant submission system, which'll make it easier to apply for grants. [1] https://blog.torproject.org/blog/tor-joins-online-day-action-net-neutrality [2] https://blog.torproject.org/blog/de-anonymization-smart-homes-and-erlang-at… As always, you can email me if you have questions or comments. -TC p.s. I'll be in San Francisco August 16-30; hope to see some of you!

1 0