Core Tor team March 2017 Report:
For March we did a good chunk of work for Proposal 140 [1] which implements consensus diffs, first we ran again our numbers for it to be able to see how much bandwidth we expect to save [2]. Then we start to work on actually change the code to implement the proposal, we did:
=> #21651 - Prop140 / compression: Refactor directory cache spooling code [3] => #21645 - Prop140 / compression: Unified directory cache backend [4] => #21644 - Prop140: Fuzz the diff and patch code [5] => fix on the performance of SHA3- keccak. This was caught by a performance report that showed the function was super slow. By fixing this we improve it speed 15%. [6]
We finished the implementation of Proposal 274! [7] It will be part of our 0.3.1 release [8]. This work will reduce the onion key rotation rate from the current value (7 days) to something closer to 28 days. Doing this will reduce the total microdescriptor download volume by approximately 70%.
We continue with our effort to revise, analyze and write proposals for ways to use less directory bandwidth. One of the ideas [9] was to explore the possibility of saving bandwidth by improving our compression system, we want to check if offering other compression mechanisms that might fit better scenarios where the trade of bandwidth usage for CPU process makes more sense.
We finished writing Proposal 278 [10], here is an email to tor-dev [11] where we shared the first draft for feedback from the community, it has a good problem statement that can help understand the benefits of this approach, and some information about the analysis we did.
Then we move to work on the actual proposal, below is the work we started in March but that will be merged in April: => #21667: Prop278: Handle new headers in directory.c [12] => #21662: prop278: Add support for LZMA2 and/or Zstandard [13] => #21663: prop278: Refactor the torgzip module to support additional compression schemes [14]
At the end of March The Tor Project had its biannual Dev meeting in Amsterdam. There we had a session with our Metrics team lead, Karsten, and Nathan from Guardian Project to discuss Directory Bandwidth Reduction [15].
This meeting was very important as we want to make sure we coordinate with the Metrics team about the work we are doing since it can influence how they count things. And we also want to take the opportunity to get feedback from Mobile, since mobile users with low-bandwidth is one of our target audience for this work.
After Amsterdam we had a follow up meeting online early this week (4/4) as well [16].
This is other work we did in March but haven't been merged yet: => Needs Revision: #20021 Require ntor-onion-key in microdescriptors, descriptors [17] => Needs Review: #21642 Prop275: Eliminate "published" times from microdescriptor consensus [18] => Needs Review: #21869 Labeled storage backend for consensus docs and diffs [19]
[1] https://gitweb.torproject.org/torspec.git/tree/proposals/140-consensus-diffs... [2] https://trac.torproject.org/projects/tor/ticket/21210 [3] https://trac.torproject.org/projects/tor/ticket/21651 [4] https://trac.torproject.org/projects/tor/ticket/21645 [5] https://trac.torproject.org/projects/tor/ticket/21644 [6] https://trac.torproject.org/projects/tor/ticket/21737 [7] https://gitweb.torproject.org/torspec.git/tree/proposals/274-rotate-onion-ke... [8] https://trac.torproject.org/projects/tor/ticket/21641 [9] https://trac.torproject.org/projects/tor/ticket/21211 [10] https://gitweb.torproject.org/torspec.git/tree/proposals/278-directory-compr... [11] https://lists.torproject.org/pipermail/tor-dev/2017-March/011959.html [12] https://bugs.torproject.org/21667 [13] https://bugs.torproject.org/21662 [14] https://bugs.torproject.org/21663 [15] https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Amsterdam/Not... [16] https://pad.riseup.net/p/chikeuBN1OUh [17] https://trac.torproject.org/projects/tor/ticket/20021 [18] https://trac.torproject.org/projects/tor/ticket/21642 [19] https://trac.torproject.org/projects/tor/ticket/21869