Core Tor Team February 2017 Report
We started to work on collecting measurements[1] of our consensus diff proposal (prop140) to analyze how much data Tor downloads today, how much data it would download after the consensus diff implementation is complete, and what we could do in order to reduce the downloaded volume further.
We created the following tools[2] to help us collect and analyze our data:
The populate.py and analyze.py scripts are for looking at the amount and causes ofmicrodescriptor change over time. The populate-consenus and analyze-consensus.py scripts look at the amount and causesof routerstatus change over time in the consensus. The diff-size tool determines the mean size for ed-based consensus diffs, after various consensus transformations and diff compression algorithms.
With these tools, we were able to improve[3] Proposal 140 [4] "Provide diffs between consensuses", which is an important part for us to do to achieve our objective. They also lead to the creation of proposals 274 through 276, each of which will reduce the required bandwidth for directory operations (see below).
We also created tools[5] to analyze different compression schemes related to Proposal 278 [6] "Directory Compression Scheme Negotiation", which is one of the design proposals we plan to implement in order to achieve our goals under this objective. We have the results of the first collection/analyses[7].
The other proposals related to directory bandwidth savings are: Proposal 274 [8] - Rotate onion keys less frequently Proposal 275 [9] - Stop including meaningful "published" time in microdescriptor consensus Proposal 276 [10] - Report bandwidth with lower granularity in consensus documents
[1] https://trac.torproject.org/projects/tor/ticket/21205 [2] https://github.com/nmathewson/consensus-diff-analysis [3] https://trac.torproject.org/projects/tor/ticket/21209 [4] https://github.com/torproject/torspec/blob/master/proposals/140-consensus-di... [5] https://gitlab.com/ahf/tor-sponsor4-compression [6] https://github.com/torproject/torspec/blob/master/proposals/278-directory-co... [7] https://docs.google.com/spreadsheets/d/1devQlUOzMPStqUl9mPawFWP99xSsRM8xWv7D... [8] https://github.com/torproject/torspec/blob/master/proposals/274-rotate-onion... [9] https://github.com/torproject/torspec/blob/master/proposals/275-md-published... [10] https://github.com/torproject/torspec/blob/master/proposals/276-lower-bw-gra... [11] https://github.com/torproject/torspec/blob/master/proposals/277-detect-id-sh...