[tor-bugs] #16914 [Tor]: improve consensus download decompress performance
Tor Bug Tracker & Wiki
blackhole at torproject.org
Fri Aug 28 01:37:14 UTC 2015
#16914: improve consensus download decompress performance
-----------------------------+---------------------------------------------
Reporter: teor | Owner:
Type: enhancement | Status: needs_information
Priority: normal | Milestone:
Component: Tor | Version: Tor: 0.2.6.10
Resolution: | Keywords: TorCoreTeam201509 Post027Freeze
Actual Points: | Parent ID:
Points: |
-----------------------------+---------------------------------------------
Changes (by teor):
* status: new => needs_information
Comment:
Using `gzip -9 -v * 2>&1 | cut -f1 -d%` on some recent tor directory
documents:
{{{
cached-consensus-20150827-1400-UTC: 66.0
cached-descriptors-20150827-1400-UTC: 55.7
cached-microdesc-consensus-20150826-1400-UTC: 56.3
cached-microdesc-consensus-20150827-2300-UTC: 56.1
cached-microdescs-20150826-1100-UTC: 51.9
cached-microdescs-20150827-2300-UTC: 52.1
}}}
The microdescriptor consensus and descriptors are downloaded by most Tor
clients (and, therefore, by most Tor instances). The descriptors are
downloaded individually, so their ratios may be slightly lower. (The
"full" consensus and descriptors aren't used by most clients, so they can
be ignored for the purposes of this analysis.)
I suggest we increase the expected ratio to 75%. We currently double the
size of the buffer when we need to reallocate anyway, so we are already
using that much RAM for every decompression, we're just allocating 50%,
then reallocating 75%. (Except if the individual microdescriptor ratios
fall under 50%, then we're only using 50%.)
Alternately, we could increase the expected ratio to 70%, saving 1
reallocation and 5% RAM in typical cases. This would be a win for both
performance and RAM usage.
Open questions:
* What is the range of compression ratios on recent microdescriptor
consensuses? Do they vary much?
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16914#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list