[tor-bugs] #22819 [Core Tor/Tor]: Choice of compressors seems to be suboptimal

[Tor Bug Tracker & Wiki]

#22819: Choice of compressors seems to be suboptimal
--------------------------+------------------------------------
 Reporter:  yurivict271   |          Owner:  ahf
     Type:  defect        |         Status:  assigned
 Priority:  Medium        |      Milestone:  Tor: 0.3.1.x-final
Component:  Core Tor/Tor  |        Version:
 Severity:  Normal        |     Resolution:
 Keywords:                |  Actual Points:
Parent ID:                |         Points:
 Reviewer:                |        Sponsor:
--------------------------+------------------------------------

Comment (by ahf):

 I just added a simple LZ4 test to our own little benchmarking tool which
 can be found at https://gitlab.com/ahf/tor-
 sponsor4-compression/tree/master/bench

 I did not measure the memory patterns of LZ4, which was the reason we
 wrote this tool in the beginning, but it does look like LZ4 is a lot
 faster than anything else we have (even beats gzip at compression level
 1), but the compression ratio is also worse than anything else we have for
 the `cached-consensus` document (~2.2 MB of structured text).

 The results can be found here: https://docs.google.com/spreadsheets/d
 /1WzFXQGfH8yI4WCCRAEQBt1Z_sC-3hMkmE6HNKDaaSiE/edit#gid=0

 If someone is able to make the ratio better in the `bench` tool I'm all
 open to rerun these tests. I could only see an `acceleration` value to
 tune in the `lz4.h` file, which would allow me to make LZ4 even faster,
 but not get a better compression ratio.

 Since we are currently batch processing the compression of our "larger"
 files in the background I don't think we would win much by including LZ4
 here.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22819#comment:9>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online