[tor-bugs] #11790 [Tor]: Our zlib deflate objects require lots of memory per directory connection

[Tor Bug Tracker & Wiki]

#11790: Our zlib deflate objects require lots of memory per directory connection
-----------------------------------------+---------------------------------
 Reporter:  nickm                        |          Owner:
     Type:  defect                       |         Status:  new
 Priority:  normal                       |      Milestone:  Tor:
Component:  Tor                          |  0.2.6.x-final
 Keywords:  tor-relay zlib 025-backport  |        Version:
Parent ID:                               |  Actual Points:
                                         |         Points:
-----------------------------------------+---------------------------------
 Compressing descriptors on the fly requires a lot of memory with zlib's
 default settings.  According to zconf.h:

 {{{
 /* The memory requirements for deflate are (in bytes):
             (1 << (windowBits+2)) +  (1 << (memLevel+9))
  that is: 128K for windowBits=15  +  128K for memLevel = 8  (default
 values)
  plus a few kilobytes for small objects.
 }}}

 We are using windowbits == 15 and memLevel == 8.  This might explain in
 part why busy directories are so memory-hungry.  In addition to our
 investigations for #11787, we should see whether we can safely reduce this
 to 32K or so with windowBits == 12 and memlevel == 5.  It appears in
 preliminary investigation that this compression level would not hurt
 compression very much (0-5% for microdescs, 3-7% for server descriptors).

 We could also choose our window size and memlevel based on current memory
 availability.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/11790>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online