[tor-bugs] #11648 [Tor]: Problem parsing .z-compressed descriptors fetched via DirPort

[Tor Bug Tracker & Wiki]

#11648: Problem parsing .z-compressed descriptors fetched via DirPort
-------------------------+-----------------------
     Reporter:  karsten  |      Owner:
         Type:  defect   |     Status:  new
     Priority:  normal   |  Milestone:
    Component:  Tor      |    Version:
   Resolution:           |   Keywords:  tor-relay
Actual Points:           |  Parent ID:
       Points:           |
-------------------------+-----------------------

Comment (by wfn):

 Hmm, it seems that ''something'' must be wrong with that zlib stream data,
 yes, because according to http://stackoverflow.com/questions/3122145/zlib-
 error-error-3-while-decompressing-incorrect-header-
 check/22310760#22310760, one of those options (one of the "window sizes")
 should work. I tried them all, and kept getting "incomplete or truncated
 stream" or "invalid stored block lengths".

 <detail>

 `common/torgzip.c`:

 {{{
  87 /** Return the 'bits' value to tell zlib to use <b>method</b>.*/
  88 static INLINE int
  89 method_bits(compress_method_t method)
  90 {
  91   /* Bits+16 means "use gzip" in zlib >= 1.2 */
  92   return method == GZIP_METHOD ? 15+16 : 15;
  93 }
 }}}

 https://docs.python.org/2/library/zlib.html:

 {{{
 zlib.decompress(string[, wbits[, bufsize]])

 [...]

 wbits must not be smaller than the size originally used to compress the
 stream; using a too-small value will result in an exception.
 The default value is [...] 15.
 }}}

 I tried all the window sizes that people try (including
 `zlib.MAX_WBITS|32` for automatic header detection), nothing worked.

 I don't know why `decompressobj().decompress()` does work. I looked at
 python's packages briefly, in particular, at
 `local/lib/python2.7/encodings/zlib_codec.py`, but didn't find anything
 useful there. Fun times.

 </details>

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/11648#comment:11>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online