[tor-bugs] #11648 [Tor]: Problem parsing .z-compressed descriptors fetched via DirPort

[Tor Bug Tracker & Wiki]

#11648: Problem parsing .z-compressed descriptors fetched via DirPort
-------------------------+-----------------------
     Reporter:  karsten  |      Owner:
         Type:  defect   |     Status:  new
     Priority:  normal   |  Milestone:
    Component:  Tor      |    Version:
   Resolution:           |   Keywords:  tor-relay
Actual Points:           |  Parent ID:
       Points:           |
-------------------------+-----------------------

Comment (by karsten):

 Replying to [comment:9 wfn]:
 > TL;DR of my thoughts right now:

 Thanks for including a tl;dr. :)

 >  * I don't think any zlib-compressed-data caching happens (can't find
 any in `buffers` or `torgzip`, etc.)
 >  * Nevertheless, the diffs ''should really be'' just because
 directories' view of relays and their descriptors is always changing (it
 would be nice to confirm this in a definite manner, of course.) Also, some
 caching could be happening "somewhere else" (i.e.: I don't know.)

 I'm not too worried about this, but I can't say for sure that there's
 ''no'' bug here.  But should we move this to a separate ticket?  The two
 possible issues seem unrelated.  Would you want to create that ticket?

 >  * `all.z` is a zlib stream, and as far as zlib streams go, everything
 is OK with it.

 I don't buy that yet.  Why would `zlib.decompress` fail if this is a valid
 zlib stream?

 (Also note that the `zlib.decompressobj` solution doesn't work for me,
 because I'm really looking for a way to make
 `java.util.zip.InflaterInputStream` work.  I just wrote the test cases in
 Python, because I was hoping to get more feedback on that.  And look how
 this worked just fine!  But I really suspect that the problem is in tor's
 use of zlib.)

 So, I looked at `turtles-server-all.z` using `xxd` with RFC 1950 opened in
 a second window.  The first bytes (`0x78da`) look like a valid header
 (`78` = "deflate with 32k window size", `da`= "max compression, no dict"),
 but I'm not sure about the last bytes (`0x0000ffff`).  These last bytes
 are supposed to be the Adler-32 checksum, but these bytes don't look like
 a valid checksum to me.  Maybe something is not initialized or not flushed
 correctly?

 >  * I'm not really qualified to make the above statements, sorry. i.e.:
 someone else should take a look at this, too.

 Same here!  Thanks for trying!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/11648#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online