[tor-bugs] #25667 [Core Tor/Tor]: LZMA/ZSTD descriptor compression support

[Tor Bug Tracker & Wiki]

#25667: LZMA/ZSTD descriptor compression support
-------------------------------------------------+-------------------------
 Reporter:  atagar                               |          Owner:  (none)
     Type:  defect                               |         Status:  new
 Priority:  Medium                               |      Milestone:  Tor:
                                                 |  0.3.3.x-final
Component:  Core Tor/Tor                         |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  regression? 033-must needs-analysis  |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by teor):

 Replying to [comment:9 atagar]:
 > Thanks Tim, but no luck. Perhaps I'm doing something wrong? The
 following should fetch a lzma compressed descriptor from radia0...

 Descriptors are not available in the lzma encoding, because they are
 streamed:

 Replying to [comment:4 teor]:
 > Directory responses choose a common, supported method in this order:
 > Precompressed, cached data: lzma > zstd > zlib > gzip > identity
 > Streamed data: zstd > zlib > gzip > identity
 > Anonymous connections (e.g. HSDirs): zlib > gzip > identity
 > https://gitweb.torproject.org/tor.git/tree/src/or/directory.c#n3575

 This is what I get:

 {{{
 $ for encoding in x-tor-lzma x-zstd deflate gzip identity; do
     for doc in server/authority status-vote/current/consensus; do
         echo "Accept-Encoding: $encoding"
         echo "Requested: $doc"
         curl -s -O --header "Accept-Encoding: $encoding" \
             91.121.230.208:9030/tor/$doc && \
         file authority
     done
 done
 Accept-Encoding: x-tor-lzma
 Requested: server/authority
 authority: ASCII text
 Accept-Encoding: x-tor-lzma
 Requested: status-vote/current/consensus
 authority: ASCII text
 Accept-Encoding: x-zstd
 Requested: server/authority
 authority: Zstandard compressed data (v0.8+), Dictionary ID: None
 Accept-Encoding: x-zstd
 Requested: status-vote/current/consensus
 authority: Zstandard compressed data (v0.8+), Dictionary ID: None
 Accept-Encoding: deflate
 Requested: server/authority
 authority: zlib compressed data
 Accept-Encoding: deflate
 Requested: status-vote/current/consensus
 authority: zlib compressed data
 Accept-Encoding: gzip
 Requested: server/authority
 authority: gzip compressed data, max compression, from Unix
 Accept-Encoding: gzip
 Requested: status-vote/current/consensus
 authority: gzip compressed data, max compression, from Unix
 Accept-Encoding: identity
 Requested: server/authority
 authority: ASCII text
 Accept-Encoding: identity
 Requested: status-vote/current/consensus
 authority: ASCII text
 }}}

 I think that the consensus might not be available in lzma because it was
 cached on disk by a previous version of tor. Maybe wait an hour or two for
 a new consensus to arrive, and try lzma again?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25667#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online