[tor-bugs] #16424 [metrics-lib]: Add code to eventually support .xz compressed tarballs

[Tor Bug Tracker & Wiki]

#16424: Add code to eventually support .xz compressed tarballs
-----------------------------+--------------------------
     Reporter:  karsten      |      Owner:  karsten
         Type:  enhancement  |     Status:  needs_review
     Priority:  normal       |  Milestone:
    Component:  metrics-lib  |    Version:
   Resolution:               |   Keywords:
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+--------------------------

Comment (by leeroy):

 >With respect to your observation of blocking until parsing is complete,
 please take a look at `DescriptorReader.setMaxDescriptorFilesInQueue`  and
 try setting it to a larger value.  It's actually supposed to be a  feature
 not to load and parse too many descriptors if the application  doesn't
 process them.

 That's a problem.

  1. The descriptor queue contains parsed descriptors. Files are added
 after parsing.
  1. Parsing runs in the same thread, which explains why it's single-core
 bound.
  1. Total read time will always be less than total parse time, unless the
 data to be parsed is specifically selected to have low parse time.
  1. Parsing is slow. Slow enough that a queue of waiting data to be parsed
 can be readily formed.

 I'll revisit this topic more in the separate ticket yet to be created.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16424#comment:34>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online