[tor-bugs] #20236 [Metrics/CollecTor]: Make changes to bridgedescs module for bulk-processing tarballs

[Tor Bug Tracker & Wiki]

#20236: Make changes to bridgedescs module for bulk-processing tarballs
-----------------------------------+-----------------
     Reporter:  karsten            |      Owner:
         Type:  enhancement        |     Status:  new
     Priority:  Low                |  Milestone:
    Component:  Metrics/CollecTor  |    Version:
     Severity:  Normal             |   Keywords:
Actual Points:                     |  Parent ID:
       Points:                     |   Reviewer:
      Sponsor:                     |
-----------------------------------+-----------------
 I recently finished re-processing the entire bridge descriptor archive for
 #19317.  However, I had to make some changes to avoid running out of
 memory or wasting time on unnecessary operations.  I now went through the
 changes and cleaned them up a bit, because I'd like to merge some/most/all
 (?) of them for the next time we need to bulk-process the bridge
 descriptor archive.  I'll post a branch once I have a ticket number.

 We should discuss which of these commits should go in by default (maybe
 ed48f03, ae5c53c, and e514d30?), which should only be enabled in a special
 bulk-processing mode (maybe df96751, 27cbfc8, and 68b29c2?), which should
 have their own config option (ugh!), or which we drop because we don't
 need as badly for processing descriptors in bulk.

 Clearly, these commits need work, but I figured it's better to clean them
 up a bit now than attempt to do that in four or eight weeks.  Branch
 follows in a minute.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20236>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online