[metrics-bugs] #20412 [Metrics/Onionoo]: Skip bad archived descriptors rather than aborting the entire import

[Tor Bug Tracker & Wiki]

#20412: Skip bad archived descriptors rather than aborting the entire import
-----------------------------+------------------------------
 Reporter:  karsten          |          Owner:  metrics-team
     Type:  defect           |         Status:  needs_review
 Priority:  Medium           |      Milestone:
Component:  Metrics/Onionoo  |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:  metrics-2017     |  Actual Points:
Parent ID:  #20548           |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------------
Changes (by karsten):

 * status:  assigned => needs_review


Comment:

 I just ran another test with the broken descriptor above and found that it
 even gets us an `IllegalStateException`:

 {{{
 Exception in thread "main" java.lang.IllegalStateException: Operation is
 not permitted before finishing to read.
         at
 org.torproject.descriptor.impl.DescriptorReaderImpl.getExcludedFiles(DescriptorReaderImpl.java:64)
         at
 org.torproject.onionoo.updater.DescriptorQueue.writeHistoryFile(DescriptorQueue.java:113)
         at
 org.torproject.onionoo.updater.DescriptorSource.readArchivedDescriptors(DescriptorSource.java:182)
         at
 org.torproject.onionoo.updater.DescriptorSource.readDescriptors(DescriptorSource.java:84)
         at org.torproject.onionoo.cron.Main.updateStatuses(Main.java:179)
         at org.torproject.onionoo.cron.Main.run(Main.java:128)
         at
 org.torproject.onionoo.cron.Main.runOrScheduleExecutions(Main.java:102)
         at org.torproject.onionoo.cron.Main.main(Main.java:34)
 }}}

 Eek. Let's do the quick fix now and skip any bad descriptors that we run
 into. Please review
 [https://gitweb.torproject.org/user/karsten/onionoo.git/log/?h=task-20412
 my task-20412 branch].

 This quick fix may even be the longer-term fix, because we have to accept
 that invalid/unparseable descriptors may exist in the archives (as much as
 they exist in CollecTor's recent descriptors, where we simply
 [https://gitweb.torproject.org/onionoo.git/tree/src/main/java/org/torproject/onionoo/updater/DescriptorQueue.java#n150
 skip them]).

 The follow-up question, what went wrong that CollecTor produced this file,
 is going to be answered in #21087.

 I'd say as soon as the quick fix is merged, there's nothing else to do
 here.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20412#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online