[tor-bugs] #20440 [Metrics/metrics-lib]: endless wait in BlockingIteratorImpl

[Tor Bug Tracker & Wiki]

#20440: endless wait in BlockingIteratorImpl
---------------------------------+-----------------------------------
 Reporter:  iwakeh               |          Owner:  iwakeh
     Type:  defect               |         Status:  needs_review
 Priority:  High                 |      Milestone:  metrics-lib 1.6.0
Component:  Metrics/metrics-lib  |        Version:
 Severity:  Normal               |     Resolution:
 Keywords:                       |  Actual Points:
Parent ID:                       |         Points:
 Reviewer:                       |        Sponsor:
---------------------------------+-----------------------------------

Comment (by karsten):

 Thanks for pointing me to that email from October.  I was able to
 reproduce the issue!

 Here's what happens: Onionoo reads descriptors from `in/archive/` and
 finds descriptor types in there that it does not recognize, including
 votes, microdescs, and certs.  It decides (for whatever reason!) that it
 should stop reading from that directory, leaves the descriptor queue
 alone, and continues with reading descriptors from `in/recent/`.  However,
 the metrics-lib non-daemon thread that reads descriptors and puts them
 into the queue blocks as soon as the queue runs full and later prevents
 the JVM from terminating.

 This means we have two bugs here:

  - metrics-lib should allow the application to stop reading descriptors
 for whatever reason.  Therefore, turning the reader thread into a daemon
 thread seems like the right fix.  Please review my
 [https://gitweb.torproject.org/user/karsten/metrics-
 lib.git/log/?h=task-20440 task-20440 branch] with your cherry-picked
 commit and a change log entry.

  - Onionoo should be a little more robust against unexpected data in the
 `in/archive/` directory.  It could simply skip the unrecognized descriptor
 and carry on with the next descriptor.  That's `s/break/continue/` in
 [https://gitweb.torproject.org/onionoo.git/tree/src/main/java/org/torproject/onionoo/updater/DescriptorSource.java#n213
 this line].  Happy to discuss that further in an Onionoo ticket, or happy
 to just make this change.

 Of course, we could as well blame the operator for putting votes,
 microdescs, and certs into the `in/archive/` directory, knowing very well
 that Onionoo would not recognize those. ;)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20440#comment:15>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online