[tor-bugs] #4439 [Metrics Utilities]: Develop a Java/Python API that wraps relay descriptor sources and provides unified access to them

[Tor Bug Tracker & Wiki]

#4439: Develop a Java/Python API that wraps relay descriptor sources and provides
unified access to them
-------------------------------+--------------------------------------------
 Reporter:  karsten            |          Owner:  karsten
     Type:  task               |         Status:  new    
 Priority:  normal             |      Milestone:         
Component:  Metrics Utilities  |        Version:         
 Keywords:                     |         Parent:         
   Points:                     |   Actualpoints:         
-------------------------------+--------------------------------------------

Comment(by atagar):

 > For example, the consensus-health checker needs to have all consensuses
 and votes available before it can do anything

 I favor the iterator for that reason - callers that want everything
 buffered can read everything into a list (simple to do with both python
 and java).

 > How about we implement both the descriptor store and a callback pattern?

 The callback is bad because you're having the handler block reads, and
 stores are bad for the reasons mentioned earlier. If we went with an
 iterator then it would be the best of both worlds: unblocked reads,
 limited memory usage if the handler is faster than reads, and can be
 converted into a store too. The only advantage to a callback is that it
 would guarantee constant memory usage (if your handlers slow then you
 could consume as much memory as your buffer size which would probably be
 unbouned). On second thought that would be likely to come up when reading
 local cached descriptors... lets do both.

 > How would the iteration pattern look like? Do you have an example?

 Iterator would just be a simple producer/consumer. The producer thread
 adds descriptors to a buffer as they're read and the consumer pops
 elements off and provides them to the caller (blocking if there's no
 input). Iirc this would be handled in both python and java by a
 synchronized queue (I forget the class...
 java.util.concurrent.BlockingQueue?).

 > I'm not entirely sure what you mean here.

 Requesting descriptors via the control socket can be for individual
 relays. I was thinking there may be some counterpart for 'give me
 descriptor for fingerprint X' via directory mirrors and authorities but on
 second thought tor wouldn't use that so it would be odd if that capability
 existed. Oh well...

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/4439#comment:11>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online