[tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances

[Tor Bug Tracker & Wiki]

#18910: distributing descriptors accross CollecTor instances
-------------------------------+--------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  assigned
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+--------------------------

Comment (by karsten):

 Fine question.  I also thought about that a while ago and concluded (but
 apparently forgot to mention) that it's fine to simply use metrics-lib's
 DescriptorCollector to mirror the other instance's `recent/` directory and
 import that.  That directory is currently 2.8G and covers roughly 72 hours
 of data.  That would be about 40M new data per hour (per mirror), which we
 might be able to shrink when using compression.  The upside is that the
 engineering effort for this solution would be almost trivial, because the
 code already exists and is used by Onionoo, Metrics, and ExoneraTor.

 I admit that your option 2 is very tempting, mostly because designing
 protocols is fun, but also because it would be the more efficient approach
 with potentially other benefits we don't get with option 1.  But it's also
 a possible time sink of unknown depth.  I'd say (but can be convinced
 otherwise) that we should go for option 1.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online