[tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances

[Tor Bug Tracker & Wiki]

#18910: distributing descriptors accross CollecTor instances
-------------------------------+-----------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_information
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+-----------------------------------

Comment (by karsten):

 I haven't fully made up my mind about the following, but maybe it's food
 for thought:

 Recently, gabelmoo's cached-descriptors file contained hundreds of server
 descriptors that had no corresponding extra-info descriptors.  We cannot
 blame gabelmoo for accepting these valid descriptors, and even if we were
 to add a @source tag to these descriptors saying they came only from
 gabelmoo, we wouldn't later go and delete all descriptors by gabelmoo.
 The real problem is that anyone can produce as many descriptors as they
 want.  Neither of the solutions above (which are based on our previous
 discussions) would help here.

 I believe the only fix is to discard relay and bridge descriptors that are
 not referenced from votes or consensuses.  And I know that I stated
 earlier that I'd also want to archive other descriptors.  But I don't see
 yet how to achieve both.

 From an implementation point of view we could build this in two phases: 1.
 we fetch from other CollecTor instances and believe everything we get
 without attaching @source tags, and 2. we create a staging area of some
 sort where we store descriptors that are not referenced yet and delete
 them after a week or so unless we see a descriptor we trust that
 references them.  It's probably smart to do 1. first in order to make
 CollecTor more robust.  We'll have to repackage old tarballs anyway after
 implementing 2., so there's no big rush there.

 Again, not sure yet what to do here.  Sorry for the confusion, but it
 seems it's not easy to do this right.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online