[tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances

[Tor Bug Tracker & Wiki]

#18910: distributing descriptors accross CollecTor instances
-------------------------------+---------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  assigned
 Priority:  High               |      Milestone:  CollecTor 1.1.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------------------

Comment (by iwakeh):

 Please review the seven commits on top of
 [https://gitweb.torproject.org/user/iwakeh/collector.git?h=task-18910
 -first-sync this branch].

 There are two new packages:
 `sync` for sync-merge functionality and `persist` as new modular way to
 persist descriptors (currently file-system, but could be extended or
 changed in future).  The latter should step by step be used for all
 persisting of descriptors, i.e. be used instead of the `store*` methods
 throughout the various modules.  (that is useful for removing the tight
 circular coupling of ArchiveWriter, DescriptorDownloader and
 DescriptorParser for example).

 Persisting is based on `DescriptorPersistence` defining methods for
 storing.  The classes extending `DescriptorPersistence` just need to
 define the explicit storage path.  For convenience
 `PersistenceUtils`provides date-time to string methods. Thus, providing
 methods for code that is repeatedly defined in the current code base.

 `CollecTorMain` extends `SyncManager` in a way that all synchronization
 options can be configured during runtime, i.e. syncing of a module can be
 turned on or off and sources can be changed without restart.

 (I'll add package-info later for the two packages.)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:18>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online