[tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances

[Tor Bug Tracker & Wiki]

#18910: distributing descriptors accross CollecTor instances
-------------------------------+---------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_review
 Priority:  High               |      Milestone:  CollecTor 1.1.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------------------

Comment (by karsten):

 Well, the sync runs don't take just 3 or 20 minutes here, but many hours.
 I could imagine that it's the backup daemon trying to capture all file
 system changes for the next backup, or something.  I could imagine that
 similar things can happen on servers.  The issue is that the update run
 cannot start if the sync run keeps running, and that means we'll lose
 data.  I'm afraid we can't ship this yet.

 I don't understand your reasoning about `index.json` not being a true
 picture of `recent/`.  We're skipping `*.tmp` files when creating that
 file, and we always append to `.tmp` and only rename to the destination
 file when we're sure the file won't change anymore.  Where does that get
 inaccurate?

 I'd rather want to change the sync code to do the same as the update code.
 I can give that a try if you don't want to touch that code anymore.

 By the way, we'll need to merge #20380 before putting out the release.
 And I'd want to start a test run over night before releasing, so the
 release cannot happen today anyway. :(

--
Ticket URL: <https://troodi.torproject.org/projects/tor/ticket/18910#comment:90>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online