[tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances

[Tor Bug Tracker & Wiki]

#18910: distributing descriptors accross CollecTor instances
-------------------------------+---------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_review
 Priority:  High               |      Milestone:  CollecTor 1.1.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------------------

Comment (by karsten):

 Replying to [comment:49 iwakeh]:
 > Replying to [comment:46 karsten]:
 >
 > I found the a good reference for the paths topic: it's in the
 [https://gitweb.torproject.org/collector.git/tree/src/main/resources
 /create-tarballs.sh#n68 create tars script]: [...]

 Indeed, that script reflects what's on the server.

 > The above makes an inconsistency between relay descriptor and bridge
 descriptor paths visible.

 Right.  The reason is that we recently switched from bridge descriptor
 tarballs containing all bridge descriptors to separate tarballs for each
 type.  I didn't change the directory structure to avoid breaking stuff.
 In theory, it would have been an easy change, but going back to a previous
 version would have become more difficult.  And it didn't seem necessary to
 change the directory structure, because nobody would look at it anyway.

 > As the new structure from #20228, i.e. votes also grouped and all dates
 in recent paths are derived from the acquisition time, will be in place
 soon, I'd like to use the 'new way' already for the merge.
 >
 > With these as background the test diff has some soon to be outdated
 corrections (comments added after '//'):
 > > [...]
 >
 > When changing the structure now shouldn't relays and bridges be stored
 in similar ways?  I think the current difference (in 'out') caused some of
 the confusion here.

 Agreed on the acquisition time thing.  We could change that now.

 Regarding the directory structure, as I said above, I didn't think this
 was necessary a few weeks ago, but I see your point that it's confusing
 for contributors.

 So, if we change paths in `out/` now, should we use the exact same
 directory structure as for the `recent/` directory?  Whoever runs a
 CollecTor instance now will have to stop it, move and rename some
 directories, upgrade, and restart.  What we should avoid is change some
 paths now and some more paths in a few weeks.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:50>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online