[tor-bugs] #20234 [Metrics/CollecTor]: Define CollecTor's file-structure protocol 1.0

[Tor Bug Tracker & Wiki]

#20234: Define CollecTor's file-structure protocol 1.0
-------------------------------+--------------------------------
 Reporter:  karsten            |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_revision
 Priority:  High               |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+--------------------------------

Comment (by iwakeh):

 Thanks for the thorough review!

 I only skimmed through yet, but want to quickly reply to the question
 about section 5.

 As we're relying on the 'out' structure to produce tars this should be
 documented, and the part of the directory structure inside the tars is
 visible to clients.

 The two use-cases I have in mind are:
 1. When parsing unpacked tar-balls part of the structure of 'out' is part
 of the tar-balls' structure, i.e. below month there is the 'out' structure
 to be found.
 2. When running a CollecTor instance for getting access to the data, it
 could make sense to use the 'out' structure to further operate on the
 data.  Here the CollecTor instance's purpose would tend to be data
 collection not mirroring.

 So, it is useful to describe 'out', I think.

 And, I think you're right that also 'stats' and 'sync' (introduced with
 #18910) should be part of the document.  That won't be much more text, but
 really clarifies what all the directories are about and gives operators an
 idea where they should place these directories etc.  And, it will help to
 get new developers started, or help us when debugging or changing things
 in a few months.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20234#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online