[tor-bugs] #22428 [Metrics/CollecTor]: add webstats module to collector

[Tor Bug Tracker & Wiki]

#22428: add webstats module to collector
-------------------------------+-----------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_information
 Priority:  Medium             |      Milestone:  CollecTor 1.4.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+-----------------------------------

Comment (by karsten):

 I'm not too deep into this topic right now, so handle the following
 comment with care.

 I wonder if we can avoid having that directory for temporary log files
 that cannot be published yet. It seems like a possible source for trouble
 when processing breaks at some point and we need to fix that, with half of
 a log file being written to the temporary directory and the rest still
 being in the import directory.

 Maybe we can simplify that by keeping a text file in `stats/` where we
 keep some state which files we already read or wrote. And we only write a
 file to `out/` and `recent/` when it's ready for publication. Not sure if
 this will solve all cases, but it seems potentially easier to understand
 for future operators of this service (including ourselves when we don't
 remember these design discussions anymore).

 Regarding `WebstatsReferenceDate`, it would be good to explain in the
 comments when this value needs to be changed, and to what value. The
 comment alone should be sufficient to know how to use the property,
 without further looking at the code.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22428#comment:25>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online