[metrics-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-------------------------------------------------+-------------------------
 Reporter:  irl                                  |          Owner:
                                                 |  metrics-team
     Type:  enhancement                          |         Status:
                                                 |  needs_review
 Priority:  Medium                               |      Milestone:
Component:  Metrics/CollecTor                    |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  metrics-roadmap-august, anti-        |  Actual Points:
  censorship-roadmap-september                   |
Parent ID:                                       |         Points:  8
 Reviewer:                                       |        Sponsor:
                                                 |  Sponsor28
-------------------------------------------------+-------------------------
Changes (by karsten):

 * status:  new => needs_review


Comment:

 Couple ideas after implementing the CollecTor and metrics-web parts:
  - As indicated before, I'm using `snowflake-stats-end` as descriptor type
 identifier, which means that future data format versions will have to keep
 that line as their first line. A better choice would have been to use
 something like `snowflake-stats $version` or similar. (If it's any relief,
 we're forced to use `[0-9]{10}` as descriptor type identifier for
 bandwidth files, so there would have been plenty of room to do worse.)
  - The current format only supports a single snowflake broker. Maybe this
 is acceptable for the snowflake design. But just in case that you'll one
 day want to add a second broker, you'll have to include some sort of
 broker identifier in the format.
  - The current format is not signed, which is somewhat related to not
 having a broker identifier in the format.
  - As a consequence of the above, CollecTor needs to make a decision
 whether it wants to archive a newly downloaded snowflake-stats snippet, if
 it already has another snippet with the same timestamp and different
 contents. Possible strategies for this specific case are to a) never
 overwrite, b) always overwrite, c) keep all versions by including a digest
 in the file name, d) maybe something else. I implemented a) for now.

 I think we can start with what we have, without changing anything of the
 above. Of course, if you want to change something with regard to future
 maintenance effort, now's the time!

 irl, please review [https://gitweb.torproject.org/user/karsten/metrics-
 lib.git/commit/?h=task-29461&id=c8f832160a1eaf5c1c57b2cb81fc849bd20cd69e
 metrics-lib commit c8f8321],
 [https://gitweb.torproject.org/user/karsten/collector.git/commit/?h=task-29461&id=cd5a27d3495f2cd8fd7c693e20a406f571d36a2f
 CollecTor commit cd5a27d], and [https://gitweb.torproject.org/user/karsten
 /metrics-
 web.git/commit/?h=task-29461&id=5958454a3d3bb6274367bd696b8cac52cde4f05a
 metrics-web commit 5958454]. If there are more changes, I'll try to add
 those as fixup/squash commits.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29461#comment:9>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online