[tor-bugs] #25383 [Metrics/Website]: Deprecate stats.html and stats/*.csv files

[Tor Bug Tracker & Wiki]

#25383: Deprecate stats.html and stats/*.csv files
-----------------------------+------------------------------
 Reporter:  karsten          |          Owner:  metrics-team
     Type:  enhancement      |         Status:  needs_review
 Priority:  High             |      Milestone:
Component:  Metrics/Website  |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:                   |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------------
Changes (by karsten):

 * status:  assigned => needs_review
 * priority:  Medium => High


Comment:

 I made more progress on this ticket. Going through the remaining steps
 from comment 16 above:

 > Next steps after that, in no particular order:
 >  - Decide where to add the legend (Java or R).

 Maybe the CSV file header is not the right place for this legend after
 all. The specification of parameters and columns can be quite long, and if
 we also plan to include scheduled and past changes, the header section
 will be even longer. Oh, and whatever we write here won't change in the
 CSV file that somebody downloaded, until they decide to download a new CSV
 file from us.

 I tried out something else: extend our existing `stats.html` to also cover
 the per-graph CSV files. The CSV file header could then include a link to
 that page or possibly even a subsection on that page.

 I'll post a branch shortly.

 >  - Discuss whether we want to use wide/long format for these CSVs. Yes,
 we should have had this discussion a few weeks back, but it's better to
 have it next week than never.

 I made remarks in the extended `stats.html` page to change the format.
 This could be the first scheduled change that would become effective a
 couple weeks later.

 >  - Decide how we announce and make changes in the future, in particular
 backward-incompatible ones. For example, Onionoo has a
 `"next_major_version_scheduled"` field to announce backward-incompatible
 changes, and we need something like that, too.

 We could include remarks like the ones I made on `stats.html`, and we
 might even add a change log to the top of that page to summarize past and
 upcoming changes.

 >  - Add a note to stats.html saying when it's going to go away.

 In the page.

 >  - Add a note to CSV file header saying it's still BETA until the same
 date as mentioned on stats.html, maybe with 2 or 4 weeks overlap.

 I did not touch CSV file headers yet. Once we have a fixed deprecation
 date, let's include it there.

 Alright, please review [https://gitweb.torproject.org/karsten/metrics-
 web.git/commit/?h=task-25383-2&id=cc81eea95ea58767aecc414f5a45165e27bf9f3a
 commit cc81eea in my task-25383-2 branch]. Couple questions:
  - Does it make sense to specify our per-graph CSV files there, rather
 than in the CSV file header?
  - Is the format with two subsections Parameters and Columns okay? Is
 something missing?
  - Are specifications roughly correct/plausible?
  - Do the suggestions make sense? The rule of thumb for deciding which
 columns we need was: "it should require a code change to change columns,
 and neither the user should be able to control which columns exist by
 their choice of parameters, nor should the available data have any
 influence on that."

 Regarding timing, how about we deploy this page still in July, make
 suggested changes by August 15, take out pre-aggregated stats files by
 September 15, and handle any questions coming out of that in the two weeks
 before the Mexico City meeting?

 Changing priority back to high for the still-in-July bit. Thanks!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25383#comment:29>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online