[tor-bugs] #21551 [Metrics/Metrics website]: Merge Metrics sub sites into main Tor Metrics website

[Tor Bug Tracker & Wiki]

#21551: Merge Metrics sub sites into main Tor Metrics website
-------------------------------------+------------------------------
 Reporter:  karsten                  |          Owner:  metrics-team
     Type:  enhancement              |         Status:  new
 Priority:  High                     |      Milestone:
Component:  Metrics/Metrics website  |        Version:
 Severity:  Normal                   |     Resolution:
 Keywords:                           |  Actual Points:
Parent ID:                           |         Points:
 Reviewer:                           |        Sponsor:
-------------------------------------+------------------------------
Changes (by karsten):

 * priority:  Medium => High


Comment:

 Agreed on moving
 [https://trac.torproject.org/projects/tor/ticket/21551#comment:1 comment 1
 above] to #19754.

 But regarding the original description, not moving forward with this
 ticket is becoming a liability. We're running
 [https://gitweb.torproject.org/karsten/metrics-web.git/log/?h=staging4 my
 staging4 branch] on the Tor Metrics website for a few months now, because
 we haven't merged the `collector.html` and `onionoo.html` pages into
 [https://gitweb.torproject.org/metrics-web.git/log/ master] yet. I'm
 cherry-picking new commits from the master branch and sometimes resolving
 conflicts while doing so. I don't expect it to be much fun to rebase my
 staging4 branch to master, and it'll become even less fun over time.

 Another issue is that I'm avoiding making changes to the CollecTor and
 Onionoo pages, because we'd currently have to make them in two places: in
 their respective repos and in my metrics-web staging4 branch. As a result,
 the typos fixed in #22169 are only fixed on
 https://onionoo.torproject.org/, but not on
 https://metrics.torproject.org/onionoo.html. And I'm holding back the
 #22263 change, because I want to avoid applying that patch twice.

 In short, let's make some decisions here and implement them. Open points,
 and my suggestions, are:

  - How should we support the use case of mirror operators running modified
 versions of Onionoo or CollecTor and wanting to reflect that by providing
 a modified specification page?
    - Our original plan was to keep specifications in the Onionoo or
 CollecTor repositories and only include them in metrics-web's `.war` file
 using a Git submodule and some Ant magic. This turns out to be non-
 trivial. While the bulk of pages stays the same, there are a lot of subtle
 differences.
    - ''My suggestion:'' How about we kill the `.html` pages in the Onionoo
 and CollecTor repositories and consider the pages in metrics-web as new
 masters? After all, Onionoo and CollecTor are Metrics subprojects and the
 Tor Metrics website is the primary website for Metrics projects. And if
 mirror operators want to adapt these pages, they can quite easily `wget`
 and update them as needed. I feel we're otherwise optimizing for a quite
 special use case here, making the main use case of providing
 specifications for Metrics services unnecessarily hard.
    - In the future we could consider providing a schema file on both
 Onionoo and Collector which metrics-web fetches and uses to generate a
 human-readable specification page. This is possibly even a fun project,
 but it's not one that should delay moving away from staging4 much longer.
  - Should we let users browser CollecTor files on the Tor Metrics website
 rather than in Apache-generated directory listings on the CollecTor page?
    - ''My suggestion:'' I'd say yes, but that can be done in a subsequent
 patch. Also a fun project.
  - How do we include `.onion` links on the placeholder pages on Onionoo
 and CollecTor?
    - We briefly speculated about possible ways to detect how the user
 accesses the placeholder page, and only show the onion or non-onion link.
    - ''My suggestion:'' For now, I'd say let's just show both.
  - How do we support the case of different Onionoo hosts running different
 Onionoo protocol versions?
    - We said we'd include something like "added in version X" and "removed
 in version Y" in the specification. Sounds totally doable, but maybe in
 the near future as a subsequent patch.
    - ''My suggestion:'' For now, let's just show the latest protocol
 version specification, because there's really just one Onionoo protocol
 version deployed.

 If these suggestions sound reasonable, I'll rebase my staging4 branch to
 master, include the #22169 and #22263 fixes, and put it here for review.
 And then we'll create tickets for the suggested follow-up tasks (schema
 files for Onionoo and CollecTor, browse CollecTor files based on
 `index.json`, create placeholder pages with onion and direct link to Tor
 Metrics, include information when an Onionoo document or field was
 added/removed) and do them in the near future as time permits.

 Thoughts, ideas, things that I overlooked? Thanks!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21551#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online