[metrics-bugs] #28137 [Metrics/Statistics]: Modify "Total consensus weights across bandwidth authorities" graph to only include relays that end up in the consensus

[Tor Bug Tracker & Wiki]

#28137: Modify "Total consensus weights across bandwidth authorities" graph to only
include relays that end up in the consensus
--------------------------------+------------------------------
 Reporter:  karsten             |          Owner:  metrics-team
     Type:  enhancement         |         Status:  new
 Priority:  Medium              |      Milestone:
Component:  Metrics/Statistics  |        Version:
 Severity:  Normal              |     Resolution:
 Keywords:                      |  Actual Points:
Parent ID:                      |         Points:
 Reviewer:                      |        Sponsor:
--------------------------------+------------------------------

Comment (by karsten):

 Okay, I might have an idea how we can implement this with reasonable
 effort:
  - We import fingerprints into a table and assign numeric identifiers that
 we use in other tables.
  - We import all fingerprints in a consensus together with the votes
 referenced from the consensus.
  - We import all fingerprints in a vote together with a way to refer to
 the consensus coming out of it.
  - When aggregating, we join votes with the consensus they refer to.
  - After aggregating, we delete all votes that we aggregated in the
 previous step and we delete all consensuses ''if'' we aggregated all votes
 referenced from consensuses.

 This approach effectively moves the aggregation step to the database. It
 was more convenient to just do it after parsing, but it's doable in the
 database as well.

 This approach also ensures that tables don't grow forever. I would expect
 that there are just a few votes and consensuses missing from the archives,
 so we'd carry just those fingerprints in the database forever.

 However, this is not a trivial change to the existing "totalcw" module.
 It's basically a rewrite.

 Are there any other changes we might want to make in the near future? For
 example, does #28328 maybe add something here?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28137#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online