[metrics-bugs] #28137 [Metrics/Statistics]: Modify "Total consensus weights across bandwidth authorities" graph to only include relays that end up in the consensus

[Tor Bug Tracker & Wiki]

#28137: Modify "Total consensus weights across bandwidth authorities" graph to only
include relays that end up in the consensus
--------------------------------+------------------------------
 Reporter:  karsten             |          Owner:  metrics-team
     Type:  enhancement         |         Status:  new
 Priority:  Medium              |      Milestone:
Component:  Metrics/Statistics  |        Version:
 Severity:  Normal              |     Resolution:
 Keywords:                      |  Actual Points:
Parent ID:                      |         Points:
 Reviewer:                      |        Sponsor:
--------------------------------+------------------------------

Comment (by teor):

 Replying to [comment:7 karsten]:
 > Replying to [comment:6 teor]:
 > > Can you do each consensus separately?
 >
 > I think that's a database optimization question in this case. I suspect
 that we're somehow matching vote entries with ''all'' consensus entries in
 the database, not just those entries that are connected via the same
 valid-after time, at least in an intermediate step. Maybe we'd have to do
 something with subselects here in order to assist the database.

 Or carefully constructed indexes?
 (For example, an index on all the join conditions, in the correct order.)

 Or datetimes converted to integers?
 (I remember native datetimes being slower to match than integers, but that
 depends on how the DBMS stores and compares them.)

 (My most recent experience is with SQL Server 2012, but I assume that
 DBMSs are roughly equivalent.)

 > > > What I'd like to try out is add a third line "Running in vote",
 which would at least kick out relays in a vote that the authority didn't
 find to be running. I'd expect that line to show up between red and blue.
 However, a relay that doesn't have the Running flag in one vote can still
 go into the consensus if the others think it's running. And a relay that
 has the Running flag from one authority can still not show up in the
 consensus if the others disagree. So, I'm unclear whether this really
 helps. Worth trying, and a much smaller change, because it doesn't require
 us to match vote entries with consensus entries.
 > >
 > > Let's see the results for Running, then decide.
 >
 > It's here:
 >
 > [[Image(totalcw-2018-11-22a.png, 500px)]]
 >
 > I had to increase line size and make lines transparent in order to make
 the green line visible at all: it almost always overlaps with the blue
 line. One exception is maatuska at the beginning of November; possibly
 related to its own reachability testing not working very well? Another
 minor exception is Faravahar on the 4th and 10th, though the difference
 there is really small.
 >
 > I tend towards green, because it adds way less code and delivers
 basically the same result.

 Seems fine to me.

 > I'd also break down vote totals by Guard/Exit flag combination (#28328).
 Note, however, that these Guard and Exit flags would also be taken from
 the vote, not from the consensus. Mostly a documentation issue again.
 >
 > And I'd include consensus totals (#28352). Just mentioning those for
 completeness.
 >
 > How does this sound?

 These all seem fine to me.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28137#comment:8>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online