[tor-bugs] #29005 [Core Tor/Tor]: PrivCount proof of concept: implement consumed bandwidth counters

[Tor Bug Tracker & Wiki]

#29005: PrivCount proof of concept: implement consumed bandwidth counters
------------------------------+--------------------------------
     Reporter:  teor          |      Owner:  teor
         Type:  defect        |     Status:  assigned
     Priority:  Medium        |  Milestone:  Tor: 0.4.0.x-final
    Component:  Core Tor/Tor  |    Version:
     Severity:  Normal        |   Keywords:  privcount
Actual Points:                |  Parent ID:  #27908
       Points:                |   Reviewer:
      Sponsor:                |
------------------------------+--------------------------------
 We want to add a copy of the current consumed bandwidth statistic to
 PrivCount:
 * consumed bandwidth by relay flags
 https://metrics.torproject.org/bandwidth-flags.html

 We can defer these statistics, because they are not as sensitive to
 manipulation by hostile clients or services:
 * Bandwidth spent on answering directory requests
 * Fraction of connections used uni-/bidirectionally

 We will need counters for:
 * read-history
 * write-history

 Split each counter into 4 (G, E, G and E, not G and not E) using these
 flags:
 * Guard
 * Exit and not BadExit
 We can do the G/E split on the Data Collector (DC).
 We can also do a smaller check aggregation for each group on the Tally
 Reporter (TR).

 Check that relays are Running.
 We can do the Running check on the DC.
 We should also check Running on the TR, and exclude DCs that weren't
 Running at all during the period.

 How often should we update the bandwidth counters?

 If we update counters at the end of the day, we can match the current
 statistics exactly:
 "we only include bandwidth histories for a given day if a relay was listed
 as running in a consensus at least once on that day. We attribute
 bandwidth to guards and/or exits if a relay was a guard and/or exit at
 least in one consensus on a day."
 But we would be storing some sensitive data in memory for the whole day.

 Instead, we could update the counters whenever we queue data, based on the
 flags in the current consensus we have. (The difference will likely be
 minimal.)

 If updating the counters multiple times per second is too CPU-intensive,
 we can update them every few seconds. If that's too often, we can update
 them just before we delete an old consensus.

 Sources:
 https://metrics.torproject.org/reproducible-metrics.html#traffic
 https://trac.torproject.org/projects/tor/wiki/org/meetings/2018MexicoCity/Notes/PrivCount

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29005>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online