[metrics-bugs] #26022 [Metrics/Statistics]: Fix a flaw in the noise-removing code in our onion service statistics

Tor Bug Tracker & Wiki blackhole at torproject.org
Thu May 17 07:22:39 UTC 2018


#26022: Fix a flaw in the noise-removing code in our onion service statistics
--------------------------------+------------------------------
 Reporter:  karsten             |          Owner:  metrics-team
     Type:  defect              |         Status:  needs_review
 Priority:  Medium              |      Milestone:
Component:  Metrics/Statistics  |        Version:
 Severity:  Normal              |     Resolution:
 Keywords:                      |  Actual Points:
Parent ID:                      |         Points:
 Reviewer:                      |        Sponsor:
--------------------------------+------------------------------

Comment (by karsten):

 Here's an updated version of the graph from comment 5 that also goes back
 to late 2014 when we started gathering onion service statistics:

 [[Image(hidserv-change-full-task-26022.png​, 700px)]]

 Some thoughts:
  - Some of the numbers for 2015 produced by the fixed `removeNoise()`
 method are up to 15% smaller than those from the flawed method. During
 2015, very few relays were reporting onion service statistics, with the
 [https://metrics.torproject.org/hidserv-frac-
 reporting.html?start=2014-10-01&end=2018-05-17 fraction not going up
 before late 2015]. The reason for new values being smaller than old values
 is that we're not erroneously adding `bin_size` to negative reported
 statistics anymore.
  - There are very few cases in the first half of 2015 where new values are
 much larger than old values. I believe this is related to another bug in
 our code that made us terminate the module immediately if a consensus did
 not contain a `bandwidth-weights` line. I'm going to fix that, too, but
 it's unrelated to the flawed `removeNoise()` method.
  - The numbers starting in 2016 are almost the same in the new and old
 approach. That's what the previous graph in comment 5 showed, too.

 All in all, I'd say the fixed `removeNoise()` method works just fine.

 I'm starting another run now that uses the simplified `removeNoise()`
 method that only subtracts `bin_size` and does no
 rounding/truncating/flooring at all (as suggested earlier). That will take
 12+ hours.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/26022#comment:14>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the metrics-bugs mailing list