[tor-bugs] #2519 [Metrics]: Change aggregation from daily averages to rolling 24-hour averages

[Tor Bug Tracker & Wiki]

#2519: Change aggregation from daily averages to rolling 24-hour averages
--------------------------+-------------------------------------------------
  Reporter:  karsten      |              Owner:  karsten
      Type:  enhancement  |             Status:  new    
  Priority:  normal       |          Milestone:         
 Component:  Metrics      |            Version:         
  Keywords:               |             Parent:         
    Points:               |   Actualpointsdone:         
Pointsdone:               |       Actualpoints:         
--------------------------+-------------------------------------------------
 We're currently aggregating most stats by calculating daily means.  The
 main reason for doing so is the smoothing effect that makes it easier to
 understand trends.  Also, daily aggregates make it easier to keep our
 materialized views up-to-date.  In general, daily aggregates are
 sufficient when we're interested in long-term developments of 1 month or
 more.

 There are at least two shortcomings of daily averages:  We need to wait
 until at least half a day is over, better one day, before displaying data
 for that day.  Another shortcoming is that 1 data point per day is not
 enough when looking at short time intervals of, say, one to two weeks.

 Instead of daily averages, we could use rolling 24-hour averages.  Every
 data point would be the average (mean) of the 24 hours ending at that data
 point.  The 24-hour rolling average removes intra-day patterns and gives
 us a smooth curve, too.  I attached an example of running relays with the
 raw data and the 24-hour rolling average.  Compare this to our
 [https://metrics.torproject.org/network.html?graph=relayflags&start=2011-01-26&end=2011-02-02&flag=Running&granularity=day&dpi=72#relayflags
 current graph].

 If we decide we want to try rolling averages, I'll have to fight R some
 more.  We should start with the relay flags graph and add other graphs
 based on the network status consensus.  Graphs based on the bandwidth
 histories in extra-info descriptors, including our user number estimates,
 are going to be more difficult.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/2519>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online