[tor-bugs] #6593 [Metrics Website]: metrics graphs are dropping at the end again

[Tor Bug Tracker & Wiki]

#6593: metrics graphs are dropping at the end again
--------------------------------+-------------------------------------------
    Reporter:  arma             |       Owner:        
        Type:  defect           |      Status:  closed
    Priority:  normal           |   Milestone:        
   Component:  Metrics Website  |     Version:        
  Resolution:  fixed            |    Keywords:        
      Parent:                   |      Points:        
Actualpoints:                   |  
--------------------------------+-------------------------------------------
Changes (by karsten):

  * status:  new => closed
  * resolution:  => fixed


Comment:

 You probably mean direct and bridge user graphs.  The network graphs
 should be fine, unless there are other problems I didn't see right now.

 Yes, we did change the number of days we cut off at the end of each graph.
 Looks like we're not cutting off enough days.  Here's an example of the
 worst case of days we need to cut off:

  - August 13: We want to graph direct or bridge usage stats for all of
 August 13.
  - August 14, 23:00: A relay or bridge has its stats interval ending at
 23:00 UTC, so the first stats interval containing the last August 13
 numbers ends on August 14, 23:00 UTC.
  - August 15, 17:00: Descriptors are published at least every 18 hours, so
 it may take until August 15, 17:00 UTC to include those stats.  That's
 also when metrics-db grabs the extra-info descriptor and metrics-web
 imports it.
  - August 16, 07:45: It takes until August 16, 07:45 UTC for metrics-web
 to finish its daily database update.
  - August 16, 13:45: Graph images are cached for up to 6 hours, so it may
 take until 13:45 UTC until drawing a graph that correctly includes August
 13.
  - August 17, 00:00: When counting in days, the first graph showing user
 numbers from August 13 can be drawn on August 17 at 00:00 UTC.

 So, to be sure, when drawing a graph, we should exclude the current day
 and the 3 previous days.  With the latest patch we were only excluding the
 current day and the day before.  Fixing.

 In the future we may reduce the time between relays or bridges publishing
 a descriptor and updating the database and graphs.  That won't help with
 stats intervals not ending at 00:00 UTC and relays publishing extra-info
 descriptors only every 18 hours.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/6593#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online