[tor-bugs] #8296 [Tor]: Add note to dir-spec.txt that cell-stats can be inaccurate

[Tor Bug Tracker & Wiki]

#8296: Add note to dir-spec.txt that cell-stats can be inaccurate
-------------------------+--------------------------------------------------
 Reporter:  karsten      |          Owner:  nickm
     Type:  enhancement  |         Status:  new  
 Priority:  normal       |      Milestone:       
Component:  Tor          |        Version:       
 Keywords:  tor-spec     |         Parent:       
   Points:               |   Actualpoints:       
-------------------------+--------------------------------------------------
 Rob and I found that cell-stats can be inaccurate under certain
 circumstances.  We should update dir-spec.txt to say this.  Making cell-
 stats more accurate should wait until we're clearer what other cell-stats
 we'd like to see.

 See the attached diagram for an example.  For cell stats, we only look at
 ''processed'' cells, here purple.  For the mean queue length (`cell-
 queued-cells`), we sum up waiting times, here 750 + 750, and divide by
 measurement interval length.  This circuit was opened before the
 statistics interval and not closed during this interval, so the interval
 length is 1000.  As a result, we come up with 1.5 as mean queue length.
 That's not quite correct.  As one can see, there's one cell in the queue
 for most of the time, with only two short phases with two cells.

 The correct way to calculate mean queue length would be to only sum up
 waiting times within the statistics interval, and to include the green
 cell that was not processed during the interval.  The result would be a
 mean queue length of 1.25.

 Similarly, mean waiting time (`cell-time-in-queue`) is currently defined
 as mean waiting time of ''processed'' cells, so (750 + 750) / 2.  We might
 also consider only waiting times inside the observed interval and include
 non-processed cells.  However, this is more complicated to calculate in a
 meaningful way.

 These inaccuracies are hardly relevant for 24-hour intervals, because only
 a small fraction of circuits is open in two such intervals.  But it
 matters for shorter intervals, like in Rob's simulated Shadow networks.
 We think that adding a note to dir-spec.txt and possibly coming up with
 more accurate cell-stats later should be fine for now.

 Please find branch
 [https://gitweb.torproject.org/user/karsten/torspec.git/shortlog/refs/heads
 /cell-stats-note cell-stats-note in my public torspec repository].

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/8296>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online