[tor-bugs] #7358 [Tor]: Decide on list of stats to collect

[Tor Bug Tracker & Wiki]

#7358: Decide on list of stats to collect
------------------------------------------------------------------------+---
 Reporter:  robgjansen                                                  |          Owner:                    
     Type:  task                                                        |         Status:  new               
 Priority:  normal                                                      |      Milestone:  Tor: 0.2.4.x-final
Component:  Tor                                                         |        Version:                    
 Keywords:  performance, simulation, statistics, tor-relay, tor-client  |         Parent:  #7357             
   Points:                                                              |   Actualpoints:                    
------------------------------------------------------------------------+---

Comment(by karsten):

 Replying to [ticket:7358 robgjansen]:
 > Client
 >    * download statistics (how long to get to first, last, ? byte)
 >    * circuit build times, build timeouts
 >    * which relays were chosen for each circuit, and during which time
 intervals

 This is basically what Torperf does.  The first item is what trivsocks-
 client tells us, and the second and third items are what we learn from
 control port events.  Implementing the first item in Tor using control
 port events is sure tempting.  We lack some information, e.g., the exact
 timestamp when Torperf started the download and the expected number of
 bytes we want to download.  But maybe we can compensate for that.  For
 example, we could collect times between sending the first byte and
 receiving 1B, 1kB, 2kB, 5kB, 10kB, 20kB, 50kB, 100kB, 200kB, 500kB, and so
 on, up to the last byte.  Not exactly the same as the deciles we have so
 far, but should be fine for most purposes.  Also, we'll have to do that
 for all circuits that the client opens on behalf of the user.

 Another item for the client section here might be directory client
 operations.  We might want to keep track how many directory requests a
 client has made and how many bytes it has sent or received for that.  Then
 we can compare different directory designs more easily.

 Also, I think we need to move all statistics from client+relay that have
 to do with streams to the client-only section.

 > client+relay
 >    * cell statistics: # queued, processed, waiting times
 >    * total number of or connections, circuits, and streams over time

 What about other connections than OR?

 >    * various throughputs (stream, circuit, connection) over various
 intervals (last second, 10 seconds, 60 seconds, 300 seconds, ? seconds)
 >    * when steams, circuits, or connections change active/inactive status
 >    * indications of congestion (inferred by how fast/often token buckets
 were emptied/empty, queuing times from above)

 I assume these will all be implemented as asynchronous control port
 events, right?  Which of them will be emitted whenever there's a change,
 and which will be emitted periodically?

 Another item might be statistics on crypto operations as described in
 #7134, but without the aggregation step that isn't necessary if we collect
 these statistics in a simulation/testing environment.  The two can
 probably share a lot of code.

 > relay
 >    * protocol overheads (raw client data vs protocol traffic)

 We also have statistics on bi-directional connection usage already in Tor.
 But these are probably contained somewhere in the client+relay section.

 And we might add statistics on directory server operations with the same
 reasoning as adding directory client operations.

 > What am I missing?

 Not sure, I guess we'll find out while working off this list.  Count me
 in, this is fun stuff. :)

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/7358#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online