[tor-bugs] #3260 [Analysis]: Learn client speed trends by evaluating directory request download times

[Tor Bug Tracker & Wiki]

#3260: Learn client speed trends by evaluating directory request download times
-----------------------------------+----------------------------------------
 Reporter:  karsten                |          Owner:  karsten 
     Type:  task                   |         Status:  assigned
 Priority:  normal                 |      Milestone:          
Component:  Analysis               |        Version:          
 Keywords:  performance bootstrap  |         Parent:          
   Points:                         |   Actualpoints:          
-----------------------------------+----------------------------------------
Changes (by karsten):

  * status:  new => assigned
  * owner:  => karsten


Comment:

 Replying to [comment:1 arma]:
 > Good idea, especially to see if we can find trends on client speeds.

 Here we go.

 Directory mirrors learn client bandwidths from measuring the time that
 clients take to download a network status consensus.  Directory mirrors
 report various percentiles (min, max, median, quartiles, deciles) of
 client bandwidths in `dirreq-v3-tunneled-dl` lines in their extra-info
 descriptors.  We can aggregate these percentiles and come up with
 percentiles of the overall client bandwidth in the network.

 I looked at the statistics reported by directory mirrors since late 2009.
 Back then, only few relays reported dirreq statistics.  So we have rather
 low-quality data until late 2010 when we explicitly asked operators to
 turn on statistics.  In 2011, statistics have been enabled by default to
 the effect that we have really good data since late 2011.  But despite the
 rather low data quality in 2009 and 2010 we should be able to observe
 basic trends.

 While aggregating statistics, I discarded 50% of statistics lines on a
 given date by number of reported completed downloads.  The idea is to
 exclude statistics from directory mirrors which didn't have enough
 bandwidth themselves to serve fast clients.  I also discarded dates with
 fewer than 10 directory mirrors reporting `dirreq-v3-tunneled-dl` lines.

 Here's the result:

 [[Image(ticket:3260:client-speed-trends.png)]]

 The median client bandwidth was roughly between 100 and 150 KiB/s in the
 past 2.5 years with no clear long-term trend upwards or downwards.  The
 10th percentile was roughly between 10 and 25 KiB/s.

 (Note that colours are less visible in the left half of the graph on
 purpose.  In theory, the data base here is too thin to be plotted at all.
 I left the lines in to visualize that there's no dramatic trend change,
 but other than that we shouldn't conclude too much from the data before
 end of 2010, or even better late 2011.)

 The [https://trac.torproject.org/projects/tor/attachment/ticket/3260
 /client-speed-trends.csv raw data] behind the graph and the
 [https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-3260
 parsing and graphing code] are available, too.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/3260#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online