[tor-bugs] #1766 [Metrics]: Project: "Improve TorPerf for more accurate measurements."

[Tor Bug Tracker & Wiki]

#1766: Project: "Improve TorPerf for more accurate measurements."
---------------------+------------------------------------------------------
 Reporter:  nickm    |       Owner:  karsten            
     Type:  task     |      Status:  assigned           
 Priority:  normal   |   Milestone:  Deliverable-Sep2010
Component:  Metrics  |     Version:                     
 Keywords:           |      Parent:                     
---------------------+------------------------------------------------------

Comment(by arma):

 I'm not sure.

 Andrew, can you give us some guidance about what the sponsor expects here?
 Or is there flexibility to turn this into whatever we see fit?

 My guess is that Andrew listed this item based on overhearing discussions
 about the dev summit. I bet the discussion went something like "we should
 keep in mind that torperf's results are an average of all user results,
 and that some users will have better or worse results based on what guards
 they pick", and so Andrew wanted us to make torperf more realistic.

 So on the one hand, we might say that torperf is already doing a fine job
 of letting us track average expected performance of the network over time.

 Here's my first go at brainstorming a way to make torperf's output more
 useful. It would be good to see what performance the, say, 90th percentile
 of users see -- the people who are unlucky enough to pick guards that are
 slower than most other people pick. One approach would be to run 100
 torperfs with entryguards on, and order them by performance, on the
 assumption that whoever gets bad torperf results clearly has worse guards
 (as long as we have enough data points). Yuck. A better approach would be
 to annotate torperf output with which first hop was used, and then do some
 smart analysis to reconstruct what the torperf graphs *would* have looked
 like for various combinations of guards they would have picked. I think
 that could yield some really useful results, if we can get the smart
 analysis right. That method would also let us simulate different guard
 selection algorithms, and reconstruct what the torperf output would have
 looked like if you'd picked that set.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/1766#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online