[tor-bugs] #4086 [Analysis]: Compare performance of TokenBucketRefillInterval params in simulated network

Tor Bug Tracker & Wiki torproject-admin at torproject.org
Thu Mar 15 21:54:13 UTC 2012


#4086: Compare performance of TokenBucketRefillInterval params in simulated
network
-------------------------------------+--------------------------------------
 Reporter:  arma                     |          Owner:       
     Type:  task                     |         Status:  new  
 Priority:  normal                   |      Milestone:       
Component:  Analysis                 |        Version:       
 Keywords:  performance flowcontrol  |         Parent:  #4465
   Points:                           |   Actualpoints:       
-------------------------------------+--------------------------------------

Comment(by robgjansen):

 Replying to [comment:27 arma]:

 > Replying to [comment:26 arma]:
 >
 >
 > > Ok. I'm increasingly thinking that a "run" should be k runs in a row,
 averaged.
 > >
 > >
 >
 > And if we want to get super fancy, we could draw bars on the data points
 in each case, to give a sense of variance between runs. (Does that notion
 even make sense for cdf graphs?)

 Doesn't it make more sense to just show the cumulative results for all
 experiments with the same configuration. I normally do this before
 publishing results in papers, after I am confident I understand the code
 changes and their effects well enough. I'm not sure we are there yet with
 the work in this ticket.

 Though, it would be nice to be able to determine how far one CDF varies
 from another. I attempted to do something like this [comment:15 above in
 comment 15], but I think in that case I wasn't comparing apples to apples.

 I'd like to emphasize that Shadow is already cutting out as many random
 variances as possible. In other words, if I run the same experiment twice
 without changing anything, the results are exactly the same (except for
 memory addresses and timestamps in log files :P). I've verified this
 several times.

 But, it is still the case that a given experiment ''could'' get unlucky
 with its seed to the master PRNG, and a configuration change ''could''
 change the randomness enough to avoid the "unlucky" behaviors. I have not
 tested "run vanilla Tor with several seeds and analyze the variances"
 recently, but its probably a good idea.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/4086#comment:29>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list