[tor-bugs] #5398 [Analysis]: Compare variance between simulation runs (Shadow and Experimentor)

[Tor Bug Tracker & Wiki]

#5398: Compare variance between simulation runs (Shadow and Experimentor)
----------------------+-----------------------------------------------------
 Reporter:  arma      |          Owner:     
     Type:  task      |         Status:  new
 Priority:  normal    |      Milestone:     
Component:  Analysis  |        Version:     
 Keywords:            |         Parent:     
   Points:            |   Actualpoints:     
----------------------+-----------------------------------------------------
 In #4086 we noticed that a given simulation can have quite a bit of
 variance depending on the topology selected, and what paths clients pick.

 How much exactly, and what can we do to quantify it and to reduce it?

 The suggestion so far is to do k runs, and then either add them all up as
 data points that go into the CDF, or look at the CDFs separately and see
 how different they are from one run to the next.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/5398>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online