[tor-bugs] #2672 [Torperf]: Fix bugs/issues with consolidate_stats

[Tor Bug Tracker & Wiki]

#2672: Fix bugs/issues with consolidate_stats
----------------------------------------+-----------------------------------
 Reporter:  mikeperry                   |          Owner:  mikeperry
     Type:  defect                      |         Status:  assigned 
 Priority:  major                       |      Milestone:           
Component:  Torperf                     |        Version:           
 Keywords:  MikePerryIteration20110320  |         Parent:           
   Points:  2                           |   Actualpoints:           
----------------------------------------+-----------------------------------

Comment(by mikeperry):

 Replying to [comment:11 karsten]:
 > Replying to [comment:9 mikeperry]:
 > > All of this + the earlier changes should be in mikeperry/ticket2672.
 >
 > Some comments on that branch:
 >
 >  - Can the note `"This script will strip all failure information from
 the output. [...]"` in `consolidate_stats.py` go away now?

 Yes. I will fix it.

 >  - I noticed that 158 of the 2428 lines in my sample `.data` file don't
 have a matching `.extradata` line with a `USED_AT` entry, but only 22 of
 these 158 runs actually timed out.  What could be the reason for
 `extra_stats.py` not setting `USED_AT` even though the circuit was
 actually used for a successful Torperf run?  I attached my data to this
 ticket; see `50kb.data-not-used` for the runs for which `extra_stats.py`
 finds no matching circuit.

 This is very strange. I will have to dig through the tarball and find out
 what is going on..

 >  - You have a comment `"Add purpose to the .extradata?"` in your code.
 Should we just add it?  Or add a ticket for this?

 Actually, I just checked extra_data.py, and it doesn't even bother to
 record streams whose PURPOSE is not USER. So I don't think this is needed.

 >  - You have another comment `"Torperf only uses 3 hop paths."`.  This
 seems to be correct, or at least I didn't find a case when we used a
 circuit with fewer or more than 3 hops in my sample data.  But why is your
 comment prefixed with `XXX`?

 Hrmm. It is my estimation that we should now have no USED_AT lines with
 path lengths other than 3, because of the PURPOSE filter in extra_data.py,
 so I think both of these XXX checks and comments can go away.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/2672#comment:12>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online