[tor-bugs] #8662 [Torperf]: Make Torperf log circuit failures

[Tor Bug Tracker & Wiki]

#8662: Make Torperf log circuit failures
-------------------------+--------------------------------------------------
 Reporter:  karsten      |          Owner:     
     Type:  enhancement  |         Status:  new
 Priority:  normal       |      Milestone:     
Component:  Torperf      |        Version:     
 Keywords:               |         Parent:     
   Points:               |   Actualpoints:     
-------------------------+--------------------------------------------------

Comment(by karsten):

 Hmm, okay, let's give up the one-output-line-per-meausurement format.  If
 we need this data for more than a few weeks, we should publish it.  And
 while I see how we could aggregate circuit failures in a single new field
 for it, this isn't going to scale forever.  I'm certain you'll want other
 circuit information in the future, and then we'd have this discussion
 again.  So, let's screw the format we have and come up with something new.
 No better time than now to do this.

 Here's a sample result line from a currently running 0.2.5.x Torperf
 (reformatted and annotated):

 {{{
 # Configuration
 SOURCE=ec2
 FILESIZE=51200

 # Measurement results
 START=1365473921.75
 SOCKET=1365473921.75
 CONNECT=1365473921.75
 NEGOTIATE=1365473921.75
 REQUEST=1365473921.75
 RESPONSE=1365473922.07
 DATAREQUEST=1365473922.07
 DATARESPONSE=1365473922.40
 DATACOMPLETE=1365473922.76
 WRITEBYTES=82
 READBYTES=51323
 DIDTIMEOUT=0
 DATAPERC10=1365473922.56
 DATAPERC20=1365473922.64
 DATAPERC30=1365473922.64
 DATAPERC40=1365473922.68
 DATAPERC50=1365473922.68
 DATAPERC60=1365473922.72
 DATAPERC70=1365473922.72
 DATAPERC80=1365473922.75
 DATAPERC90=1365473922.75

 # Additional information
 LAUNCH=1365473682.13
 USED_AT=1365473922.76
 PATH=$62680CF0743460E5F836F949E37A6DEC22622F9E,
      $11064D066F892DC38AAEFDA5EDAE1A227D07D182,
      $35F51DCE73B988CBAE06B06312B4D1271979FE3B
 BUILDTIMES=0.53,1.10,1.32
 TIMEOUT=4833
 QUANTILE=0.800000
 CIRC_ID=501
 USED_BY=884
 }}}

 I could imagine a JSON format with three event types for file_download,
 circuit, and buildtimeout_set:

 {{{
 [
   {
     "type": "buildtimeout_set",
     "id": "1365473682.03",
     "timeout": 4833,
     "quantile": 0.800000
   },
   {
     "type": "circuit",
     "id": "1365473682.13_501",
     "launch": 1365473682.13,
     "used_at": 1365473922.76,
     "path": [ "$62680CF0743460E5F836F949E37A6DEC22622F9E",
               "$11064D066F892DC38AAEFDA5EDAE1A227D07D182",
               "$35F51DCE73B988CBAE06B06312B4D1271979FE3B" ],
     "buildtimes": [ 0.53, 1.10, 1.32 ],
     "circ_id": 501,
     "used_by": 884
   },
   {
     "type": "file_download",
     "id": "ec2_51200_1365473921.75",
     "source": "ec2",
     "filesize": "51200",
     "start": 1365473921.75,
     "socket": 1365473921.75,
     "connect": 1365473921.75,
     "negotiate": 1365473921.75,
     "request": 1365473921.75,
     "response": 1365473922.07,
     "datarequest": 1365473922.07,
     "dataresponse": 1365473922.40,
     "datacomplete": 1365473922.76,
     "writebytes": 82,
     "readbytes": 51323,
     "didtimeout": 0,
     "dataperc": { "10": 1365473922.56,
                   "20": 1365473922.64,
                   "30": 1365473922.64,
                   "40": 1365473922.68,
                   "50": 1365473922.68,
                   "60": 1365473922.72,
                   "70": 1365473922.72,
                   "80": 1365473922.75,
                   "90": 1365473922.75 },
     "circ_id": "1365473682.13_501",
     "buildtimeout_used": "1365473682.03"
   }
 ]
 }}}

 This is just a draft that may change during implementation.  We could also
 add a separate event type for more detailed information about the stream
 used for the measurement.

 We can easily add reasons for circuit failures, which would mostly be in
 "circuit" events that are ''not'' referenced from "file_download" events.

 Anything else that comes to mind that we should add?

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/8662#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online