[tor-bugs] #33420 [Metrics/Onionperf]: Add CBT events to Onionperf result files

[Tor Bug Tracker & Wiki]

#33420: Add CBT events to Onionperf result files
---------------------------------------+-----------------------------------
 Reporter:  acute                      |          Owner:  karsten
     Type:  enhancement                |         Status:  needs_information
 Priority:  Medium                     |      Milestone:
Component:  Metrics/Onionperf          |        Version:
 Severity:  Normal                     |     Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID:  #33321                     |         Points:  1
 Reviewer:                             |        Sponsor:  Sponsor59-must
---------------------------------------+-----------------------------------
Changes (by karsten):

 * cc: mikeperry, gaba (added)
 * status:  accepted => needs_information


Comment:

 I'm afraid I don't understand what needs to be done here.

 Here's what we have right now: OnionPerf logs `BUILDTIMEOUT_SET` and
 `CIRC` events and includes `TIMEOUT_MS` and `CUTOFF_QUANTILE` values from
 the last known `BUILDTIMEOUT_SET` event when parsing a `CIRC` event with
 status `LAUNCHED`. The result is the following structure in the analysis
 .json file:

 {{{
           "15965": {
             "build_quantile": 0.8, <---------------------------- from
 CUTOFF_QUANTILE
             "build_timeout": 1500, <---------------------------- from
 TIMEOUT_MS
             "buildtime_seconds": 0.25999999046325684,
             "circuit_id": 15965,
             "elapsed_seconds": [
               [
                 "GENERAL:LAUNCHED",
                 0.0
               ],
               [
                 "GENERAL:EXTENDED",
                 0.07999992370605469
               ],
               [
                 "GENERAL:EXTENDED",
                 0.14999985694885254
               ],
               [
                 "GENERAL:EXTENDED",
                 0.25999999046325684
               ],
               [
                 "GENERAL:BUILT",
                 0.25999999046325684
               ],
               [
                 "GENERAL:CLOSED",
                 670.9900000095367
               ]
             ],
             "path": [
               [
 "$1CD48F4ED0F1821FFBF1940802A13EEFD4C27502~Piratenpartei00",
                 0.07999992370605469
               ],
               [
                 "$8C00FA7369A7A308F6A137600F0FA07990D9D451~GrmmlLitavis",
                 0.14999985694885254
               ],
               [
                 "$A99AC59B0C94FBBBC44FA2660CEE1A1362F4C649~deepSpace1001",
                 0.25999999046325684
               ]
             ],
             "unix_ts_end": 1590624905.93,
             "unix_ts_start": 1590624234.94
           },
 }}}

 Note that not all `circuit` objects in OnionPerf's analysis .json files
 contain these two fields. OnionPerf analyzes (processes) log files on a
 daily basis. Now, if a `CIRC` even with status `LAUNCHED` gets fired
 before the first `BUILDTIMEOUT_SET` event arrives for the day, it won't be
 able to tell the quantile and timeout used for building this circuit. The
 result is that these two fields are missing, which we should treat as "we
 don't know what the values were" rather than "there was no timeout set at
 the time".

 Also note that I have not yet seen a `BUILDTIMEOUT_SET` event of
 ''another'' type than `COMPUTED` in the wild. I scanned all past logs from
 op-hk and op-hk2, but did not find a single one. I know they are specified
 and implemented, but I just haven't seen them. I would have expected them
 to occur on op-hk and op-hk2, because those typically have the most
 interesting network conditions of all OnionPerf instances.

 So, what is this ticket about? Do we want to include more information from
 the `BUILDTIMEOUT_SET` event in the `circuit` object? If so, what are we
 interested in?

 And once we have these additional fields (assuming this is what the ticket
 is about), what are we going to do with them? Should we visualize this
 information, or is it sufficient to keep it in the analysis .json files
 for later use?

 Is the goal here to prepare for measuring changes in a future Tor network
 that fires wildly different `BUILDTIMEOUT_SET` events than what we see
 today? If so, what can we expect?

 Setting to needs_information and cc'ing mikeperry and gaba to shed light
 on this before we implement something at random.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33420#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online