[metrics-bugs] #25774 [Metrics/Ideas]: Record latency measurements using OnionPerf

[Tor Bug Tracker & Wiki]

#25774: Record latency measurements using OnionPerf
---------------------------+------------------------------
 Reporter:  irl            |          Owner:  metrics-team
     Type:  project        |         Status:  merge_ready
 Priority:  Medium         |      Milestone:
Component:  Metrics/Ideas  |        Version:
 Severity:  Normal         |     Resolution:
 Keywords:                 |  Actual Points:
Parent ID:                 |         Points:
 Reviewer:  irl            |        Sponsor:
---------------------------+------------------------------
Changes (by karsten):

 * status:  needs_revision => merge_ready


Comment:

 Replying to [comment:13 irl]:
 > Replying to [comment:11 karsten]:
 > > I experimented a bit with this and came to the conclusion that we
 should probably avoid mixing 50 KiB, 1 MiB, and 5 MiB measurements and
 instead focus on 1 MiB measurements only. Otherwise we might see confusing
 results where, for example, partial 5 MiB downloads are completed faster
 than full 1 MiB downloads on some days. Also, I could imagine that 1 MiB
 is still a reasonable size for LTE experiments, whereas 5 MiB is too large
 and 50 KiB too small.
 >
 > My understanding for this one was that instead of performing three
 downloads, we would modify OnionPerf to only perform the one download, and
 derive values for smaller file sizes from the partial completion times.
 >
 > Perhaps we need to modify OnionPerf to report different numbers, instead
 of percentiles. I think useful sizes would be 50K, 200K, 500K, 1M, 2M and
 5M. To conserve bandwidth on the LTE probe(s) we could limit the downloads
 to 1M, although we'd have to see if we actually want to include those on
 Tor Metrics or just do a one-off analysis.

 Ah, got it. I'd say let's postpone this topic then, as it requires making
 code changes to OnionPerf and setting up new measurements which is out of
 scope for the current roadmap.

 I'll finally open a new ticket for this before closing this one.

 > I am confused as to why exit circuits completed faster than onion
 circuits. Are you comparing `DATAPERCx` to `DATARESPONSE` or to `START`? I
 wonder if there's enough overhead in circuit setup that 1M is not enough
 time to see benefit from using an onion circuit instead.

 I compared `DATAPERCx` to `START` to make the graph comparable to existing
 graphs. Keep in mind that onion circuits are twice as long as compared to
 exit circuits, so that downloads should still take longer after circuit
 setup. I didn't compare to `DATARESPONSE`, though.

 > Replying to [comment:12 karsten]:
 > > In addition to the third graph on partial downloads above, please also
 review [https://gitweb.torproject.org/karsten/metrics-
 web.git/commit/?h=task-25774&id=2761d1f733989a5c5aa5d9a4af69ea9153bc4b8f
 commit 2761d1f in my task-25774 branch] which implements the first two
 graphs on build times and latencies as discussed earlier.
 >
 > I do not have an environment set up for metrics-web's statistics or
 Rserve, only for running enough to test Relay Search at the moment, so I
 have not run the code. The titles, descriptions and URLs all look good and
 I did not find any obvious errors in the Java, R or SQL.
 >
 > I would not object to this being merged, but it's up to you if you want
 to wait for me to have a test environment set up.

 That's good enough. Having descriptions reviewed matters most to me, and
 it's good to know that there are no obvious errors in the code. I'll find
 any remaining bugs when deploying things. (FWIW, I don't have a full
 metrics-web setup here, either.)

 Thank you! I'll merge and deploy now. Setting to merge_ready for the first
 and second graph only, the third graph will go into its own ticket.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25774#comment:14>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online