[tor-scaling] Analyzing the Predictive Capability of Tor Metrics

[Roger Dingledine]

On Wed, Jul 03, 2019 at 11:43:23AM +1000, teor wrote:
> It looks like latency varies a lot before 2015, and throughput varies
> a lot after late 2013.

The theorized explanation here is that throughput varying a lot
is at least in part a good thing -- before 2015, latency was all
over the place, because things weren't working right (there was too
much congestion, not enough capacity, whatever the reason will turn
out to be), and before 2015, throughput was *consistent*, but it was
consistently *bad*. Whereas after 2015, when the bottleneck issues let
up a bit, latency became better, and throughput often (but not always)
became better.

Making the throughput both consistent and consistently good will probably
require figuring out what the deal is what the small fraction of paths
that are crappy. Maybe it is related to a small fraction of relays that
are crappy. Or maybe it is related to transient congestion, which might
just be a fact of life for a network with millions of users producing
high-variance flows. Transient congestion issues could be harder to
fix with path selection changes, and might instead require per-path
congestion-noticing-and-reacting designs like
https://www.freehaven.net/anonbib/#congestion-tor12

> And due to the way we
> measure capacity, we may also "discover" extra Total Capacity as
> Throughput increases.

Speaking of this one, I encouraged Rob to do an experiment where he
puts load through each relay for a 20-second burst. The goal would
be to learn how much extra capacity the relays "discover" this way:
is it 5%? 50%? 500%? Is it evenly distributed?

It will help him with the load balancing paper he's working on, it will
give us better intuition about how far off our self-measured relay
capacities are, and it will have minimal impact on the network to do
the experiment once or twice.

--Roger