[tor-relays] Measuring the Accuracy of Tor Relays' Advertised Bandwidths

[Matt Traudt]

On 8/6/19 7:05 PM, grarpamp wrote:
> On 8/6/19, Roger Dingledine <arma at torproject.org> wrote:
>> On Tue, Aug 06, 2019 at 05:31:39PM -0400, Rob Jansen wrote:
>>> Today, I started running the speedtest on all relays in the network.
> 
>> There will be another confusing (confounding) factor, which is that the
>> ...
>> as intended. :) So, call it another thing to keep an eye out for during
>> the experiment.
> 
> Someone here posted they were testing with sub-minute
> durations... tens of seconds. That's unlikely enough to
> allow TCP to adjust over everything in the circuit to really
> measure "bandwidth". And is instead likely to be measuring
> something between "setup latency" and that, with an
> uncharacterized ramp in the middle.
> 

- That person was Rob, the one who just said they've started their
measurements. Rob's original announcement is here[0].

- We've been looking into stuff like this for the last year and have
some promising results from sub-minute-duration measurements with
measurement hosts spread around the world. This is despite suspected
sources of inaccuracy such as TCP slow start and high bandwidth delay
products.

- What Rob is doing isn't even trying to get an accurate measurement of
a relay's capacity. It's solely to test the hypothesis that observed
bandwidth is a poor estimate of capacity*. I refer again to [0] for the
motivation, design, etc.

> You probably want to be scatterplotting a bunch
> of different things and durations on metrics.tpo.
> 
> And isolating out path nodes and things from
> whichever it is you're trying to measure.
> Introducing known inputs. Etc.

Thanks for the input. I'm sure our analysis of the collected data will
be ... thorough and exciting.

Matt

* You might argue that an artificial 20 second or even 2 minute burst of
traffic is still bad estimate for long term sustained capacity. That may
be a good argument. But I'd argue that it's strictly better than the
current strategy: keep track of your biggest natural 10 second burst
observed in the last 5 days.

[0]: https://lists.torproject.org/pipermail/tor-relays/2019-July/017535.html