[metrics-team] Measuring bwscanner changes

[micah anderson]

Hi all,

We have been working on making some changes in bwscanning, and we
*think* that the changes are going to improve measurements on the
network as a whole, but this is only a guess and we would like to find
out if there are any ways to determine with some actual metrics what
kind of affects the changes do actually have.

As of right now, one of the bwauthorities is pulling its bandwidth files
from two locations: Hong Kong, and Santiago, Chile. The Chile change was
made relatively recently (approximately April 13th), and we are
wondering if this had any measurable impact on the network.

The theory is that Bandwidth Authority location matters because Tor
sends more Guard and Middle bandwidth to relays close to the bandwidth
scanner, and more Exit bandwidth to relays close to the bandwidth
server.

Additionally, the location impact on tor clients is likely impacted. The
current bandwidth authority locations mean that relays in North America
and Europe handle more traffic[0]:

. The Tor network is faster for all clients. Clients are more likely to
choose a path with relays that are near each other. This affects hidden
services the most, because they have 6-hop long paths.

. But Exits in North America and Europe can get overloaded, and slow
down the network for all clients. This does not happen as much for
Guards, because there are more Guards than Exits. (TODO: measure Exit
congestion)

. Tor clients in North America and Europe are even faster, because their
Guard is closer (on average).

. Websites with servers in North America or Europe are faster through
Tor Exits (on average).

. Websites that use a CDN are faster if the CDN DNS sends the connection
to nearby data center, and if the CDN has many servers in North America
or Europe.

Another theory is that adding or moving bandwidth authorities changes
relay measurements: relays closer to the new location will be measured
higher. A bandwidth scanner affects Guard and Middle measurements. A
bandwidth server affects Exit measurements.

So... What happens if we add a bandwidth server in South America to a
bandwidth authority?

Teor sketched out a number of theories that it would be interesting to
find out are valid with some metric data:

. Adding a bandwidth server in South America will shift Exit bandwidth
away from Europe, and maybe North America.

. Websites in South America will become faster through Tor. (Tor is
mainly used for web traffic.)

. The average distance between middles and exits will increase, so tor
will become slightly slower. But there will be less load on European
Exits, which will make them faster for all clients. (TODO: work out
which effect wins?)

. Guards and Middles that are closer to Exits that are closer to South
America get more bandwidth. But it matters much more if Guards and
Middles are close to the scanner.

. People will put more Exits (and relays) in South America, because they
measure better.

. There will be a small change to a small number of relays. We use the
median measurement, so changing 1/5 bandwidth authorities will not
change many relays. And changing 1/2 bandwidth servers on 1 authority
makes the size of the impact small. We would need to change scanners and
servers on 3/5 bandwidth authorities to change a lot of relay
measurements.

So how can we figure these things out? TorPerf/OnionPerf tracks the Tor
network's overall speed, so they could help us work out whether the
network is faster or slower after the change. Are we using TorPerf or
Onion perf for reporting? Where are the current TorPerf measurement
nodes?

The current OnionPerf nodes are in EU/US/HK. We could put OnionPerf
nodes in South America, Africa, and Australia to make the measurements
more representative:

https://trac.torproject.org/projects/tor/wiki/org/operations/Infrastructure/onionperf

In addition to watching overall network performance with TorFlow, there
are existing graphs for measured relays here:
https://consensus-health.torproject.org/graphs.html#bwauthstatus

But that's not really enough. I think it would also be helpful to know:

Graph for number of relays that bwauths decided the median for
https://trac.torproject.org/projects/tor/ticket/21882 (There is a sample
graph on that ticket)

What is the distribution of a bandwidth authority's measurements?
https://trac.torproject.org/projects/tor/ticket/21994 I'm not sure if we
want this, so if you do, comment on the ticket!

In the future, the plan is to roll out a series of other bandwith
authority changes, and see how they affect things. The idea would be to
make one change at a time, and then observe what it does to the network.

1. bw server on a CDN, with one bandwidth authority
2. run a scanner with and without PreferIPv6 to measure IPv6 exit
performance
3. Try to use the global, dual-stack, HTTP/2 CDN map 
4. Change the default set of servers in the bwauth code to use the CDN
5. Convince all bwauthorities to use multiple redundant servers
6. Run a single-onion bw server
7. Run an ipv6 bw server
8. ?


0. https://trac.torproject.org/projects/tor/wiki/doc/BandwidthAuthorityMeasurements