[tor-scaling] High level scaling goals (and metrics categories)

[Mike Perry]

teor:
> 
> On 9 Jun 2019, at 09:30, Mike Perry <mikeperry at torproject.org 
> <mailto:mikeperry at torproject.org>> wrote:
> 
>>> ==== Network capacity
>>>     (or, how many more clients can this network fit?)
>>>
>>>     Metrics: Per-Flag Spare Network Capacity, Per-Relay Spare Network Capacity
>>
>> Yep.
>>
>> Aside: A related "capacity balance" (or "throughput variance") metric is
>> "Per-relay spare stream capacity". "Per-relay spare stream capacity" is
>> what Torflow and sbws bandwidth authorities use as their load balancing
>> target. We can derive the bwauth measurements of this value from the
>> consensus, by dividing a relay's consensus "measured" value by its
>> descriptor "observed bandwdith" value. We can also take it directly from
>> sbws or torflow bandwidth files. A plot of these values will show us how
>> individual relays vary in their ability to carry an additional stream.
>
> How does torflow's scaling affect this calculation?
> (sbws implements the same scaling algorithm as torflow.)

Scaling changes in sbws will change this calculation, but since our
first task is to ensure smooth transition to sbws, and since sbws and
Torflow use the same scaling right now, I think we should use the
consensus+votes, and take advantage of our historical record as well.

(Interestingly, changing the value of KISTSchedRunInterval on sbws and
Torflow scanner clients will also have an effect on load balancing
numbers as well, since that parameter effectively caps the bwscanner
throughput. I bet after this change we suddenly see fast relays getting
measured as much faster; we may already be seeing discrepancies on the
high end if some Torflow instances are still on 0.2.9, but sbws is all
KIST :/).

> If we need the raw bandwidth values, sbws lists them as bw_mean and
> bw_median:
>
https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt#n757

I prefer to use the consensus and votes. Those values are easiest to
work with using existing metrics tooling, and we have a complete set of
10+ years worth of historical data for them.

If we ignore "filtering" (just to simplify discussion), then the
consensus "measured" value for a relay from either Torflow and sbws is:

  relay_measured_bw = relay_observed_bw * relay_stream_bw /
                              network_avg_stream_bw

Since, relay_measured_bw comes from the consensus, and relay_observed_bw
comes from the relay's descriptor, we can take the consensus, and/or the
individual authority votes, and compute:

  relay_balance_ratio = relay_measured_bw / relay_observed_bw

Then, from algebra, the result is a number for each relay that is:

  relay_balance_ratio = relay_stream_bw / network_avg_stream_bw

This number tells us how much faster or slower each relay is than the
network average stream capacity (in multiples of whatever the current
average stream capacity is).

In a balanced network (balanced in terms of "spare stream capacity"),
every relay would have 1.0 for their relay_balance_ratio. However,
because we do not perform PID feedback, we don't actually converge on
this balancing criterion.

Therefore, when graphed as a CDF, we should see most of the density (aka
the "cliff" of the CDF) centered around 1.0, but we also have lots of
slow overloaded relays, so we'll see bumps below 1.0, and for our very
fast relays that are always under-utilized, we'll see bumps well above 1.0.

Does all of this make sense? Should we include this metric for
sbws+torflow experiments? I kinda think so; it doesn't require us to
record anything extra since it can be derived from the consensus for all
time (it's also very interesting to examine this metrics for historical
days the bwauths have completely failed, btw).

> I'm not sure if torflow's pid_bw is useful:
> https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/README.spec.txt#n447

Yeah, I would not go down that rabbit hole now. Once we move to all
sbws, we can examine alternate scaling mechanisms using just sbws values
and sbws tooling.

(So far best scaling change idea I have heard is to deliberately shift
consensus weight away from slow relays, so that they are less impacted
by short-term bursts of load. But, implementing memory+CPU+socket
overload feedback and PID control will have a similar shifting effect, I
believe.)

-- 
Mike Perry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-scaling/attachments/20190610/970e384a/attachment.sig>