[tor-relays] Consensus Weight calculation

[teor]

> On 13 Jun 2017, at 00:58, Vort <vvort at yandex.ru> wrote:
> ...
>> This is a question that gets asked a lot:
>> "Many people set up new fast relays and then wonder why their bandwidth
>> is not fully loaded instantly…"
>> https://blog.torproject.org/blog/lifecycle-of-a-new-relay
> 
> Maybe this is correct in many cases.
> But definitely not in all of them.
> 
> For example, this line:
> "once the bwauths have measured you and the directory authorities lift
>  the 20KB cap, you'll attract more and more traffic"
> Events can go other way: bwauths will assign lower weight, and relay
>  will be getting less and less traffic.

We know of relays that have improved their bandwidth measurements by
changing their keys (this resets the measurements).

But most relays get low weights because:
* they do not get good bandwidth over time,
* they do not get good bandwidth to the rest of the tor network,
* they have high latency to the rest of the tor network,
* they can not get enough CPU or RAM,
* they can not keep enough connections open,
* they go up and down a lot,
* they change IP address a lot, or
* some other reasons that make them less useful to clients.

Here is more information about this:
https://lists.torproject.org/pipermail/tor-relays/2016-November/010928.html

> ...
>> I'm not sure if this is a problem. And I'm not sure how many relays it
>> impacts.
> 
> Hundreds, I guess.
> 
> Here is some examples:
> 
> https://atlas.torproject.org/#details/9FC2673BB2704C2AAB851F8334938565DF1D0819
> Now used bandwidth:     1    KiB/s
> Advertised Bandwidth: 131.38 KiB/s
> Top used bandwidth:   250    KiB/s
> Bandwidth rate:      4000    KiB/s

Here are the measurements from each bandwidth authority (large page):
https://consensus-health.torproject.org/consensus-health-2017-06-12-22-00.html#9FBEB75E8BC142565F12CBBE078D63310236A334

And look at other relays in the same AS:
https://atlas.torproject.org/#search/as:AS8100

But this isn't enough information to work out what the problem is.
Maybe there is a problem with the relay, not the measurements.
We just can't tell.

> ...
> As you can see, most of them can handle a lot more traffic: 50x-4000x.
> Also don't see why they can have high latency.
> Good relays, on my opinion.

Maybe the relay has low CPU, international bandwidth, or connection
limits. We just don't know.

We would need to talk to the operator to find out.

> ...
> I have launched my own instance of BwAuthority and I see, that measured
>  "filt_bw" values are pretty close to "Advertised Bandwidth":
> ...

These measurements are updated over time.
Please check again after a few weeks.

> The problem is on the next step, I think.
> 
>>> The result has revealed some anomalies:
>>> https://s8.hostingkartinok.com/uploads/images/2017/06/fed1cf8b57fc027223c8eaf3deb0d28a.png
>>> First, and most important, - a lot of relays have bandwidth estimate
>>> in range 0-50: 1082 of them.
> 
>> I don't know what each axis is on this graph.
> 
> x is KiB/s, y is count
> (yellow bars are for "Advertised Bandwidth", blue - for
> "Consensus Weight", grey mean both values)
> ...
> 
>>> Second - there are incorrect estimates
>>> for popular bandwidths of 5, 10 and 20 MBits.
> 
>> I don't understand what you mean here. The advertised bandwidth is in
>> kilobytes per second, and the consensus weight is dimensionless (but
>> scaled from kilobytes per second).
> 
>> Can you point out the lines you mean?
> 
> Look at the yellow spike at x = ~1200.
> Low blue bars at the same point means that "Consensus Weight" model
>  did not take into account that there are many 1200 KiB/s nodes on
>  the network, which will result in theirs underload.

The consensus weight model does not fit relays to a curve.
It can have lots of relays at the same bandwidth.

I think this spike means:

"You think your provider is giving you 100 Mbps, but they are
actually giving you much less. Talk to them about it."

Usually this is because the provider only tries to give everyone
100Mbps, or they limit everyone and don't tell them, or they don't
pay enough to get good international bandwidth.

T

--
Tim Wilson-Brown (teor)

teor2345 at gmail dot com
PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n
xmpp: teor at torproject dot org
------------------------------------------------------------------------



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20170613/b5b61215/attachment.sig>