[tor-relays] consensus "Bandwidth=" values

[Scott Bennett]

     On Thu, 1 Dec 2011 04:55:06 -0500 Roger Dingledine <arma at mit.edu> wrote:
>On Wed, Nov 30, 2011 at 05:51:13PM -0600, Scott Bennett wrote:
>>      Something new appears to be happening in the authorities' assignments
>> of "Bandwidth=" values in the consensus documents over the past few days.
>> Here are some statistics from the recent consensus with the following valid
>> times:
>[snip]
>> The top ten in the above list account for 814 relays or 32.73% of all relays
>> in the consensus, yet they will get little or no traffic because of the
>> assigned "Bandwidth=" values.  My relay's traffic went from around 55% - 60%
>> of what I had hoped to contribute down to almost nothing in the course of
>> about three days, and the "Bandwidth=" value assigned to it in the consensus
>> is 1.  For a long time, it had typically been in the high 20s to high 30s
>> with occasional excursions from slightly lower to somewhere in the 50s.
>>      So is this sudden, massive reassignment of loads due to a bug in the
>> newest tor version(s)?  Or is it an intended change by the developers?
>
>Mike Perry is trying out more network rebalancing ideas. See
>
>https://gitweb.torproject.org/torflow.git/blob/HEAD:/NetworkScanners/BwAuthority/README.spec.txt#l354
>https://trac.torproject.org/projects/tor/ticket/1976
>https://trac.torproject.org/projects/tor/ticket/4425

     Thanks for those.  I'll probably get a chance to take a look at them
later today.
>
>The "bandwidth authority" scripts do active bandwidth measurements,
>and adjust the bandwidth weights they vote about based on how a relay
>performs compared to its peers. Last week "peers" were characterized
>based on the self-advertised bandwidth, that is, the bandwidth in the
>relay descriptor. With the new PID design, "peers" are now a function
>of the weight in the consensus.

     Okay, but that bring in again the old problem of flailing values
reported due to the feedback.  If you're going to include those reported
values, then tor needs to be fixed first, so that it never reports a
lesser value than it has previously reported since initialization except
in "unusual" circumstances, where "unusual" might be defined as something
like a current value that were less than half of the value in the last
descriptor published.  IOW, the value should ordinarily approach the true
capacity asymptotically over successive descriptor publications.  If this
fix isn't made, then randomly occurring slow traffic period for a given
relay will result in less traffic being sent to it in the future, which
would not reflect the actual capacity of the relay.
     Another problem is that, as I understood from earlier links you
provided, Mike's testers were measuring latency, where latency referred
not to end-to-end nor to round-trip packet travel times, but rather to
circuit construction times.  (BTW, I'm interested in the equations/matrices
used to invert the data collected for multi-hop circuits to yield values
for each individual node.  Is there a page on your web servers that shows
them?)  Circuit construction, of course, requires different resources from
those required simply to pass data along already created circuits, so the
latency times and the data rate capacities really are not comparable types
of measurements.  (What is a typical correlation coefficient between them?
I would expect some positive correlation, but not necessarily close to 1.)
>
>In theory this feedback loop should drive even more traffic toward relays
>that can handle it, and even less traffic toward relays that can't,
>until we reach an equilibrium.
>
>In practice, it drives super-fast relays to a weight of infinity, and
>all the rest to a weight of zero.
>
>Mike thought he'd gotten things (more) right this time, but it would
>appear that has hasn't:
>https://metrics.torproject.org/performance.html#torperf
>
>Sorry for the disturbance. I think we should probably turn the experiment
>off soon. But the little dip you see in the torperf graph before the
>spike makes me think there *is* a point here that is better, even if we
>haven't found it yet. :)
>
     Are you sure that little dip isn't just an artifact of the T-Day
holiday weekend in the U.S.?  (Cool graph, BTW.:-)
     Anyway, thanks much for the explanations, Roger.  Glad to know that
2.3.8-alpha isn't at fault.


                                  Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet:       bennett at cs.niu.edu                              *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."                                               *
*    -- Gov. John Hancock, New York Journal, 28 January 1790         *
**********************************************************************