On Thu, 1 Dec 2011 04:55:06 -0500 Roger Dingledine arma@mit.edu wrote:
On Wed, Nov 30, 2011 at 05:51:13PM -0600, Scott Bennett wrote:
Something new appears to be happening in the authorities' assignmentsof "Bandwidth=" values in the consensus documents over the past few days. Here are some statistics from the recent consensus with the following valid times:
[snip]
The top ten in the above list account for 814 relays or 32.73% of all relays in the consensus, yet they will get little or no traffic because of the assigned "Bandwidth=" values. My relay's traffic went from around 55% - 60% of what I had hoped to contribute down to almost nothing in the course of about three days, and the "Bandwidth=" value assigned to it in the consensus is 1. For a long time, it had typically been in the high 20s to high 30s with occasional excursions from slightly lower to somewhere in the 50s. So is this sudden, massive reassignment of loads due to a bug in the newest tor version(s)? Or is it an intended change by the developers?
Mike Perry is trying out more network rebalancing ideas. See
https://gitweb.torproject.org/torflow.git/blob/HEAD:/NetworkScanners/BwAutho... https://trac.torproject.org/projects/tor/ticket/1976 https://trac.torproject.org/projects/tor/ticket/4425
Thanks for those. I'll probably get a chance to take a look at them later today.
The "bandwidth authority" scripts do active bandwidth measurements, and adjust the bandwidth weights they vote about based on how a relay performs compared to its peers. Last week "peers" were characterized based on the self-advertised bandwidth, that is, the bandwidth in the relay descriptor. With the new PID design, "peers" are now a function of the weight in the consensus.
Okay, but that bring in again the old problem of flailing values reported due to the feedback. If you're going to include those reported values, then tor needs to be fixed first, so that it never reports a lesser value than it has previously reported since initialization except in "unusual" circumstances, where "unusual" might be defined as something like a current value that were less than half of the value in the last descriptor published. IOW, the value should ordinarily approach the true capacity asymptotically over successive descriptor publications. If this fix isn't made, then randomly occurring slow traffic period for a given relay will result in less traffic being sent to it in the future, which would not reflect the actual capacity of the relay. Another problem is that, as I understood from earlier links you provided, Mike's testers were measuring latency, where latency referred not to end-to-end nor to round-trip packet travel times, but rather to circuit construction times. (BTW, I'm interested in the equations/matrices used to invert the data collected for multi-hop circuits to yield values for each individual node. Is there a page on your web servers that shows them?) Circuit construction, of course, requires different resources from those required simply to pass data along already created circuits, so the latency times and the data rate capacities really are not comparable types of measurements. (What is a typical correlation coefficient between them? I would expect some positive correlation, but not necessarily close to 1.)
In theory this feedback loop should drive even more traffic toward relays that can handle it, and even less traffic toward relays that can't, until we reach an equilibrium.
In practice, it drives super-fast relays to a weight of infinity, and all the rest to a weight of zero.
Mike thought he'd gotten things (more) right this time, but it would appear that has hasn't: https://metrics.torproject.org/performance.html#torperf
Sorry for the disturbance. I think we should probably turn the experiment off soon. But the little dip you see in the torperf graph before the spike makes me think there *is* a point here that is better, even if we haven't found it yet. :)
Are you sure that little dip isn't just an artifact of the T-Day holiday weekend in the U.S.? (Cool graph, BTW.:-) Anyway, thanks much for the explanations, Roger. Glad to know that 2.3.8-alpha isn't at fault.
Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at cs.niu.edu * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************