[tor-relays] current HSDir flag requirements

[Scott Bennett]

I <bennett at sdf.org> wrote:

> Logforme <m7527 at abc.se> wrote:
>
> >
> > On 2021-05-26 08:18:32, "Scott Bennett" <bennett at sdf.org> wrote:
> > >I interpret that as meaning that one
> > >or more criteria being used by one or more authorities has changed,
> > What I have noticed on my relay is that the "Consensus Weight" is 
> > fluctuating.
> > CW is too complicated for my tiny brain but I believe the measurements 
> > from the Bandwidth Authorities is involved. The BWAuths are spread 
> > around the world and depending on current internet conditions they get 
> > different speed values to your relay. But can it cause massive swings in 
> > CW?
>
>      Yes.  My relay is on a residential service connection and its "bandwidth"
> rating from the Authority relays typically oscillates between ~30 and ~120, so
> proportionally the swings are often fairly wild.

     A new, more extreme problem has emerged in the last two or three days.  My
relay's "Bandwidth" in the consensus documents dropped suddenly to 14, then 4 for
about 18 hours, and finally to 3, where it has remained ever since.  This is still
the first time I can remember it showing up in the consensus with a value lower
than "Bandwidth=20 Unmeasured=1".
     On Wednesday I took a quick look at the then current consensus file and found
the following, regarding single-digit "Bandwidth" values.


Script started on Wed Jun  2 10:36:48 2021
hellas#	grep '^w Bandwidth=' /var/db/tor/cached-consensus | wc -l
    6777
hellas#	grep '^w Bandwidth=.$' /var/db/tor/cached-consensus | wc -l
     610
hellas#	grep '^w Bandwidth=1$' /var/db/tor/cached-consensus | wc -l
     359
hellas#	^1^2
grep '^w Bandwidth=2$' /var/db/tor/cached-consensus | wc -l
      53
hellas#	^2^3
grep '^w Bandwidth=3$' /var/db/tor/cached-consensus | wc -l
      32
hellas#	^3^4
grep '^w Bandwidth=4$' /var/db/tor/cached-consensus | wc -l
      25
hellas#	^4^5
grep '^w Bandwidth=5$' /var/db/tor/cached-consensus | wc -l
      31
hellas#	^5^6
grep '^w Bandwidth=6$' /var/db/tor/cached-consensus | wc -l
      29
hellas#	^6^7
grep '^w Bandwidth=7$' /var/db/tor/cached-consensus | wc -l
      27
hellas#	^7^8
grep '^w Bandwidth=8$' /var/db/tor/cached-consensus | wc -l
      32
hellas#	^8^9
grep '^w Bandwidth=9$' /var/db/tor/cached-consensus | wc -l
      22
hellas#	exit
exit

Script done on Wed Jun  2 10:39:46 2021

     As you can see, 9% of relays in that consensus had single-digit "Bandwidth"
values assigned to them by the Authority relays, and the sizable majority (almost
59%) of those were dumped into the lowest bin ("Bandwidth=1").
     I frequently observe my relay chugging along at 330 KB/s to 355 KB/s and
occasionally at more than 400 KB/s.  It appears that the Authority relays no
longer consider that worthwhile, so is there any reason that I should continue to
run tor as a relay?  Or should I reconfigure it to run as a client only and stop
needing to pay atention to it?  I started running it around sixteen years ago, but
if it's no longer going to be used, maybe I should shut it down instead of letting
it just add clutter to the consensus..
>
> > Maybe the BWAuths have changed their measurement technique during the 
> > last couple of months?
>
>      Well, I first noticed it late last year, IIRC.  The measurement technique
> will, of course, often give deceptive results.  For example, if the connection
> supports ~350 KB/s and the relay has little traffic at the time measurement
> begins, the result should be fairly close to the true value.  OTOH, if the
> relay is handling 200 KB/s of traffic for other circuits at the time measurement
> begins, then the result should be at most only ~150 KB/s, which is far from the
> true value.
> >
> > >  A further question I would like to raise is why do the Authority relays
> > >use different criteria from one to another for the automatic assignment of
> > >flags?  Should they not be consistent in using the same rules?
> > >
> > I agree that it is confusing that 2 auths don't assign the HSDir flag 
> > according to the spec.
> > I have no explanation apart from that AFAIK moria1 is run by Roger 
> > Dingledine and I guess he runs a lot of beta and test stuff.
> > moria1 publishes 2 HSDir "Flag Threshold" values (hsdir-wfu and 
>
>      Yeah, I saw that, but don't know quite what to make of it.
>
> > hsdir-tk) that no other auth publishes which leads me to believe moria1 
> > runs another version of the auth software that handles the HSDir flag 
> > differently. That don't explain bastet though.
>
>      And it only accounts for two Authority relays, whereas you said five are
> refusing to assign HSDir to my relay, which, as you pointed out, may depend
> upon network conditions between those Authority relays and my relay at the time
> and have nothing at all to do with my relay or how much traffic my relay could
> handle or might actually be handling at the time.
> >
> > It's fun to speculate :)
> >
>      I would rather not be kept in the dark.  It should not be like trying to
> get information on what the criminals who rule over us are up to.
>      The problems outlined above would be mitigated somewhat if the measurements
> were filtered somehow, which could be as simple a filter as a boxcar moving
> average.  Yes, I know that for many purposes a rectangular window gives lousy
> results, but for the purpose of understanding relays' capacities over time as
> having values that usually change slowly if at all a boxcar moving average should
> be plenty good enough.  An exponential moving average would probably also be
> fine.  The point of using a filter for the measurements would be to minimize the
> temporary interference of transient network conditions affecting the measurement
> process and corrupting some of the results at some times but not at others.  Of
> course, measurements outside some number of standard deviations from the mean
> for a relay could be discarded, as well.  At present, it is difficult to separate
> the deficiencies of the measurement method from the network realities in trying
> to interpret the measurements.


                                  Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet:   bennett at sdf.org   *xor*   bennett at freeshell.org  *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."                                               *
*    -- Gov. John Hancock, New York Journal, 28 January 1790         *
**********************************************************************