[tor-bugs] #27338 [Core Tor/sbws]: How long should sbws keep measured and observed bandwidths?

[Tor Bug Tracker & Wiki]

#27338: How long should sbws keep measured and observed bandwidths?
---------------------------+-------------------------------------
 Reporter:  teor           |          Owner:  (none)
     Type:  task           |         Status:  new
 Priority:  Medium         |      Milestone:  sbws 1.0 (MVP must)
Component:  Core Tor/sbws  |        Version:
 Severity:  Normal         |     Resolution:
 Keywords:                 |  Actual Points:
Parent ID:  #27108         |         Points:
 Reviewer:                 |        Sponsor:
---------------------------+-------------------------------------

Comment (by teor):

 Replying to [comment:5 juga]:
 > Replying to [comment:4 teor]:
 >
 > > The minimum viable product must:
 > > * Use the latest descriptor bandwidth limit, because:
 > >   - the latest descriptor contains the limit that the operator has
 asked for
 > > * Use at least 2 sbws measured bandwidths over at least 2 days,
 because:
 > >   - tor relay usage varies on a daily cycle
 > >   - each sbws measurement depends on the time of day
 > > * Use at least 2 descriptor observed bandwidths over at least 2 days,
 because:
 > >   - a single download by a single client can increase the observed
 bandwidth
 > >   - for security, we want results that don't depend on a single
 client's behaviour
 >
 > Currently, it's possible that after 2 (or more) days we didn't collected
 less than 2 measurements and descriptor observed bandwidths for some
 relays.
 > It's possible that prioritization might need some changes, which i think
 might be related to https://github.com/pastly/simple-bw-
 scanner/issues/136.
 > If prioritization can't change that, it might be the case that is not
 possible to obtain 2 measurements in the last 2 days.

 Ok, I think those rules are confusing.

 Let's try to split them up:

 If any of these things are true, do not put the relay in the bandwidth
 file:
 * there are less than 2 sbws measured bandwidths
 * all the sbws measured bandwidths are within 24 hours of each other
 * there are less than 2 descriptor observed bandwidths
 * all the descriptor observed bandwidths are within 24 hours of each other

 We will need to make these settings configurable, so we can get test
 network results in less than 1 day.

 > > * Don't keep bandwidths for more than 1 week
 > >   - old bandwidths do not help us work out current relay capacity
 >
 > For bandwidth files, that's the default. Raw measurements are keep 90
 days by default

 Sorry, I meant:
 * Don't use sbws measured bandwidths that are older than 1 week
 * Don't use descriptor observed bandwidths that are older than 1 week

 > > If you want, I can write or review patches, or write a detailed spec.
 >
 > I've already the code except for the comments above (need to clean a bit
 commits). Reviews and spec would help.

 Ok, when you finish a ticket, let me know, and I will do the review and
 spec.

 > > The descriptor has:
 > > {{{
 > > "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed NL
 > > }}}
 > >
 > > Torflow does:
 > > {{{
 > > bw_observed = min(bandwidth-avg, bandwidth-burst, bandwidth-observed)
 > > }}}
 > > https://gitweb.torproject.org/pytorctl.git/tree/TorCtl.py#n459
 > >
 > > But that's redundant, because tor relays do:
 > > {{{
 > > "bandwidth" min(RelayBandwidthRate, RelayBandwidthBust, BandwidthRate,
 BandwidthBurst, MaxAdvertisedBandwidth) min(RelayBandwidthBust,
 BandwidthBurst) bandwidth-observed NL
 > > }}}
 > > See get_effective_bwrate() and get_effective_bwburst().
 >
 > i've been collecting and documenting all these possible values and the
 different names they could have so i don't get confused.
 > I've just not put that notes online somewhere yet but intend to do so.

 Thanks!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27338#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online