[tor-bugs] #22453 [Core Tor/Tor]: Relays should regularly do a larger bandwidth self-test

[Tor Bug Tracker & Wiki]

#22453: Relays should regularly do a larger bandwidth self-test
-------------------------------------------------+-------------------------
 Reporter:  arma                                 |          Owner:  juga
     Type:  defect                               |         Status:
                                                 |  needs_information
 Priority:  Medium                               |      Milestone:  Tor:
                                                 |  0.4.0.x-final
Component:  Core Tor/Tor                         |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  034-triage-20180328,                 |  Actual Points:
  034-removed-20180328, tor-bwauth,              |
  035-backport, 034-backport-maybe, 033          |
  -backport-maybe, 029-backport-maybe-not        |
Parent ID:  #25925                               |         Points:
 Reviewer:  teor                                 |        Sponsor:
-------------------------------------------------+-------------------------
Changes (by teor):

 * status:  needs_revision => needs_information


Comment:

 Replying to [comment:32 juga]:
 > Currently sbws prioritizes measuring relays for which it doesn't have
 results, it probably should then prioritize the relays with less uptime
 and not being measured according to the consensus.
 > I can try to figure out how long it takes currently to sbws to measure
 those relays.
 > If any of this sounds like a good idea, then i'll create tickets for
 sbws.

 Yes, sbws should prioritise relays:
 * with no sbws measurements, and
 * with no measured bandwidth in the consensus

 I am not so sure about uptime.

 Replying to [comment:31 arma]:
 > Replying to [comment:28 teor]:
 > > > I don't think sbws suffers from similar issues, so once all the
 torflow scanners are gone, we can disable the bandwidth self-test code.
 Until then, it should be on and larger.
 >
 > I don't think we need to wait for all of the torflows to be gone. Isn't
 it enough to have one sbws scanner? Since that scanner would generate
 appropriate load on relays? And if we're worried that one isn't a stable
 situation, we should work on getting a second one, rather than messing
 with this design?

 == Typical sbws measurement times

 Let's try to define "appropriate load" more precisely.

 When sbws measures a relay, it starts by pairing it with a randomly chosen
 relay. Then it takes the median of all the measurements. So, on average,
 sbws starts new fast relays with the median relay bandwidth. Once a relay
 has a bandwidth measurement, sbws tries to pair it with relays that have
 twice its bandwidth. So sbws can achieve 2x exponential growth every 5
 days, from a base of the median relay bandwidth.

 The median relay bandwidth is 20 megabits a second:
 https://metrics.torproject.org/advbwdist-
 relay.html?start=2018-08-21&end=2018-11-19&n=3000

 The fastest relay bandwidth is 1500 megabits a second:
 https://metrics.torproject.org/advbwdist-
 relay.html?start=2018-08-21&end=2018-11-19&n=1

 Therefore, it would take log,,2,,(1500 / 20)*5 = 31 days for sbws to get
 an accurate bandwidth for the fastest relay, if there was no client
 traffic.

 More realistically, the top 10% of relays are at 125 megabits per second:
 https://metrics.torproject.org/advbwdist-
 relay.html?start=2018-08-21&end=2018-11-19&n=500

 Therefore, it would take log,,2,,(125 / 20)*5 = 13 days for sbws to get an
 accurate bandwidth for most (90%) of relays, if there was no client
 traffic.

 Do you think that's ok?

 == Asking other developers

 I also think we need to open this discussion up to a wider audience. I'll
 ask for opinions in today's network team meeting. If there's no consensus,
 let's use the tor proposals process to decide.

 I wrote a tor-dev email with the questions I think we need to answer:
 https://lists.torproject.org/pipermail/tor-dev/2018-November/013546.html

 If we decide to remove bandwidth self-tests, then we can close this
 ticket, and every child ticket except #28511. (Failed ORPort reachability
 tests open 1200 circuits, and failed DirPort tests open 240 circuits:
 that's too many.)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22453#comment:33>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online