[tor-bugs] #24113 [Core Tor/Tor]: We stop trying to download an md after 8 failed tries

[Tor Bug Tracker & Wiki]

#24113: We stop trying to download an md after 8 failed tries
-----------------------------------------------+---------------------------
 Reporter:  asn                                |          Owner:  (none)
     Type:  defect                             |         Status:  new
 Priority:  Medium                             |      Milestone:  Tor:
                                               |  0.3.2.x-final
Component:  Core Tor/Tor                       |        Version:  Tor:
                                               |  0.3.0.6
 Severity:  Normal                             |     Resolution:
 Keywords:  tor-guard, tor-bridge, tor-client  |  Actual Points:
Parent ID:  #21969                             |         Points:
 Reviewer:                                     |        Sponsor:
-----------------------------------------------+---------------------------

Comment (by teor):

 Replying to [comment:6 asn]:
 > Replying to [comment:5 teor]:
 > > We can only remove the retry limit if we are sure exponential backoff
 works.
 > > (There is still a global retry limit of 255 for every individual
 directory document.)
 > >
 >
 > Hm. Are you saying that there is a chance that exponential backoff
 doesn't work?

 Yes, we have regular bugs in this subsystem.

 And yes, if application activity keeps on resetting the exponential
 backoff on md fetches, we will reach the download limit.

 > > Why not work out the actual number of retries we need, and increase it
 to that?
 > >
 > > (I would guess that we shouldn't retry a single md more than 16 or 20
 times, but that's just a guess.)
 > >
 >
 > Is there actually a number of retries that guarantees to give us mds? I
 don't think so, especially when you consider edge-cases like
 comment:1:ticket:23863.

 Yes. 99.9% will get them after 20 tries, but it will take them days.

 At hh:00, no tries will ever work.
 At hh:01, 30 tries.
 At hh:02, 15 tries.
 At hh:03, 10 tries.
 At hh:06, 5 tries.
 At hh:10, 3 tries.
 At hh:15, 2 tries.
 From hh:30 to hh:59, 1 try.

 If you think about how exponential backoff works, clients will try their 8
 attempts after this many seconds on average (last + 1 + (1 + 4*last))/2:
 0, 1, 3, 8, 21, 53, 133, 333 (8 tries), 833, 2083

 For a client that started at hh:00, this means they try in minute:
 0, 0, 0, 0, 0, 1, 2, 6 (8 tries), 14, 35
 This gives them about a 30% chance of fetching a new md after 8 tries and
 6 minutes in the worst case scenario. Or a 100% chance of fetching a new
 md after 10 tries and 35 minutes in the worst case scenario.

 So here are your options:

 If you want all clients to get guard mds after 30 seconds, you should make
 them try an authority on the 4th try. (Remember, the 5th try is an average
 of 21 seconds.)

 If you want all clients to get all their mds after 1 minute, you should
 make them try an authority on the 5th try.

 If want all clients to get all their mds after 30 minutes, you should make
 them try 8 or 9 times, and then make the remainder try an authority.
 (Which is a strange coincidence, because I bet someone guessed the default
 of 8 tries when they were designing the old fixed-delay schedules.)

 (Not trying an authority is either slow, or it is unreliable, or both.)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/24113#comment:7>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online