[tor-bugs] #6752 [Tor]: TestingTorNetwork doesn't lower the dir fetch retry schedules
Tor Bug Tracker & Wiki
blackhole at torproject.org
Thu Mar 7 13:40:33 UTC 2013
#6752: TestingTorNetwork doesn't lower the dir fetch retry schedules
--------------------------------------+-------------------------------------
Reporter: arma | Owner:
Type: enhancement | Status: new
Priority: major | Milestone: Tor: 0.2.4.x-final
Component: Tor | Version:
Keywords: tor-client small-feature | Parent: #7172
Points: | Actualpoints:
--------------------------------------+-------------------------------------
Comment(by karsten):
I think I found out what prevented us from making new directory requests
every 60 seconds as defined in the lowered refetch schedule: there's a
hard-coded 5 minute timeout for letting a directory connection stall
before expiring it: `DIR_CONN_MAX_STALL`. We'll probably need to lower
this timeout to, say, 30 seconds in a testing network.
Also, I found a problem from lowered refetch schedules that we need to
fix: there are (at least) four constants defining how many times we try a
descriptor download before giving up:
`CONSENSUS_NETWORKSTATUS_MAX_DL_TRIES`,
`MAX_ROUTERDESC_DOWNLOAD_FAILURES`, `MAX_MICRODESC_DOWNLOAD_FAILURES`,
`MAX_CERT_DL_FAILURES`. They're all set to 8, but we'll probably want to
set them to 80 or 100 in a testing network. If we don't do that,
especially with retries every 60 seconds we may easily run into 7, 8, 9
failed attempts, and then we stop trying. This is particularly bad on
non-clients which aren't triggered by an external application to reset the
download schedule. In my experiments I had some nodes not bootstrap at
all within 1 hour if these constants are set to 8.
If you like these changed constants in a testing network, I'll include
them as five new torrc options in my patch.
Want to see some results? When I use the lowered refetch schedules and
set the constants above as described, I get the following "Boostrapped
100%" lines:
{{{
0:0:11:753547 [thread-0] 0:15:3:173889106 [scallion-message]
[1uthority-73.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:11:763847 [thread-0] 0:15:3:289621246 [scallion-message]
[2uthority-74.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:11:792273 [thread-0] 0:15:3:673929774 [scallion-message]
[3uthority-75.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:12:794207 [thread-0] 0:15:21:967156005 [scallion-message]
[nonexit2-84.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:12:859044 [thread-0] 0:15:22:882704137 [scallion-message]
[exit1-78.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:13:226706 [thread-0] 0:15:28:358539690 [scallion-message]
[webclient1-88.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:14:458496 [thread-0] 0:15:53:001822682 [scallion-message]
[exit5-82.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:14:471866 [thread-0] 0:15:53:334226076 [scallion-message]
[exit4-81.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:14:536720 [thread-0] 0:15:54:845187276 [scallion-message]
[nonexit4-86.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:14:648466 [thread-0] 0:15:55:760034257 [scallion-message]
[nonexit5-87.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:16:346193 [thread-0] 0:16:39:063729918 [scallion-message]
[exit3-80.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:17:050978 [thread-0] 0:16:53:675256680 [scallion-message]
[nonexit3-85.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:17:401589 [thread-0] 0:16:59:462751388 [scallion-message]
[exit2-79.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:17:542252 [thread-0] 0:17:0:740213796 [scallion-message]
[nonexit1-83.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:19:847435 [thread-0] 0:17:38:479260129 [scallion-message]
[4uthority-76.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:23:362068 [thread-0] 0:20:14:179430946 [scallion-message]
[5uthority-77.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
}}}
The second timestamp in these log lines is simulated time
(0:15:3:173889106 in the first line). The very first consensus in this
network is published at 0:15:0. That means that the longest time to
bootstrap is 2:38 minutes. 5uthority starts at time 0:20:2 and takes 12
seconds to bootstrap. This seems very acceptable. For comparison, here's
the situation before changing the five constants above:
{{{
0:0:11:532841 [thread-0] 0:15:3:107393402 [scallion-message]
[1uthority-73.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:11:533525 [thread-0] 0:15:3:127368755 [scallion-message]
[2uthority-74.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:11:675343 [thread-0] 0:15:4:268884973 [scallion-message]
[3uthority-75.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:13:908788 [thread-0] 0:18:28:931568544 [scallion-message]
[nonexit4-86.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:13:929487 [thread-0] 0:18:29:280680071 [scallion-message]
[exit1-78.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:14:414258 [thread-0] 0:18:45:522758087 [scallion-message]
[4uthority-76.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:16:414355 [thread-0] 0:20:13:955995654 [scallion-message]
[5uthority-77.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:16:651176 [thread-0] 0:20:21:901359712 [scallion-message]
[nonexit1-83.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:16:804762 [thread-0] 0:20:24:057107728 [scallion-message]
[nonexit5-87.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:16:870519 [thread-0] 0:20:27:890874732 [scallion-message]
[exit2-79.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:19:363695 [thread-0] 0:23:23:469750053 [scallion-message]
[exit4-81.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:19:920079 [thread-0] 0:24:20:250686967 [scallion-message]
[exit5-82.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:20:673098 [thread-0] 0:24:49:322543288 [scallion-message]
[webclient1-88.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:21:749740 [thread-0] 0:25:38:208126411 [scallion-message]
[nonexit3-85.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
}}}
And finally, here's the situation before applying any patch:
{{{
0:0:11:448903 [thread-0] 0:15:2:582133247 [scallion-message]
[3uthority-75.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:11:526322 [thread-0] 0:15:3:715646237 [scallion-message]
[1uthority-73.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:11:527393 [thread-0] 0:15:3:736179239 [scallion-message]
[2uthority-74.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:13:131711 [thread-0] 0:16:38:776506156 [scallion-message]
[4uthority-76.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:13:502492 [thread-0] 0:17:20:866255937 [scallion-message]
[nonexit2-84.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:13:580818 [thread-0] 0:17:22:338375323 [scallion-message]
[exit1-78.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:13:709414 [thread-0] 0:17:23:833521695 [scallion-message]
[nonexit5-87.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:15:607681 [thread-0] 0:19:12:118365753 [scallion-message]
[exit3-80.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:15:917059 [thread-0] 0:19:19:662494634 [scallion-message]
[exit5-82.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:17:402870 [thread-0] 0:20:14:413988393 [scallion-message]
[5uthority-77.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:17:694102 [thread-0] 0:20:20:697006789 [scallion-message]
[exit4-81.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:19:807030 [thread-0] 0:23:26:615529962 [scallion-message]
[nonexit1-83.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:20:406078 [thread-0] 0:24:24:836237885 [scallion-message]
[exit2-79.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:21:459315 [thread-0] 0:26:10:050116590 [scallion-message]
[webclient1-88.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:25:253972 [thread-0] 0:29:31:480640102 [scallion-message]
[nonexit4-86.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
0:0:35:308164 [thread-0] 0:38:43:258560918 [scallion-message]
[nonexit3-85.1.0.0] [intercept_logv] [notice] Bootstrapped 100%: Done.
}}}
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/6752#comment:35>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list