[tor-bugs] #20910 [Core Tor/Fallback Scripts]: Make the fallback list automatically adapt to minor changes

Tor Bug Tracker & Wiki blackhole at torproject.org
Wed Dec 20 01:14:41 UTC 2017


#20910: Make the fallback list automatically adapt to minor changes
---------------------------------------+----------------------------------
 Reporter:  teor                       |          Owner:  (none)
     Type:  enhancement                |         Status:  new
 Priority:  Medium                     |      Milestone:  Tor: unspecified
Component:  Core Tor/Fallback Scripts  |        Version:
 Severity:  Normal                     |     Resolution:
 Keywords:  fallback                   |  Actual Points:
Parent ID:                             |         Points:  1
 Reviewer:                             |        Sponsor:
---------------------------------------+----------------------------------

Comment (by teor):

 This is the original wiki text:

 == 1. Checking Existing Fallbacks ==

 Fallbacks can go down, become slow, or change their identity key
 (fingerprint), addresses, or ports.
 We:
 a. check if any existing fallbacks have changed,
 b. inform relay operators,
 c. modify the hard-coded list, and
 d. modify the whitelist and blacklist.

 === a. Finding Broken Fallbacks ===

 To check existing fallbacks, we use the existing fallback list as the
 whitelist.

 i. Run the fallback script in check_existing mode, saving the list:
 {{{
 scripts/maint/updateFallbackDirs.py check_existing > good_fallbacks 2>
 good_fallbacks.log
 }}}
 This can take a long time, as it downloads ~150MB of OnionOO data, parses
 it, then tries each fallback's DirPort.

 === b. Handling Broken Fallbacks ===

 To produce a log that only has the whitelisted fallbacks in it, do:

 {{{
 for fp in `cat scripts/maint/fallback.whitelist | grep id= | cut -d" " -f3
 | cut -d= -f2`; do
   grep $fp good_fallbacks.log >> good_fallbacks.whitelist.log;
 done
 }}}

 The script will log warnings like:
 {{{
 WARNING::0C2C599AFCB26F5CFC2C7592435924C1D63D9484 excluded: has it gained
 an IPv6 address [2001:41d0:a:fb7a::1]:9001?
 WARNING::FCB6695F8F2DC240E974510A4B3A0F2B12AB5B64 excluded: has it changed
 IPv4 from 195.154.8.111 to 178.62.12.24?
 WARNING::Consensus download: 20.5s too slow from IPredator
 (197.231.221.211:9030), max download time 15.0s.
 }}}

 It will also produce info-level messages about other excluded fallbacks
 (see "Handling Missing Fallbacks"):
 {{{
 INFO::01A9258A46E97FF8B2CAC7910577862C14F2C524 not a candidate: running
 avg too low (0.116029)
 INFO::1ECD73B936CB6E6B3CD647CC204F108D9DF2C9F7 not a candidate: guard avg
 too low (0.000000)
 INFO::6A640018EABF3DA9BAD9321AA37C2C87BBE1F907 not a candidate: version
 not recommended
 }}}

 If relays are down, stem will produce log messages like this:
 {{{
 INFO::Initiating consensus download from armbrust (176.31.180.157:143).
 INFO::Unable to retrieve a consensus from armbrust: <urlopen error [Errno
 61] Connection refused>
 }}}

 Other relays may have been missing for so long, they aren't even in the
 OnionOO data.

 For each warning and each excluded relay:
 i. email the operator (see "How to Contact Relay Operators"), and ask them
 if the change is permanent,
   * if there's nothing an operator can do to fix the issue (for example
 guard average), there's no reason to email them,
 ii. remove the fallback from the hard-coded list, by commenting it out,
   * you must modify the start of each line, so that stem's automated
 parser detects the fallback change,
   * backport the modified hard-coded list to every Tor version >= 0.2.8
 that might ever be re-released,
 iii. move the relay from the fallback.whitelist to the fallback.blacklist,
   * if the script will pick up improvements automatically (uptime, guard
 average, version), there's no reason to blacklist the relay.

 If the operator has fixed the issue, leave the fallback in the hard-coded
 list and the whitelist. But if they have a history of unexpected changes,
 it is safer to remove the relay from the hard-coded list, and blacklist
 it.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20910#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list