[tor-relays] Opt-In Trial: Fallback Directory Mirrors

[starlight.2015q3 at binnacle.cx]

Found a serious bug in the

   updateFallbackDirs.py

script used to generate the fallback relay
candidate list.  The OnionOO retrieval used
to pull flag history returns the entire
history of each relay rather than the
120 days requested.

This results in 145 relays left
off the list as too-old history is
averaged into the percentages.

While this bug appears to be in either
OnionOO or in the constructed request
URL (not sure which), I opted to add a
maximum history age to the script.  The
result is 558 relays instead of the
original 413.  (My relay Binnacle now
easily makes the cut with guard flag at
97.4 percent.)

In addition several data validation bugs
were discovered empirically and corrected.

Changes:

* support maximum history age in _avg_generic_history()

* create and apply MAX_AGE_DAYS=120 in calls
  to _avg_generic_history()

* fix division-by-zero trap in _avg_generic_history()

* skip missing (i.e. null/None) intervals in _avg_generic_history()

* Python timedelta.total_seconds() function not
  available in 2.6; replace with equivalent expression

* set DEBUG logging level to make relay
  exclusion reasons visible

* move CUTOFF_GUARD test to end in order to
  expose more exclusion reasons

Attached to this message are

1) revised script
2) script patch

Posted to PasteBin (one month expire):

3) fallback_dirs.inc as-of 2016/01/11
   http://pastebin.com/8cxKCEP6

4) debug output showing causes of relay exclusions
   http://pastebin.com/raw/3SBpgECm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: updateFallbackDirs.py.patch
Type: application/octet-stream
Size: 4208 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20160111/e342cb1a/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: updateFallbackDirs.py
Type: application/octet-stream
Size: 20905 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20160111/e342cb1a/attachment-0003.obj>