[tor-bugs] #9765 [TorDNSEL/TorBEL]: TorDNSel exit lists are missing expected data

[Tor Bug Tracker & Wiki]

#9765: TorDNSel exit lists are missing expected data
---------------------------------+----------------------
     Reporter:  Ry               |      Owner:
         Type:  defect           |     Status:  assigned
     Priority:  normal           |  Milestone:
    Component:  TorDNSEL/TorBEL  |    Version:
   Resolution:                   |   Keywords:  TorCheck
Actual Points:                   |  Parent ID:
       Points:                   |
---------------------------------+----------------------

Comment (by karsten):

 Replying to [comment:7 arlolra]:
 > The data used in the above investigation was from the metrics project. I
 noticed the cron that collects the data runs at around 2 minutes past the
 hour but TorDNSEL is busy collecting new data for what looks like 20
 minutes. Is it getting everything in exit-addresses.new? Should probably
 rerun the tests with a file straight from TorDNSEL in production to
 confirm the above.

 That's correct, the cronjob runs at 2 minutes past the hour.  If there's a
 better time to fetch the time, I can change that to any minute past the
 hour.  I ran a quick experiment where I downloaded the file every minute
 today between 11:04 and 12:09 UTC.  Here are the file sizes and last-
 modified times:

 {{{
 116408 11:04
 116408 11:05
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116408 11:06
 116721 11:29
 116721 11:30
 116721 11:31
 117034 11:32
 117034 11:33
 117034 11:34
 117034 11:35
 117191 11:36
 117191 11:37
 117350 11:38
 117350 11:39
 117351 11:40
 117509 11:41
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117509 11:42
 117666 12:02
 117822 12:03
 117822 12:04
 117822 12:05
 117822 12:06
 117822 12:06
 117822 12:06
 117822 12:06
 }}}

 It seems that the :06 and :42 files are left unchanged for long enough to
 fetch them.  So, should I change the cronjob to either 25 or 55 minutes
 past the hour?

 Or should I learn more about the `exit-addresses.new` file and maybe fetch
 that one, too?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9765#comment:8>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online