[tor-bugs] #19018 [Metrics/CollecTor]: run CollecTor modules without crontab

Tor Bug Tracker & Wiki blackhole at torproject.org
Wed May 11 10:20:44 UTC 2016

#19018: run CollecTor modules without crontab
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  new
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:

Comment (by karsten):

 I tried to apply the suggested configuration scheme to match what we're
 currently doing with cron.  But the simplest way I could come up with that
 does what we need requires two values per module:
  - `offsetMinutes`: offset in minutes since the epoch and
  - `periodMinutes`: period in minutes.

 Using these two values we can compute `initialDelay` and `period` for
 `ScheduledExecutorService.scheduleAtFixedRate()` as follows (mostly

 long ONE_MINUTE = 60L * 1000L;
 long initialDelay = periodMinutes * ONE_MINUTE
     - (System.currentTimeMillis() - offsetMinutes * ONE_MINUTE)
     % (periodMinutes * ONE_MINUTE);
 long period = periodMinutes * ONE_MINUTE;

 Here's how I would configure runs on collector.torproject.org using that

 bridgedescs 9,60
 bridgepools -1,-1
 exitlists 2,60
 relaydescs 5,30
 torperf 1,360
 updateindex 0,2

  - Bridge descriptors are copied over from the bridge authority at :07
 every hour, so we should start the sanitizing soon after at :09 every hour
 and be ready when Onionoo and others fetch sanitized bridge descriptors at
  - Bridge pool assignments are not sanitized anymore, so I picked -1,-1 as
 special value to disable the module, similar to your suggestion.  (Note
 that we could also remove the entire code for sanitizing bridge pool
 assignments.  If you currently don't have any branches based on master and
 if you'd like me to remove that code, I would go ahead and do that today.
 But regardless, we would need a way to disable a module, and -1,-1 should
 work okay for that.)
  - Exit lists should be fetched once per hour at :02.
  - Relay descs can be fetched at :05, but we should try hard not to miss
 an hour there.  If the server happens to be down at :05 we should try
 again half an hour later at :35.  We're currently not doing that, but I
 think it would be smart to do it.
  - Torperf results are only fetched every six hours to reduce load on the
 Torperf hosts.
  - Updating the index should happen every two minutes.

 How does that sound?

Ticket URL: <https://trac.torproject.org/projects/tor/ticket/19018#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online

More information about the tor-bugs mailing list