[metrics-bugs] #27055 [Metrics/CollecTor]: Find out why syncing descriptors from collector2.tp.o did not time out

[Tor Bug Tracker & Wiki]

#27055: Find out why syncing descriptors from collector2.tp.o did not time out
-----------------------------------+--------------------------
     Reporter:  karsten            |      Owner:  metrics-team
         Type:  defect             |     Status:  new
     Priority:  Medium             |  Milestone:
    Component:  Metrics/CollecTor  |    Version:
     Severity:  Normal             |   Keywords:
Actual Points:                     |  Parent ID:
       Points:                     |   Reviewer:
      Sponsor:                     |
-----------------------------------+--------------------------
 On August 4, collector.tp.o attempted to fetch remote files from
 collector2.tp.o during its normal sync run. However, this process stalled,
 because of an unknown problem on collector2.tp.o. The effect was that
 neither of the two downloaded consensuses and we missed almost an entire
 day of consensuses. (We got them back from friendly relay operators in the
 end, but that should not become the new way of feeding the archive with
 data.)

 From the log:

 {{{
 2018-08-04 17:37:17,556 INFO o.t.m.c.c.CollecTorMain:75 Starting sync-run
 of module relaydescs of CollecTor.
 2018-08-04 17:37:17,556 INFO o.t.m.c.s.SyncManager:57 Collecting Relay
 from collector2.torproject.org ...
 2018-08-04 17:37:17,556 INFO o.t.d.i.DescriptorIndexCollector:46 Starting
 descriptor collection.
 2018-08-04 17:37:17,556 INFO o.t.d.i.DescriptorIndexCollector:62 Indexing
 local directory /srv/collector.torproject.org/collector/sync/Relay-
 collector2.torproject.org.
 2018-08-04 17:37:17,562 INFO o.t.d.i.DescriptorIndexCollector:73 Fetching
 remote index file https://collector2.torproject.org/index/index.json.
 2018-08-04 17:37:18,053 INFO o.t.d.i.DescriptorIndexCollector:82 Fetching
 remote files from https://collector2.torproject.org.
 [...]
 2018-08-05 18:27:13,331 INFO o.t.d.i.DescriptorIndexCollector:88 Deleting
 extraneous files from local directory sync/Relay-
 collector2.torproject.org.
 2018-08-05 18:27:13,375 INFO o.t.d.i.DescriptorIndexCollector:93 Finished
 descriptor collection.
 }}}

 I'm not sure what to do here. The log doesn't reveal anything obviously
 going wrong. I didn't look at the code yet, so maybe there's an obvious
 way to prevent such issues in the future. Hmm.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27055>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online