[metrics-bugs] #20723 [Metrics/CollecTor]: Fix torperf file download from moria

[Tor Bug Tracker & Wiki]

#20723: Fix torperf file download from moria
-----------------------------------+--------------------------
     Reporter:  karsten            |      Owner:  metrics-team
         Type:  defect             |     Status:  new
     Priority:  High               |  Milestone:
    Component:  Metrics/CollecTor  |    Version:
     Severity:  Normal             |   Keywords:
Actual Points:                     |  Parent ID:
       Points:                     |   Reviewer:
      Sponsor:                     |
-----------------------------------+--------------------------
 A few days ago I started finding logs like the following written by
 CollecTor's torperf module:

 {{{
 2016-11-16 06:01:00,002 INFO o.t.c.c.CollecTorMain:66 Starting torperf
 module of CollecTor.
 2016-11-16 06:04:08,431 WARN o.t.c.t.TorperfDownloader:260 The last
 timestamp line in '/srv/collector.torproject.org/collector/out/torperf
 /moria-50kb.data' is not contained in the new
 file downloaded from 'http://www.freehaven.net/~karsten/perf/50kb.data'.
 Cannot append new lines without possibly leaving a gap.  Skipping.
 }}}

 I figured it might be related to that server automatically redirecting
 http to https and CollecTor not following that redirect, so I changed to
 the https URL in `collector.properties`.

 Now I receive the following log statements:

 {{{
 2016-11-17 18:03:03,215 WARN o.t.c.t.TorperfDownloader:267 Failed
 downloading and/or merging
 'https://www.freehaven.net/~karsten/perf/50kb.data'.
 javax.net.ssl.SSLHandshakeException: Remote host closed connection during
 handshake
         at
 sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:963)
 ~[na:1.7.0_111]
         at
 sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1342)
 ~[na:1.7.0_111]
         at
 sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1369)
 ~[na:1.7.0_111]
         at
 sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1353)
 ~[na:1.7.0_111]
         at
 sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
 ~[na:1.7.0_111]
         at
 sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
 ~[na:1.7.0_111]
         at
 sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
 ~[na:1.7.0_111]
         at
 org.torproject.collector.torperf.TorperfDownloader.downloadAndAppendFile(TorperfDownloader.java:238)
 [collector-1.1.0.jar:1.1.0-d9e32d5]
         at
 org.torproject.collector.torperf.TorperfDownloader.downloadAndMergeFiles(TorperfDownloader.java:171)
 [collector-1.1.0.jar:1.1.0-d9e32d5]
         at
 org.torproject.collector.torperf.TorperfDownloader.startProcessing(TorperfDownloader.java:80)
 [collector-1.1.0.jar:1.1.0-d9e32d5]
         at
 org.torproject.collector.cron.CollecTorMain.run(CollecTorMain.java:67)
 [collector-1.1.0.jar:1.1.0-d9e32d5]
         at
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 [na:1.7.0_111]
         at
 java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
 [na:1.7.0_111]
         at
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
 [na:1.7.0_111]
         at
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 [na:1.7.0_111]
         at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 [na:1.7.0_111]
         at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 [na:1.7.0_111]
         at java.lang.Thread.run(Thread.java:745) [na:1.7.0_111]
 Caused by: java.io.EOFException: SSL peer shut down incorrectly
         at sun.security.ssl.InputRecord.read(InputRecord.java:482)
 ~[na:1.7.0_111]
         at
 sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:944)
 ~[na:1.7.0_111]
         ... 17 common frames omitted
 }}}

 It seems that `wget` works just fine on that host to download that file.

 Any idea what I could try?

 I'll have to fix this in the next couple of days, or moria will delete
 older data and CollecTor won't be able to archive that.  Therefore setting
 priority to high.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20723>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online