[tor-bugs] #20516 [Metrics/CollecTor]: CollecTor's exitlists module should avoid httpurlconnection

[Tor Bug Tracker & Wiki]

#20516: CollecTor's exitlists module should avoid httpurlconnection
-------------------------------+---------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_review
 Priority:  Medium             |      Milestone:  CollecTor 1.2.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  metrics-help       |  Actual Points:
Parent ID:  #8799              |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------------------
Changes (by iwakeh):

 * status:  accepted => needs_review


Comment:

 Simply switching to file copy would mean a major change here as the
 current download approach reads the data to memory for verifying and after
 that writes to file(s).
 A toy example of an http-server that accepts connections, but never writes
 the response code yielded the stack-trace provided in #22602.  The easiest
 remedy is setting a read-timeout.
 (A quick inspection of the code that gets used when using Files.copy
 indicates that a timeout seems to be set there, too.)

 Open points:
 * the current read timeout is arbitrarily set to 5000ms. How to find the
 best value?
 * Should a connect timeout be set, too?  There is no bug report that
 indicates necessity for this timeout type.

 Please review
 [https://gitweb.torproject.org/user/iwakeh/collector.git/commit/?h=task-20516&id=78c7727538709a3cec2d1f2ba950bc23dbc5dc8c
 this change].

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20516#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online