[tor-bugs] #22836 [Metrics/Website]: Parse CollecTor's index.json and provide our own directory listing

[Tor Bug Tracker & Wiki]

#22836: Parse CollecTor's index.json and provide our own directory listing
-----------------------------+--------------------------------
 Reporter:  karsten          |          Owner:  karsten
     Type:  enhancement      |         Status:  needs_revision
 Priority:  Medium           |      Milestone:
Component:  Metrics/Website  |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:                   |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+--------------------------------

Comment (by iwakeh):

 Replying to [comment:12 karsten]:
 > Replying to [comment:11 iwakeh]:
 > > I'm wondering about the [https://gitweb.torproject.org/karsten
 /metrics-
 web.git/tree/website/src/main/java/org/torproject/metrics/web/CollecTorServlet.java?h=task-22836&id=59fa501ea830504534b3b309ed6280719edc1b9d#n60
 wait-construct] when no index is available:
 > > The thread could be interrupted for any reason a short time (e.g. 5ms)
 after calling wait and the index preparation might not be done.  And, more
 likely, it is not interrupted and then there is a wait of 10 seconds, but
 I'd guesstimate index creation takes way less than one second.  Thus, it
 might be useful to 1) embed the wait in a loop checking if the waiting
 time elapsed or if there is an index available.
 > > 2) Secondly, the actual call to wait should use only 200 ms (or some
 other amount way smaller than 10000) as value.
 >
 > Agreed about the loop.
 >
 > But keep in mind that the timeout is for the time between sending a
 request for an `index.json` file until receiving and processing the
 response. If this were a local operation I'd say 200 ms are enough, but
 with the possible network delay I'd say we need to give it at least a few
 seconds. If 10000 ms are too much, how about 2000 ms?

 I was referring to the wait-statement not the entire waiting time:
 {{{
 do
    wait(200)
 until(10 sec are over or an index is available)
 }}}

 The checks in the 'until' part are cheap and will improve response times.
 I assume we'll hardly ever reach the ten second limit.
 In addition, there could be some logging (on debug) in order to adjust the
 values later.

 >
 > > There are three `format*` methods together with
 `extractDirectoryListings`, which do not depend on the actual object and
 should be made static.  As these `format*` plus `extractDirectoryListings`
 methods prepare the output, I really would like to see tests here.  This
 is not an empty exercise, but facilitates easier maintenance in future and
 enables review in first place.
 >
 > Agreed.
 >
 > This is on my list now. Will post a revised branch once I have one.
 (Thanks!)

 Looking forward to the new branch :-)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22836#comment:13>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online