[metrics-bugs] #25924 [Metrics/Statistics]: Improve execution time of onion service statistics module

[Tor Bug Tracker & Wiki]

#25924: Improve execution time of onion service statistics module
------------------------------------+--------------------------
     Reporter:  karsten             |      Owner:  metrics-team
         Type:  enhancement         |     Status:  new
     Priority:  Low                 |  Milestone:
    Component:  Metrics/Statistics  |    Version:
     Severity:  Normal              |   Keywords:
Actual Points:                      |  Parent ID:
       Points:                      |   Reviewer:
      Sponsor:                      |
------------------------------------+--------------------------
 Two years ago, in February 2016, we
 [https://lists.torproject.org/pipermail/tor-dev/2016-February/010408.html
 noticed] that some of our back-end modules had really long execution
 times. Back then we made improvements to two of these modules. I'm going
 to post a graph of execution times in the comments.

 Looks like we'll have to do it again, and this time work even harder on
 improving execution times. The onion service statistics module again takes
 roughly 2 hours to complete, with a clear trend. I assume that we'll reach
 execution times of 3 hours by the end of 2018 and 5 to 6 hours by the end
 of 2019.

 I'm going to post another graph with recent execution times. (Note that we
 don't have logs for just the onion service statistics module, but only for
 that module plus the previous module together; but that other module is
 still relatively fast, contributing with just a few minutes execution time
 to the graph.)

 I don't have concrete suggestions for improvements. And it's still early
 enough to try to get this work funded before we dive into it. That's why
 I'm setting priority to low. But it's clear that we'll have to do
 something here in 2019.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25924>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online