[tor-bugs] #25100 [Metrics/CollecTor]: Make CollecTor's webstats module use less RAM and CPU time

[Tor Bug Tracker & Wiki]

#25100: Make CollecTor's webstats module use less RAM and CPU time
-------------------------------+--------------------------------
 Reporter:  karsten            |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_revision
 Priority:  High               |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+--------------------------------

Comment (by karsten):

 I think that an in-memory database is the second-best solution. The
 "manual compression" sounds more promising to me, because it leverages a
 specific redundancy of web server logs. Of course, we could further
 normalize the data and store request line parts in separate tables. But
 I'd say that the effort to make code changes and get them reviewed is
 several times as high as using the suggested data structure, whereas we'd
 already achieve 2/3 or 3/4 of possible improvements just from that without
 using a database.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25100#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online