[tor-bugs] #11573 [Onionoo]: Ponder using a database for Onionoo rather than keeping indexes in memory and contents on disk

[Tor Bug Tracker & Wiki]

#11573: Ponder using a database for Onionoo rather than keeping indexes in memory
and contents on disk
-----------------------------+-----------------
     Reporter:  karsten      |      Owner:
         Type:  enhancement  |     Status:  new
     Priority:  minor        |  Milestone:
    Component:  Onionoo      |    Version:
   Resolution:               |   Keywords:
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+-----------------

Comment (by iwakeh):

 Thanks, for the access/minute summary!

 > These steps are already separated: lookup is what `RequestHandler` does
 and compile+respond is what `ResponseBuilder` does.  Should be pretty easy
 to measure these steps separately in a lab setting.
 That's true.
 And, you're right to postpone major testing of the server until after
 #13089.

 >
 > Agreed.  I'm mostly thinking that it's not a priority to do this, but I
 don't question that using a database with good indexes is the better
 design.
 >
 I understand. Onionoo is running fine, currently. No need to change it in
 a hurry.
 Postgres ought to be fine.
 Considering Onionoo's "document retrieval setup" it would also be
 interesting to look into
 NoSQL databases. But, currently I have not a good idea about their
 performance.

 I would like to keep performance measuring separate from the application
 itself.
 As tomcat6 supports jmx and the considered embedded server ought as well,
 an MBean (re-usable with the coming embedded server) would be nice. It
 could even be used for nagios monitoring.

 (btw: The response time could be measured via the access log, cf. option
 {{{%D}}} and others
 [https://tomcat.apache.org/tomcat-6.0-doc/api/org/apache/catalina/valves/AccessLogValve.html
 here].
 Of course, currently it is easier to log it in addition to the other
 values.)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/11573#comment:9>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online