[tor-bugs] #11573 [Onionoo]: Ponder using a database for Onionoo rather than keeping indexes in memory and contents on disk

Tor Bug Tracker & Wiki blackhole at torproject.org
Fri Sep 5 07:01:56 UTC 2014


#11573: Ponder using a database for Onionoo rather than keeping indexes in memory
and contents on disk
-----------------------------+-----------------
     Reporter:  karsten      |      Owner:
         Type:  enhancement  |     Status:  new
     Priority:  minor        |  Milestone:
    Component:  Onionoo      |    Version:
   Resolution:               |   Keywords:
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+-----------------

Comment (by karsten):

 Your patch was merged together with the #12882 patch.  I tweaked it a bit
 to log all requests taking more than 1 second ''in total'' and included
 three durations in the log message:

  - `lookup`: how long did it take to parse the request and look up relays
 and bridges to include in the response?
  - `compile`: how long did it take to read files from disk and compile a
 response string (this required a code change, because we're now writing
 into a local StringWriter first and then write the result into the
 response's PrintWriter)?
  - `respond`: how long did it take to send the response to the client?

 I deployed the patched version on the production server and got statistics
 from the last 12 hours.  Here's the summary:

 {{{
                 datetime        lookup           compile          respond
  2014-09-04 18:27:12:  12   Min.   :   4.00   Min.   :   0.0   Min.   :
 0
  2014-09-05 03:27:14:  12   1st Qu.:  13.00   1st Qu.:  78.0   1st Qu.:
 1129
  2014-09-04 18:27:23:  11   Median :  15.00   Median :  89.0   Median :
 1455
  2014-09-04 21:27:34:  11   Mean   :  19.42   Mean   : 280.9   Mean   :
 3713
  2014-09-04 19:28:07:  10   3rd Qu.:  17.00   3rd Qu.: 111.0   3rd Qu.:
 3772
  2014-09-04 22:27:10:  10   Max.   :3021.00   Max.   :6071.0   Max.
 :503809
  (Other)            :6438
     document                              params         chars
  details:1768   []                           :5060   Min.   :     213
  summary:4670   [flag, type, running]        : 994   1st Qu.: 1437133
  uptime :  30   [lookup, fields]             : 112   Median : 1437411
  weights:  36   [fingerprint]                :  64   Mean   : 2487374
                 [type, running, fields]      :  49   3rd Qu.: 1437830
                 [flag, type, running, fields]:  46   Max.   :14389566
                 (Other)                      : 179
 }}}

 There are plenty of bottlenecks to analyze here, which I don't have the
 cycles for right now.  I'm attaching the .csv file.  Maybe you want to
 take a look?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/11573#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list