[tor-bugs] #17939 [Metrics/Onionoo]: Optimize the construction of details documents with field constraints

[Tor Bug Tracker & Wiki]

#17939: Optimize the construction of details documents with field constraints
-----------------------------+------------------------------
 Reporter:  fmap             |          Owner:  metrics-team
     Type:  enhancement      |         Status:  needs_review
 Priority:  Low              |      Milestone:
Component:  Metrics/Onionoo  |        Version:
 Severity:  Minor            |     Resolution:
 Keywords:                   |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:  irl              |        Sponsor:
-----------------------------+------------------------------
Changes (by karsten):

 * status:  assigned => needs_review
 * reviewer:   => irl


Comment:

 I picked this up yesterday evening. It's tempting to try out different
 approaches here to optimize this code. The most efficient code would
 likely skip the JSON parser/generator and operate on the string with
 regular expressions. And storing string indices in memory or on disk would
 certainly make this even more efficient. Alternatively, I could imagine
 using some fancy Jackson features like filters to skip fields we're not
 interested in during parsing.

 However, I think that these optimizations are not necessary, in particular
 with Varnish cashes in place. The resource that is even more limited than
 CPU time is developer time.

 Let's instead find a solution that turns that huge, hard-to-maintain code
 block into something we don't have to worry about in the next 4 years.

 I came up with another relatively simple idea: we let Jackson parse the
 pre-formatted details document string as `LinkedHashMap<String, Object>`,
 rather than `DetailsDocument`, throw out all keys except for those given
 in the `fields` parameter, and let Jackson format the remaining keys and
 values as JSON string again. The advantage is that we can refer to details
 document parts by key rather than by getter and setter. After all, we
 don't care about the values here, just the keys.

 I did some very quick performance tests showing that the patch takes about
 as long as the current code to build a response. That's not a surprise,
 because Jackson still has to read and write the same amount of data.

 Please review
 [https://gitweb.torproject.org/user/karsten/onionoo.git/commit/?h=task-17939&id=330f6f9d6895eaa14182d7435db526880bc3cea3
 commit 330f6f9 in my task-17939 branch].

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/17939#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online