[metrics-bugs] #32660 [Metrics/Onionoo]: onionoo-backend is killing the ganeti cluster

[Tor Bug Tracker & Wiki]

#32660: onionoo-backend is killing the ganeti cluster
-----------------------------+------------------------------
 Reporter:  anarcat          |          Owner:  metrics-team
     Type:  defect           |         Status:  new
 Priority:  Medium           |      Milestone:
Component:  Metrics/Onionoo  |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:                   |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------------

Comment (by irl):

 I have suspended the hourly updater on this host, but this is the normal
 expected operation for Onionoo's hourly updater. There are state files
 that get updated every run including those for which nothing has changed.
 This is a known flaw in Onionoo but until now it hasn't been a problem.

 karsten is going to look at how difficult it would be to reduce the number
 of writes performed. Is the problem total IO or is it just the writes? Are
 reads cached? i.e. if we read it again to compare before writing, does
 that help?

 There's very little we can do about the CPU load. We already use optimized
 parsing libraries for JSON, and quite simple parsers for Tor descriptors.
 Metrics does involve some computation. If CPU load is a problem then
 perhaps the Ganeti cluster is the wrong place for Onionoo to live and we
 need something else.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/32660#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online