[tor-bugs] #27356 [Metrics/ExoneraTor]: Reduce database size and variance of query response times

[Tor Bug Tracker & Wiki]

#27356: Reduce database size and variance of query response times
--------------------------------+------------------------------
 Reporter:  karsten             |          Owner:  metrics-team
     Type:  enhancement         |         Status:  merge_ready
 Priority:  High                |      Milestone:
Component:  Metrics/ExoneraTor  |        Version:
 Severity:  Normal              |     Resolution:
 Keywords:                      |  Actual Points:
Parent ID:                      |         Points:
 Reviewer:  irl                 |        Sponsor:
--------------------------------+------------------------------

Comment (by karsten):

 Replying to [comment:10 irl]:
 > Replying to [comment:9 karsten]:
 > > Replying to [comment:8 irl]:
 > > > I've checked it over for any obvious errors and the tests pass, but
 I notice that none of the tests actually use the database.
 > >
 > > That's true. Having more useful tests in ExoneraTor is, unfortunately,
 a little project of its own. We already have #24365 for this, but it's not
 as high priority as it could/should be. Let's try to leave room for these
 things in the next roadmap.
 >
 > Ok.
 >
 > > The commit message of 8159855 explains what the changes are all about.
 >
 > Ok. The changes look to implement what is described and the strategy
 looks good too.

 Perfect! I'll start the migration then by updating the local snapshot and
 uploading it back to the server. This is probably going to take a few
 days.

 > Thinking about handling schema changes, the comment says that
 `exonerator.sql` will go away and the new one will be modified to replace
 it. I've seen other software keep all the revisions and upgrade scripts
 since the beginning of the project (for example, observium) and
 installation starts with the original schema and then upgrades it. Perhaps
 this is useful to prevent bugs creeping in when the script is changed to
 replace the original script?

 We can do that, too. I wonder if there's a way to provide the latest
 schema after applying all upgrade scripts. Maybe we can auto-generate that
 by running all scripts and then exporting the schema without data. Without
 something like this, new contributors will have a hard time going through
 all scripts just to learn the current state of things.

 > Other than that, I think this is the limit of what I can review without
 standing up an instance to test on.

 Thanks! At least it's good to know that I did not make any obvious
 mistakes. And we'll still find out if it's ''100% bug free'' after
 deploying and running it for a while. ;)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27356#comment:11>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online