[tor-bugs] #4673 [Metrics Website]: Partition the 67, 000, 000 row statusentry table in the metrics database

[Tor Bug Tracker & Wiki]

#4673: Partition the 67,000,000 row statusentry table in the metrics database
-----------------------------+----------------------------------------------
 Reporter:  karsten          |          Owner:  karsten
     Type:  enhancement      |         Status:  new    
 Priority:  major            |      Milestone:         
Component:  Metrics Website  |        Version:         
 Keywords:                   |         Parent:         
   Points:                   |   Actualpoints:         
-----------------------------+----------------------------------------------
 Thomas Termin suggested splitting the huge statusentry table in the
 metrics database into multiple tables to solve some of our metrics
 website/database performance problems.  Tim Wilde chimed in saying they're
 already doing that for large tables down to the hour level.

 There was some more discussion about splitting the whole table covering 3+
 years of data into 36+ month tables and adding a new month table every
 month.  Another suggestion was to move old data into a history table of
 some kind using a cron-job-like stored procedure.  I later saw Tim explain
 something about using a year table, month tables, etc. down to hour tables
 and deciding in the application which table(s) to query.  Basically, there
 was some discussion whether to do the splitting and merging in the
 database or in the application.

 A good next step would be to look at the PostgreSQL documentation for
 [http://www.postgresql.org/docs/8.4/static/ddl-partitioning.html
 partitioning tables].  I'm also going to look more at the schema and Java
 code to find the places which are affected by such a change.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/4673>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online