[tor-bugs] #4673 [Metrics Website]: Partition the 67, 000, 000 row statusentry table in the metrics database

Tor Bug Tracker & Wiki torproject-admin at torproject.org
Thu Dec 8 07:59:35 UTC 2011

#4673: Partition the 67,000,000 row statusentry table in the metrics database
 Reporter:  karsten          |          Owner:  karsten
     Type:  enhancement      |         Status:  new    
 Priority:  major            |      Milestone:         
Component:  Metrics Website  |        Version:         
 Keywords:                   |         Parent:         
   Points:                   |   Actualpoints:         
 Thomas Termin suggested splitting the huge statusentry table in the
 metrics database into multiple tables to solve some of our metrics
 website/database performance problems.  Tim Wilde chimed in saying they're
 already doing that for large tables down to the hour level.

 There was some more discussion about splitting the whole table covering 3+
 years of data into 36+ month tables and adding a new month table every
 month.  Another suggestion was to move old data into a history table of
 some kind using a cron-job-like stored procedure.  I later saw Tim explain
 something about using a year table, month tables, etc. down to hour tables
 and deciding in the application which table(s) to query.  Basically, there
 was some discussion whether to do the splitting and merging in the
 database or in the application.

 A good next step would be to look at the PostgreSQL documentation for
 partitioning tables].  I'm also going to look more at the schema and Java
 code to find the places which are affected by such a change.

Ticket URL: <https://trac.torproject.org/projects/tor/ticket/4673>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online

More information about the tor-bugs mailing list