[tor-bugs] #2334 [Torouter]: Torouter breaks with large cached-descriptors[.new] files

[Tor Bug Tracker & Wiki]

#2334: Torouter breaks with large cached-descriptors[.new] files
----------------------+-----------------------------------------------------
 Reporter:  karsten   |       Owner:  ioerror      
     Type:  defect    |      Status:  new          
 Priority:  blocker   |   Milestone:               
Component:  Torouter  |     Version:  Tor: 0.2.1.26
 Keywords:            |      Parent:               
----------------------+-----------------------------------------------------
Changes (by karsten):

  * priority:  major => blocker


Comment:

 My Torouter bridge breaks after 2.5 days without a single client
 connecting. Here's the course of action:

  - Dec 29 14:06:59: started Tor
  - Dec 31 13:30:15: 17.5M left in /var/lib/tor/, 17704 bytes RAM left, Tor
 still running
  - Jan 1 06:28:43: Tor complains for the first time that there's no space
 left to write cached-descriptors.tmp, but it keeps running
  - Jan 1 18:38:13: 9.4M left in /var/lib/tor/, 11072 bytes RAM left, Tor
 still running
  - Jan 3 07:47:55: 0 bytes left in /var/lib/tor/, 1220 bytes RAM left, Tor
 still running, killed

 As a next step we should investigate how many descriptors we need to cache
 as a bridge (or as a relay). We can probably throw out descriptors more
 quickly.

 I'm going to set up a new directory mirror to a) take hourly snapshots of
 the cached-descriptors* files and b) log the requested descriptor digests
 (using log granularity of 15 minutes). With these data we can answer 1)
 for what consensuses we're keeping descriptors and 2) which consensuses
 clients use to decide which descriptors to download. Once we know that
 clients don't download old descriptors, we can stop caching them.

 Another question is whether we can avoid writing the cached-
 descriptors.tmp file and instead delete cached-descriptors and recreate
 it. In the unlikely case that we crash during this operation we'd have to
 download them once again. Maybe we can make this an option for devices
 with limited disk space.

 The next problem we'll have to solve is that we're running out of RAM.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/2334#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online