[tor-bugs] #9645 [Tor]: Tor client goes berserk downloading microdescriptors forever, if you rm -rf its datadir

[Tor Bug Tracker & Wiki]

#9645: Tor client goes berserk downloading microdescriptors forever, if you rm -rf
its datadir
------------------------+--------------------------------
     Reporter:  arma    |      Owner:
         Type:  defect  |     Status:  needs_review
     Priority:  normal  |  Milestone:  Tor: 0.2.4.x-final
    Component:  Tor     |    Version:
   Resolution:          |   Keywords:  tor-client
Actual Points:          |  Parent ID:
       Points:          |
------------------------+--------------------------------

Comment (by mr-4):

 Replying to [comment:9 arma]:
 > > I don't think we can afford to exit here, so big_hammer isn't a good
 option.
 +1. To me, "bug9645" is the best solution of all 3 proposals.

 > Are we likely to try writing them again later when we reform our journal
 file?
 This is what I was going to suggest as well (see below).

 > Are there bugs there? I was about to merge bug9645 into maint-0.2.4 for
 the new release when I realized it's the sort of thing that will produce
 weird edge cases. Maybe it's best to let it bake in master for a while
 before backporting it (or deciding not to).
 OK, I had a chance to run "bug9645" (although using 2.4.16-rc instead of
 .14) for the last 14 hours or so and here are my findings:

 1. The memory leak is now completely gone (so #9639 is fixed as Nick
 suggested earlier) - tor's memory footprint is extremely stable at ~17 to
 19MB.

 2. When I delete all files in datadir, *but* keep datadir (i.e. "rm -rf
 /var/lib/tor/*"), tor recreates them all, except the "cached-certs" file.
 This is valid for all cases of executing the file removal command: pre-,
 during and post- tor boot up.

 3. Things are a bit different when I delete all files *and* datadir (i.e.
 "rm -rdf /var/lib/tor"):

 - in "pre" tor boot up, tor just recreates everything, including datadir.

 - in "post" tor boot up (i.e. after I see that tor has booted up 100%),
 the situation is the same as described in the initial report above, minus
 the memory leaks. In other words, I still get the error messages, but have
 no memory leaks. Interestingly, the tor memory footprint doesn't seem to
 increase that much - it is very stable between 17 and 19MB.

 - in "during" tor boot up, the situation is similar as in "post" above,
 with the exception that the error messages are not that frequent (I had
 about a dozen such messages when running it for a couple of hours or so,
 compared to about 3 times that amount in running it in just over an hour
 in the "post" scenario).

 So, I think the tor code should be changed slightly so that the behaviour
 in "post" and "during" is consistent with "pre" boot scenario - if datadir
 doesn't exist, recreate it and if a specific file doesn't exist, do the
 same.

 Display error message only when that is not possible, but don't flood the
 logs (as is the case now). Maybe issue an error at the HEARTBEAT interval
 (6 hours in my case). Just a thought!

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9645#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online