[tor-bugs] #3406 [Torctl]: High memory usage from parse_ns_body

[Tor Bug Tracker & Wiki]

#3406: High memory usage from parse_ns_body
--------------------+-------------------------------------------------------
 Reporter:  atagar  |          Owner:              
     Type:  defect  |         Status:  needs_review
 Priority:  normal  |      Milestone:              
Component:  Torctl  |        Version:              
 Keywords:          |         Parent:              
   Points:          |   Actualpoints:              
--------------------+-------------------------------------------------------

Comment(by atagar):

 > Any reason why you didn't also provide the getIterator option for
 get_consensus?

 Because I've never used it so didn't think of it.

 > The problem is that the ConsensusTracker actually keeps all of
 NetworkStatus objects

 Gotcha. I was expecting for ns_map to contain a set of all NetworkStatus
 objects and for the following get_network_status to allocate a second set
 (so this change would halve the memory) but guess that's not the case - oh
 well.

 I'm not personally concerned about the memory usage of the
 ConsensusTracker since I don't use it. I was just hoping that this would
 be a nice side effect.

 > FYI, all your other branches look good to merge, but I can't merge them
 easily because the commit for #3406 is in them for some reason.

 Great. I'm not clear how you want this patch changed. It lowered the
 get_network_status memory usage dramatically so it's doing exactly what
 I'd intended.

 Btw, the reason the patches are based on each other was to avoid merging
 headaches for you (and also let me test the end result over these last few
 days). Otherwise you'd be having conflicts with many of these patches.

 If separate patches based on master is *really* what you want then you can
 still get this by cherry-picking the changes... and then doing the
 conflict resolution and retested you would have needed via that route. :P

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/3406#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online