[tor-bugs] #25141 [Core Tor/Tor]: enabling CellStatistics results in gigabytes of incremental memory consumption

[Tor Bug Tracker & Wiki]

#25141: enabling CellStatistics results in gigabytes of incremental memory
consumption
-------------------------------------------------+-------------------------
 Reporter:  starlight                            |          Owner:  (none)
     Type:  defect                               |         Status:  new
 Priority:  High                                 |      Milestone:  Tor:
                                                 |  0.3.3.x-final
Component:  Core Tor/Tor                         |        Version:  Tor:
                                                 |  0.3.2.9
 Severity:  Major                                |     Resolution:
 Keywords:  031-backport, 029-backport, ddos,    |  Actual Points:
  must-fix-before-033-stable                     |
Parent ID:  #24806                               |         Points:  1
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------
Changes (by arma):

 * cc: karsten (added)


Comment:

 We're keeping a whole lot of stuff in memory.

 {{{
 diff --git a/src/or/rephist.c b/src/or/rephist.c
 index 15fb674..bb59b6b 100644
 --- a/src/or/rephist.c
 +++ b/src/or/rephist.c
 @@ -2390,6 +2390,8 @@ rep_hist_add_buffer_stats(double
 mean_num_cells_in_queue,
    if (!circuits_for_buffer_stats)
      circuits_for_buffer_stats = smartlist_new();
    smartlist_add(circuits_for_buffer_stats, stats);
 +  log_info(LD_HIST, "circuits_for_buffer_stats now size %d",
 +           smartlist_len(circuits_for_buffer_stats));
  }

  /** Remember cell statistics for circuit <b>circ</b> at time
 }}}

 Every time a circuit closes, we malloc some stuff and add it onto this
 smartlist. And we only do something with the contents of the smartlist
 after 24 hours.

 During this overload period, my relay handles many hundreds of millions of
 circuits in a day. Keeping *anything* about each one of them is going to
 be too much.

 What do we actually use this option for? Some experiment we did in the
 distant past? It sounds like "recommend against enabling it" is the first
 step. And then either redesigning it to accomplish whatever goals remain,
 or stripping it out.

 I am cc'ing Karsten because I bet he can help us with the 'distant past'
 part.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25141#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online