[tor-bugs] #24737 [Core Tor/Tor]: Recommend a MaxMemInQueues value in the Tor man page

[Tor Bug Tracker & Wiki]

#24737: Recommend a MaxMemInQueues value in the Tor man page
----------------------------+----------------------------------
 Reporter:  starlight       |          Owner:  (none)
     Type:  defect          |         Status:  new
 Priority:  Medium          |      Milestone:  Tor: unspecified
Component:  Core Tor/Tor    |        Version:
 Severity:  Normal          |     Resolution:
 Keywords:  doc, tor-relay  |  Actual Points:
Parent ID:                  |         Points:  0.5
 Reviewer:                  |        Sponsor:
----------------------------+----------------------------------

Comment (by starlight):

 update from the trenches:

 Situation is dramatically better and better understood.

 Major caveat, certainly should be mentioned: all-bets-are-off if
 collection for either of

 `CellStatistics 1`
 `ConnDirectionStatistics 1`

 are set.  CellStatistics in particular results in excess gigabytes of
 memory consumption on busy relays.


 0.3.4.1-alpha stats for two relays running now for ten days
 relays not experiencing attack or abuse activity

 {{{
 medium-size guard with slow CPU, excess BW capacity
 ===================================================
 ~11M self-measure
 ~4M Blutmagie BW avg

 MaxMemInQueues 1024MB

 /proc/meminfo

 MemAvailTot:     3071560
 MemTotal:        4059768 kB
 MemFree:         2808820 kB
 Cached:           262740 kB
 Mlocked:          698840 kB
 SwapTotal:       2097144 kB
 SwapFree:        2083096 kB
 Dirty:                72 kB
 Slab:             295564 kB
 CommitLimit:     4127028 kB
 Committed_AS:     594968 kB

 /proc/$(pgrep tor)/status

 VmPeak:   836876 kB
 VmSize:   689024 kB
 VmHWM:    777472 kB
 VmRSS:    629680 kB
 VmData:   551248 kB
 }}}




 {{{
 somewhat fast Exit, consensus rank ~170, exit rank ~60
 ======================================================
 ~25M self-measure
 ~16M Blutmagie BW avg

 MaxMemInQueues 2048MB

 /proc/meminfo

 MemAvailTot:     13784300
 MemTotal:       16457808 kB
 MemFree:        13482956 kB
 Cached:           301344 kB
 Mlocked:          939808 kB
 SwapTotal:       4194296 kB
 SwapFree:        4194296 kB
 Dirty:               236 kB
 Slab:             754436 kB
 CommitLimit:    12423200 kB
 Committed_AS:    1506840 kB

 /proc/$(pgrep tor)/status

 VmPeak:  1181552 kB
 VmSize:   988900 kB
 VmHWM:   1129152 kB
 VmRSS:    936500 kB
 VmData:   855580 kB
 }}}

 Observed similar values in recent months running 0.3.3, including the
 final days
 of last winter's overload attacks.

 In light of the observations and the numerous improvements in memory OOM
 accounting,
 reporting and mitigation, plus the new circuit queued-cell maximum logic,
 it appears
 safe to recommend MaxMemInQueues values incorporating reasonable premiums
 that allow
 for usual OS-process overheads.  Perhaps physical memory of 120% or 130%
 MaxMemInQueues
 per dameon instance?  If Shadow-environment tests for simulating attacks
 exist it would
 be worth running them against 0.3.4 before arriving at final
 recommendations.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/24737#comment:13>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online