On 22 Jul (23:28:51), Fran via tor-relays wrote:
Hey,
new non-exit relay, Debian 11, tor 0.4.7.8-1~d11.bullseye+1, ~ 1 week old (-> no guard)
KVM VM with atm 4 cores, host passthrough AMD EPYC (-> AES HW accel.).
As can be seen at the attached screenshots memory consumption is irritating as well as the quite high CPU load.
All was fine when it had ~100 Mbit/s but then onion skins exploded (110 per second -> up to 4k per second) as well as CPU and memory.
Tor complains:
Your computer is too slow to handle this many circuit creation requests!
Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy.
And from time to time memory killer takes action
torrc is pretty basic:
Nickname 123 ContactInfo 123 RunAsDaemon 1 Log notice syslog RelayBandwidthRate 2X MBytes RelayBandwidthBurst 2X MBytes SocksPort 0 ControlSocket 0 CookieAuthentication 0 AvoidDiskWrites 1 Address xxxx OutboundBindAddress yyyy ORPort xxxx:yyy Address [zzzz] OutboundBindAddress [zzz] ORPort [zzz]:xxx MetricsPort hhhh:sss MetricsPortPolicy accept fffffff DirPort yy Sandbox 1 NoExec 1 CellStatistics 1 ExtraInfoStatistics 1 ConnDirectionStatistics 1 EntryStatistics 1 ExitPortStatistics 1 HiddenServiceStatistics 1
Ideas/suggestions (apart from limiting BW) to fix this?
We are currently seeing huge memory pressure on relays. I was unsuccessful at finding any kind of memory leaks at the moment and so there is a distinct possibility that relays have been accumulating somehow legit memory. We are still heavily investigating all this and coming up with ways to reduce the footprint.
In the meantime, we know that the "CellStatistics" option is very very memory hungry and so you could disable that one and see if this stabilizes thing for you.
The other option that can help with memory pressure usually is the "MaxMemInQueues" (man 1 tor). Essentially, it tells "tor" when to start running its "out of memory handler" (OOM). It is usually set around 75% of your total memory but you could reduce it and see if this helps.
I would although, in the current network conditions, really NOT put it below 2GB. And if the OOM gets triggered too many times and you can spare memory, bump it up to 4GB at the very least.
The current network conditions are abnormal and, often couple with other things, creates these resource pressure on relays that we rarely experience and so our team needs to investigate a needle in a haystack when it happens.
Thanks for the report! And thanks to all to help us through these difficult times for our relays and users.
Cheers! David