[tor-relays] 100K circuit request per minute for hours killed my relay

teor teor2345 at gmail.com
Sat Jul 22 11:11:46 UTC 2017


> On 22 Jul 2017, at 20:45, Felix <zwiebel at quantentunnel.de> wrote:
> 
> Hi everybody
> 
> I observed two circuit bursts around July 19th. On a poor server Tor crashed, on a strong it didn't. The logs below are from the crashed one.
> 
> I'm on FreeBSD. To my mind the circuit burst exhausts the nmbclusters (not memory, Inact tells). Strong servers have more. How long is more enough ? On FreeBSD nmbclusters is a boot tunable.

So increase kern.ipc.nmbclusters on your machine?

Using the calculations in 11.11.2. Network Limits in:
https://www.freebsd.org/doc/en/books/handbook/configtuning-kernel-limits.html

Tor uses a 16kB send and receive buffer by default.
There are ~7000 relays in the network, plus client and exit connections.
(Your relay is connected to about 5500 relays and clients/internet servers.)
But let's say there are 1000 client connections on Guards, and
10000 exit connections on Exits.

So for a Guard, you want   8,000 * (16kB + 16kB) * 2 / 2kB = 256,000 mbufs
And for an Exit, you want 17,000 * (16kB + 16kB) * 2 / 2kB = 544,000 mbufs

The exact figures will depend on your relay's bandwidth weights.
You should probably try doubling the current setting, until the issue goes
away.

Of course, if you don't have 512MB or 1GB of spare RAM for your kernel,
then you'll want to tune Tor instead.

> The strong server took 140k circuits for several hours.
> 
> Is it worth thinking about a Tor setting like MaxCircuitsPerTime ?
> MaxAdvertisedBandwidth won't help directly.

If you want to tune tor, if you're concerned about socket buffers, try:

ConstrainedSockets
ConstrainedSockSize

Or open sockets:

ConnLimit

If you're concerned about pending circuits or data, try:

MaxClientCircuitsPending
MaxOnionQueueDelay
MaxMemInQueues

> 1) Normal operation
> 
> kern.openfiles: 5516
> Mem: 36M Active, 1288M Inact, 455M Wired, 1452K Cache, 425M Buf, 191M Free
> Swap: 3072M Total, 3072M Free
> USERNAME   PRI NICE   SIZE    RES STATE    TIME     CPU COMMAND
> _tor        87    0   309M   283M RUN    125.7H  50.00% tor{tor}
> _tor        22    0   309M   283M uwait  167:32   3.96% tor{tor}
> 
> 
> 2) Last minute before crash:
> 
> kern.openfiles: 5467
> Mem: 296M Active, 935M Inact, 692M Wired, 21M Cache, 425M Buf, 27M Free
> Swap: 3072M Total, 1592K Used, 3070M Free
> USERNAME   PRI NICE   SIZE    RES STATE    TIME     CPU COMMAND
> _tor        85    0   465M   438M RUN    126.8H  43.99% tor{tor}
> _tor        33    0   465M   438M uwait  186:39  15.97% tor{tor}
> 
> 
> 3) Crashed:
> 
> kern.openfiles: 5455
> Mem: 295M Active, 936M Inact, 699M Wired, 21M Cache, 425M Buf, 21M Free
> Swap: 3072M Total, 1592K Used, 3070M Free
> kernel: [zone: mbuf_cluster] kern.ipc.nmbclusters limit reached
> 
> 
> 
> sysctl kern.ipc.nmbclusters
> 126138
> 
> 
> 
> Heartbeat: Tor's uptime is 15 days 16:00 hours, with 1611 circuits Heartbeat: Tor's uptime is 15 days 17:00 hours, with 40471 circuits Heartbeat: Tor's uptime is 15 days 18:00 hours, with 88052 circuits Heartbeat: Tor's uptime is 15 days 19:00 hours, with 78444 circuits [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy.

T

--
Tim Wilson-Brown (teor)

teor2345 at gmail dot com
PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n
xmpp: teor at torproject dot org
------------------------------------------------------------------------



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20170722/5feb185a/attachment.sig>


More information about the tor-relays mailing list