[tor-relays] A bit more evidence on circuit creation storms

[Dan Staples]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Just to add my experiences to the mix:

I started running a RPi relay back in January. It ran fine for several
months, until I started to get these circuit creation storms
periodically. It would come at random times, maybe once a week, and
would sometimes last for enough hours that it would knock down the Pi
and I'd have to reboot it.

While it was clearly CPU bound during the storms (90%+ shown by top),
my bandwidth was also completely saturated. I was seeing 3 Mb/s
traffic, as shown by ntop (great for monitoring bandwidth over time).
Shutting down Tor during the storms would reduce the traffic to <
100kb/s...so clearly the circuit storms eat bandwidth too. Gordon,
perhaps you had an upstream router that was preventing the traffic
flood during the circuit storms?

I asked Roger Dingledine about it at PETS a couple months ago, and he
suggested it might be a case where there is a nearby popular hidden
service that picked my relay as a guard node, and all of a sudden I
get flooded by requests for the hidden service. No idea how to test
the accuracy of this hypothesis.

Finally, I noticed that bandwidth-related config options had no effect
on the 3 Mb/s traffic flood during the circuit creation storms. I had:

RelayBandwidthRate 200 KB
RelayBandwidthBurst 200 KB
MaxAdvertisedBandwidth 200KB

...yet, still 3 Mb/s traffic floods. Even MaxOnionsPending 250, NumCPU
1, and AvoidDiskWrites 1 made no difference in my RPi's ability to
weather the storms. I eventually had to use QoS on my DD-WRT router to
set limits on the traffic it would pass to the Pi.

I will try your builds of 0.2.4 to see if that makes a difference.

cheers,
Dan

> Since I originally started keeping an eye on these on my Raspberry
> Pi relay (read: slow, resource-limited), I've got to wonder if the 
> circuit creation storms I was seeing months ago weren't normal
> network phenomena but some kind of test run.
> 
> We are talking going from 50-250 circuits to thousands of requests
> per *second* out of nowhere, and then if the machine survived it,
> the storm disappearing as suddenly as it came.  This was happening
> months ago, but less frequently and only on lower-end hardware.
> Now it's happening everywhere.
> 
> Even if the previous case *were* "normal" Tor network operation,
> I'd say it's a bug, but I'm suspicious that it was whatever is
> going on now in its test phase.
> 
> tor at t-3.net:
>> Also see a repeat of the odd log message with the 154.x net
>> address someone else described with the huge hexidecimal string
>> (40 hex chars, + sign, 40 more, on and on).
> 
> Here as well.  I believe this is the sign of an overloaded Tor 
> directory server.
> 
>>> Over roughly the same time frame I received an incredibly high
>>>  number of spam e-mails in one e-mail account that normally
>>> gets 20 or so a day on quiet days.  Perhaps this is another
>>> example of mal-ware in action.
> 
> Funny, one of the dropped connections during my storm last night
> was to port 993... :P
> 
> Best, - -Gordon M.

- -- 
http://disman.tl
OpenPGP key: http://disman.tl/pgp.asc
Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJSKInBAAoJEPZwdO29hkOpDZ4QAIjZqi9kXrcubZUg6v0nH6mP
V9LT4j2Md0MqE74JcyEIgIBYKMWbun+n417LMMkb9uLu2mld6Lf/YIU9iO57JqQJ
8dM1uX0LJb51MbOj6ZTZEwMgHI8pX/ZeLj4zQscAz2C2oBwapnOpxqKTbg76CM1G
DrP39iai9TpN+YfUY0FkxwdYTM46ADUvk/hbu/34CLXklOvrqWkK/Ta/nZ9zTebt
HkB/KJn1P0I5L87x2VmIhIH5JKK6lszDlZg+k96PRC8oK9a1icECCGsnFJpqwVrv
UyIsutezQHjT7HT4zJCNBxacHuK8VKbwayX6P/MSPpAGaobqEzSvWJmHdYRrIv2h
glM5qIIzB3RmJxvHW8hCF47T8n9B6fo+J5PZly3xtwLGzCATjehnnhGTzigIw8ro
WkQwMQr/3F5yQ0YS2YKh+KsQDhe2d1fYbrGtSPAG/z1RMuk2i5tmgyhAFJ8HvIfw
GozzAaxcy1dCdEvQIyv/j1LCtUUpCrPVGd51grDI+GB9b5HuIEsPoQCvisshVEpu
1vLKyP7VLv+fOnvVb7MCwiWnU1Q5uw3yC3kdig9tTuAWGiMgbGoWpkGiy5MNVVQA
Z2Q7uLnd7j6scTbpWgaa0Fx0HQ2euAXMwC+zS8g0PQXOEslGnnPs2Kqz5YTimtuh
LWLcZ3WMZvlShurfNLse
=mqNB
-----END PGP SIGNATURE-----