Michael Berlin:
Hi Gordon and Matthias,
I've split your discussion from the original thread "Running exit-node in Germany" and created a new one.
I fully agree with you that the Raspberry Pi is the perfect device to let others run a Tor Relay Node very easily. What follows is a long mail about my experiences and more thoughts about the Pi as relay.
[snip]
I've also attached the log of notices of the last 4 days (unfortunately no longer logs available). Within four days there was only one "Your computer is too slow to handle this many circuit creation requests!" warning and one "Failed to hand off onionskin.". In general I would say, I haven't seen any serious issues so far.
Here's why I'm still working on tuning. After discovering that my relay on the Pi was being rate-limited by an error in QoS in my router, I fixed that and the traffic immediately spiked (used up the BandwidthBurst bucket, I presume), then settled down - but after I went to bed, the following happened. It's a classic example of what I call a "circuit creation storm," which my bigger VPS nodes can handle, but the Pi can't. After the final log message, as far as I can tell, the process was killed for eating too much memory.
Note the serendipitous heartbeat message in the middle, which mentions 2680 circuits open. I freed the Pi up to push about 2.6 Mbps. On my bigger VPS relays, *most* of the time a 5Mbps relay will stabilize well below 2500 circuits in my experience.
At the last line, the log ends, and tor was not running when I woke up, so without checking further I'm going to assume the OS killed it as it chewed through all available memory. I'm kind of amazed that it didn't crash my router, but I did some tuning on the router the last time it did so. It's a consumer router running an alt firmware, but the NAT table only has 4096 entries or so - after the prior times where these "circuit creation storms" crashed my *router* I set timeouts on half-closed and abandoned TCP connections very aggressively, and it seems to survive now.
The point, though, is that for sticking these things on the broadband of friends and family:
* Tor shouldn't crash
* Tor shouldn't crash their routers, which are often less forgiving than mine with the NAT table space and default timeouts
* Tor shouldn't make them notice any degradation in streaming video services. I'm serious. If we can't manage that *on the Tor box itself* (possibly by using an alternate, highly conservative TCP congestion avoidance algo[1]?) then a lot of people will end up taking them off their network eventually.
I will continue to research ways to avoid this - MaxAdvertisedBandwidth is a very crude tool, and I swear I read someplace about a MaxOpenCircuits or MaxCircuitRequests type setting slated to go in soon...?
For now (and please feel free to shoot this full of holes, I'm by no means an iptables wizard, and this was cooked up before *any* caffeine this morning), I'm going to try limiting all SYNs with iptables:
iptables -A INPUT -p tcp --syn -m limit --limit 4/s --limit-burst 10 -j ACCEPT iptables -A INPUT -p tcp --syn -j LOG iptables -A INPUT -p tcp --syn -j REJECT
[1] http://arstechnica.com/information-technology/2012/05/codel-buffer-managemen... ... or for easy pasting, http://v.gd/An7s4B
Aug 06 23:46:19.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. Aug 06 23:46:19.000 [warn] Failed to hand off onionskin. Closing. Aug 06 23:49:21.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [1337 similar message(s) suppressed in last 60 seconds] Aug 06 23:50:42.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [121 similar message(s) suppressed in last 60 seconds] Aug 06 23:51:03.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [445 similar message(s) suppressed in last 60 seconds] Aug 06 23:51:43.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [5248 similar message(s) suppressed in last 60 seconds]
[snipping more of the same]
Aug 06 23:55:49.000 [notice] Heartbeat: Tor's uptime is 2 days 12:00 hours, with 2680 circuits open. I've sent 5.67 GB and received 5.32 GB.
Aug 06 23:56:28.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [2580 similar message(s) suppressed in last 60 seconds] Aug 06 23:58:26.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [271 similar message(s) suppressed in last 60 seconds] Aug 06 23:58:30.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [938 similar message(s) suppressed in last 60 seconds] Aug 06 23:59:30.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [1014 similar message(s) suppressed in last 60 seconds] Aug 07 00:00:30.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [1511 similar message(s) suppressed in last 60 seconds]