June 2013 - tor-relays - lists.torproject.org

Circuit creation "storms" overwhelming Raspberry Pi?
by torsion＠ftml.net 08 Jul '13

08 Jul '13

Hi there, I just joined the mailing list and apologized if this has been discussed before. I did find discussion of a similar issue in January 2013's archive: https://lists.torproject.org/pipermail/tor-relays/2013-January/001809.html It's important to note that I believe I've seen (but didn't save logs) a couple "circuit creation burst" events on my established relay (about 5Mbps, stable, guard, non-exit) which was mostly able to handle it without crashing as it has plenty of RAM and the above-mentioned messages - "Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a m ore restricted exit policy." - appear only with the relay is under load for other reasons AND a large number of circuits are being suddenly created. I wondered if this was some kind of DOS attempt but didn't think much of it because my fast relay continued working fine. However, I've just set up a Raspberry Pi, the 512MB model, as a relay on a slower connection. Here are the relevant settings on this relay: RelayBandwidthRate 130 KB RelayBandwidthBurst 340 KB The Pi has a fairly slow CPU, so I'd occasionally get messages about log deduplication being too slow or something, but didn't think much of it. I finally got the relay up and left it up for over 24 hours. When I woke up this morning it had crashed. Here are the relevant log messages - note the huge jump in number of circuits between 22:35 and 04:35 (maybe I got the Stable flag), then the storm of circuit open requests starting at 05:49. Eventually I believe the Pi ran out of memory and killed the tor process. What's very interesting here is that my fast VPS relay with a RelayBandwidthRate over 5x faster is almost never handling much more than 1000 circuits, so why all of a sudden the demand on the Pi that's advertising a lower bandwidth rate? Mar 17 22:35:00.000 [notice] Heartbeat: Tor's uptime is 1 day 0:00 hours, with 26 circuits open. I've sent 974.13 MB and received 969.92 MB. Mar 18 04:35:00.000 [notice] Heartbeat: Tor's uptime is 1 day 6:00 hours, with 972 circuits open. I've sent 1.61 GB and received 1.59 GB. Mar 18 05:49:44.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. Mar 18 05:49:44.000 [warn] Failed to hand off onionskin. Closing. Mar 18 05:50:44.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [5817 similar message(s) suppressed in last 60 seconds] Mar 18 05:52:30.000 [warn] Your system clock just jumped 101 seconds forward; assuming established circuits no longer work. Mar 18 05:53:51.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [1055 similar message(s) suppressed in last 60 seconds] Mar 18 05:55:14.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [329 similar message(s) suppressed in last 60 seconds] I'd like to figure out just how much the Raspberry Pi is capable of, because it could be a cheap way to build out the relay network by people who want to donate bandwidth - but of course it needs to be stable, and something about my setup is not. Also: Mar 16 20:55:33.000 [notice] No AES engine found; using AES_* functions. I have no idea if the Broadcom BCM2835 SoC (ARM1176JZF-S CPU) in the Pi has any AES capability, but it'd be great to find out.

3 4

Tor Exit Node and Skype
by var 28 Jun '13

28 Jun '13

Hello everybody, I am running a Tor exit node since some year´s. In the last month I experienced problems with Skype. Skype won´t connect anymore....but if I shutdown the EXIT NODE Skype connects again. Just want to ask all of you if you experienced similar problems? I also opened a ticket at Skype.com So if you got the same problem like me pls go there and post some additional information. > http://community.skype.com/t5/Security-Privacy-Trust-and/Is-Skype-blocking-… Cheers VarVarna

6 5

Munin graphs from fast relays before/after crypto acceleration
by Karsten Loesing 19 Jun '13

19 Jun '13

Hi fast-relay operators, I'm looking for some Munin graphs (or similar) showing the effect of turning on crypto acceleration on fast relays. Does anyone have such graphs to share that show decrease in CPU load and corresponding increase in bandwidth use? Thanks! Karsten

1 0

Upload and download data usage do not match
by Torry Torah 17 Jun '13

17 Jun '13

Hi, I've set up my relays as obfuscated bridges and when I run arm, I see that there's been ~100MB of data downloaded but only ~20MB of data uploaded. Is it the desired behaviour, or is there a config issue on my side? Cheers, -- Torry

1 0

Re: [tor-relays] [tor-talk] Tor 0.2.4.13-alpha is out
by Mike Perry 17 Jun '13

17 Jun '13

Roger Dingledine: > Tor 0.2.4.13-alpha fixes a variety of potential remote crash > vulnerabilities, makes socks5 username/password circuit isolation > actually actually work (this time for sure!), and cleans up a bunch > of other issues in preparation for a release candidate. > > https://www.torproject.org/dist/ As a heads up, a bug was introduced in this release that allows malicious websites to discover a client's Guard nodes in a very short amount of time (on the order an hour), if those Guard nodes upgrade to this release. Unfortunately, the bug was introduced by fixing another issue that allows Guard nodes to be selectively DoSed with an OOM condition, so Guard node (and Guard+Exit node) operators are kind of in a jam. I think the best course of action is to suggest that nodes with the Guard flag *not* upgrade to this release, unless they are experiencing unexplained OOMing? If we can't find a solution that rigorously fixes both issues, I think that future releases should have the OOM DoS fix off by default but available through a torrc option. See also: https://trac.torproject.org/projects/tor/ticket/9072 > Changes in version 0.2.4.13-alpha - 2013-06-14 > o Major bugfixes (robustness): > - Close any circuit that has too many cells queued on it. Fixes > bug 9063; bugfix on the 54th commit of Tor. This bug is a further > fix beyond bug 6252, whose fix was merged into 0.2.3.21-rc. > - Prevent the get_freelists() function from running off the end of > the list of freelists if it somehow gets an unrecognized > allocation. Fixes bug 8844; bugfix on 0.2.0.16-alpha. Reported by > eugenis. > - Avoid an assertion failure on OpenBSD (and perhaps other BSDs) > when an exit connection with optimistic data succeeds immediately > rather than returning EINPROGRESS. Fixes bug 9017; bugfix on > 0.2.3.1-alpha. > - Fix a directory authority crash bug when building a consensus > using an older consensus as its basis. Fixes bug 8833. Bugfix > on 0.2.4.12-alpha. > > o Major bugfixes: > - Avoid a memory leak where we would leak a consensus body when we > find that a consensus which we couldn't previously verify due to > missing certificates is now verifiable. Fixes bug 8719; bugfix > on 0.2.0.10-alpha. > - We used to always request authority certificates by identity digest, > meaning we'd get the newest one even when we wanted one with a > different signing key. Then we would complain about being given > a certificate we already had, and never get the one we really > wanted. Now we use the "fp-sk/" resource as well as the "fp/" > resource to request the one we want. Fixes bug 5595; bugfix on > 0.2.0.8-alpha. > - Follow the socks5 protocol when offering username/password > authentication. The fix for bug 8117 exposed this bug, and it > turns out real-world applications like Pidgin do care. Bugfix on > 0.2.3.2-alpha; fixes bug 8879. > - Prevent failures on Windows Vista and later when rebuilding the > microdescriptor cache. Diagnosed by Robert Ransom. Fixes bug 8822; > bugfix on 0.2.4.12-alpha. > > o Minor bugfixes: > - Fix an impossible buffer overrun in the AES unit tests. Fixes > bug 8845; bugfix on 0.2.0.7-alpha. Found by eugenis. > - If for some reason we fail to write a microdescriptor while > rebuilding the cache, do not let the annotations from that > microdescriptor linger in the cache file, and do not let the > microdescriptor stay recorded as present in its old location. > Fixes bug 9047; bugfix on 0.2.2.6-alpha. > - Fix a memory leak that would occur whenever a configuration > option changed. Fixes bug 8718; bugfix on 0.2.3.3-alpha. > - Paste the description for PathBias parameters from the man > page into or.h, so the code documents them too. Fixes bug 7982; > bugfix on 0.2.3.17-beta and 0.2.4.8-alpha. > - Relays now treat a changed IPv6 ORPort as sufficient reason to > publish an updated descriptor. Fixes bug 6026; bugfix on > 0.2.4.1-alpha. > - When launching a resolve request on behalf of an AF_UNIX control > socket, omit the address field of the new entry connection, used in > subsequent controller events, rather than letting tor_dup_addr() > set it to "<unknown address type>". Fixes bug 8639; bugfix on > 0.2.4.12-alpha. > > o Minor bugfixes (log messages): > - Fix a scaling issue in the path bias accounting code that > resulted in "Bug:" log messages from either > pathbias_scale_close_rates() or pathbias_count_build_success(). > This represents a bugfix on a previous bugfix: the original fix > attempted in 0.2.4.10-alpha was incomplete. Fixes bug 8235; bugfix > on 0.2.4.1-alpha. > - Give a less useless error message when the user asks for an IPv4 > address on an IPv6-only port, or vice versa. Fixes bug 8846; bugfix > on 0.2.4.7-alpha. > > o Minor features: > - Downgrade "unexpected SENDME" warnings to protocol-warn for 0.2.4.x, > to tolerate bug 8093 for now. > - Add an "ignoring-advertised-bws" boolean to the flag-threshold lines > in directory authority votes to describe whether they have enough > measured bandwidths to ignore advertised (relay descriptor) > bandwidth claims. Resolves ticket 8711. > - Update to the June 5 2013 Maxmind GeoLite Country database. > > o Removed documentation: > - Remove some of the older contents of doc/ as obsolete; move others > to torspec.git. Fixes bug 8965. > > o Code simplification and refactoring: > - Avoid using character buffers when constructing most directory > objects: this approach was unwieldy and error-prone. Instead, > build smartlists of strings, and concatenate them when done. > > _______________________________________________ > tor-talk mailing list > tor-talk(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk -- Mike Perry

2 1

Email to post to list
by S.P. Ace 14 Jun '13

14 Jun '13

Just emailing to be able to post. Thanks.

1 0

Multiple Bridges on same AWS Account
by torrelay＠mail.md 10 Jun '13

10 Jun '13

Greetings! I currently have 1 bridge relay running on my AWS account. Is it possible to add another 2 bridge relays to run as separate instances from the same AWS account? I'm wondering if there might be issues with IP address allocation? For example, would I need to make any ammendment to the tor configuration file a la 'tor family' to differentiate between the different nodes running on the same amazon account? Any advise appreciated. Best regards, Johnnie

5 10

Bandwidth/Accounting considerations
by Conrad Hoffmann 07 Jun '13

07 Jun '13

Dear all, I have now been running an exit node for approximately two weeks, it has a fast connection (100Mbit) but I have a traffic limit of 1000GB per month. I have been closely monitoring it via [1] and Munin. I believe to have read that using accounting/hibernation is preferrable over rate limiting with fast connections, but I can't seem to find the exact page at the moment. As you can see (well, guess) from [1], the node is up for about 6-8 hours until its daily quota (currently 10GB, may become a little more) is exhausted, with peaks of around 900K/s but usually around 400-500K/s. In my quest to maximize the usefulness of my node, I would very much appreciate if someone could explain or make educated guesses about the following questions: 1. Note that there are two significant dips in the exit probability chart [1]. The first one I can explain, because I had to reconfigure and restart the server. Nothing special happened after that. What may have caused the second dip? What tools could I use to figure it out/monitor it in the future? 2. My server is bored in terms of memory/CPU usage. I set NumCPUs to 2 and BandwidthRate 100 MB, BandwidthBurst 200 MB as suggested in [2]. Besides the fact that I wouldn't want to exhaust my daily quota in a few hours, shouldn't the traffic become more than 900K/s peaks? 3. What is a good trade-off in terms of speed and uptime with such a daily quota? Should I go for speed? If so, are there any pretty graphs that show at what time of the day the Tor network needs the most bandwidth? If I should go for uptime, what is a good bandwidth limit to keep this a fast node? 4. I enabled directory mirroring, but apparently this does not work with hibernation (i.e. it is not advertised). If I understand correctly, directory mirrors are very useful, so how does that weigh in on the decision to maybe limit the bandwidth to keep the node up 24/7? 5. Related to 4., why is DirPort not advertised when hibernation is configured? References to papers are sufficient if this requires a complicated answer :) [1] https://atlas.torproject.org/#details/5A91910C1B3F3FCC15EA6C6538FB9A5FAF360… [2] http://archives.seul.org/or/relays/Aug-2010/msg00034.html Thank you very much for any pointer on any of these questions! Cheers, Conrad

2 1

Re: [tor-relays] Circuit creation "storms" overwhelming Raspberry Pi?
by temp5＠tormail.org 06 Jun '13

06 Jun '13

I've been seeing these storms as well on my relay. I average something like 100 connections for weeks and weeks per the tor logs, but then suddenly it will jump into the thousands and I'll see the "Failed to hand off onionskin." and "Your computer is too slow to handle this many circuit creation requests!" messages. I wonder if it's some type of DDOS too. I thought about using this method (http://www.debian-administration.org/articles/187) on the relay and dir ports, but I'm not sure what sort of limits to set. Like does 1 Tor circuit = 1 iptables connection? Or if a user hits a webpage with 100 ads on it, maybe it would be 1 Tor circuit = 100 iptables connections? That's about as far as I got. I didn't want to break things by trying to fix another. > I did a lot of tuning on the Raspberry Pi and it's now much, much more > stable as a Tor relay, but just now I had another "circuit creation > storm." Interestingly, the Pi remained up, and my *router* crashed. > I've also seen huge bursts of circuit creation on a relay I run on a > VPS, but as it's a much more powerful box it rarely complains (and thus > I rarely notice). > > This many circuits and outbound connections is highly unusual for the > small relay I'm running on the Pi. And this behavior definitely occurs > in bursts. Is this an outbound DDOS or an attack on Tor itself? If the > former (or maybe the latter), is there some way I could perhaps use > iptables to temporarily "clamp" the ability to open TCP connections when > Tor (or anything on the Pi) opens a number over some threshold in some > short period of time? > > Here's log output (via 'arm') from the relay after my router crashed > twice, I went to the admin panel and noted hundreds of outbound > connections from my Tor box. Time is America/Los_Angeles. > > │ 13:55:00 [ARM_NOTICE] Relay unresponsive (last heartbeat: Sat May > 4 13:54:14 2013) > │ 13:52:25 [WARN] Your computer is too slow to handle this many > circuit > creation > │ requests! Please consider using the MaxAdvertisedBandwidth > config > option or choosing > │ a more restricted exit policy. [404 similar message(s) > suppressed > in last 60 seconds] > │ 13:51:07 [WARN] Your computer is too slow to handle this many > circuit > creation > │ requests! Please consider using the MaxAdvertisedBandwidth > config > option or choosing > │ a more restricted exit policy. [75 similar message(s) > suppressed in > last 60 seconds] > │ 13:50:52 [WARN] Your computer is too slow to handle this many > circuit > creation > │ requests! Please consider using the MaxAdvertisedBandwidth > config > option or choosing > │ a more restricted exit policy. [601 similar message(s) > suppressed > in last 60 seconds] > │ 13:48:39 [WARN] Your computer is too slow to handle this many > circuit > creation > │ requests! Please consider using the MaxAdvertisedBandwidth > config > option or choosing > │ a more restricted exit policy. [99 similar message(s) > suppressed in > last 60 seconds] > │ 13:47:34 [WARN] Your computer is too slow to handle this many > circuit > creation > │ requests! Please consider using the MaxAdvertisedBandwidth > config > option or choosing > │ a more restricted exit policy. [22 similar message(s) > suppressed in > last 60 seconds] > │ 13:46:17 [WARN] Your computer is too slow to handle this many > circuit > creation > │ requests! Please consider using the MaxAdvertisedBandwidth > config > option or choosing > │ a more restricted exit policy. [253 similar message(s) > suppressed > in last 60 seconds] > │ 13:43:47 [WARN] Your computer is too slow to handle this many > circuit > creation > │ requests! Please consider using the MaxAdvertisedBandwidth > config > option or choosing > │ a more restricted exit policy. [1396 similar message(s) > suppressed > in last 60 > │ seconds] > │ 13:42:48 [WARN] Your computer is too slow to handle this many > circuit > creation > │ requests! Please consider using the MaxAdvertisedBandwidth > config > option or choosing > │ a more restricted exit policy. [16 similar message(s) > suppressed in > last 60 seconds] > > Here's how it crashed my router (blowing ip_conntrack limits is > sufficient only to mess up many of my TCP connections, but eventually > the router runs out of memory and starts killing processes): > > May 4 13:51:24 dedmaus user.warn kernel: ip_conntrack: table full, > dropping packet. > May 4 13:51:24 dedmaus user.warn kernel: ip_conntrack: table full, > dropping packet. > May 4 13:51:24 dedmaus user.warn kernel: ip_conntrack: table full, > dropping packet. > May 4 13:51:25 dedmaus user.warn kernel: ip_conntrack: table full, > dropping packet. > May 4 13:51:29 dedmaus user.warn kernel: NET: 152 messages suppressed. > May 4 13:51:29 dedmaus user.warn kernel: ip_conntrack: table full, > dropping packet. > May 4 13:51:34 dedmaus user.warn kernel: NET: 193 messages suppressed. > May 4 13:51:34 dedmaus user.warn kernel: ip_conntrack: table full, > dropping packet. > May 4 13:51:39 dedmaus user.warn kernel: NET: 227 messages suppressed. > > ...ad infinitum with the number of messages suppressed per 5 sec > increasing until the router crashes. > > > > On Mon, Mar 18, 2013, at 06:18 PM, torsion at ftml.net wrote: >> I'm also seeing occasional messages like this on the Pi (it never lasts >> long): >> >> 18:13:24 [ARM_NOTICE] Relay resumed >> 18:13:18 [ARM_NOTICE] Relay unresponsive (last heartbeat: Mon Mar 18 >> 18:13:04 2013) >> 17:28:43 [ARM_NOTICE] Relay resumed >> 17:28:38 [ARM_NOTICE] Relay unresponsive (last heartbeat: Mon Mar 18 >> 17:28:25 2013) >> 14:12:26 [ARM_NOTICE] Relay resumed >> 14:12:20 [ARM_WARN] Deduplication took too long. Its current >> implementation has difficulty handling large logs so disabling it to >> keep the interface responsive. >> 14:12:20 [ARM_NOTICE] Relay unresponsive (last heartbeat: Mon Mar 18 >> 14:12:06 20 >> >> On Mon, Mar 18, 2013, at 01:00 PM, torsion at ftml.net wrote: >> > Hi there, I just joined the mailing list and apologized if this has >> been >> > discussed before. I did find discussion of a similar issue in January >> > 2013's archive: >> > >> > https://lists.torproject.org/pipermail/tor-relays/2013-January/001809.html >> > >> > It's important to note that I believe I've seen (but didn't save logs) >> a >> > couple "circuit creation burst" events on my established relay (about >> > 5Mbps, stable, guard, non-exit) which was mostly able to handle it >> > without crashing as it has plenty of RAM and the above-mentioned >> > messages - "Your computer is too slow to handle this many circuit >> > creation requests! Please consider using the MaxAdvertisedBandwidth >> > config option or choosing a m ore restricted exit policy." - appear >> only >> > with the relay is under load for other reasons AND a large number of >> > circuits are being suddenly created. >> > >> > I wondered if this was some kind of DOS attempt but didn't think much >> of >> > it because my fast relay continued working fine. >> > >> > However, I've just set up a Raspberry Pi, the 512MB model, as a relay >> on >> > a slower connection. Here are the relevant settings on this relay: >> > >> > RelayBandwidthRate 130 KB >> > RelayBandwidthBurst 340 KB >> > >> > The Pi has a fairly slow CPU, so I'd occasionally get messages about >> log >> > deduplication being too slow or something, but didn't think much of >> it. >> > I finally got the relay up and left it up for over 24 hours. When I >> > woke up this morning it had crashed. Here are the relevant log >> messages >> > - note the huge jump in number of circuits between 22:35 and 04:35 >> > (maybe I got the Stable flag), then the storm of circuit open requests >> > starting at 05:49. Eventually I believe the Pi ran out of memory and >> > killed the tor process. >> > >> > What's very interesting here is that my fast VPS relay with a >> > RelayBandwidthRate over 5x faster is almost never handling much more >> > than 1000 circuits, so why all of a sudden the demand on the Pi that's >> > advertising a lower bandwidth rate? >> > >> > Mar 17 22:35:00.000 [notice] Heartbeat: Tor's uptime is 1 day 0:00 >> > hours, with 26 circuits open. I've sent 974.13 MB and received 969.92 >> > MB. >> > Mar 18 04:35:00.000 [notice] Heartbeat: Tor's uptime is 1 day 6:00 >> > hours, with 972 circuits open. I've sent 1.61 GB and received 1.59 GB. >> > Mar 18 05:49:44.000 [warn] Your computer is too slow to handle this >> many >> > circuit creation requests! Please consider using the >> > MaxAdvertisedBandwidth config option or choosing a more restricted >> exit >> > policy. >> > Mar 18 05:49:44.000 [warn] Failed to hand off onionskin. Closing. >> > Mar 18 05:50:44.000 [warn] Your computer is too slow to handle this >> many >> > circuit creation requests! Please consider using the >> > MaxAdvertisedBandwidth config option or choosing a more restricted >> exit >> > policy. [5817 similar message(s) suppressed in last 60 seconds] >> > Mar 18 05:52:30.000 [warn] Your system clock just jumped 101 seconds >> > forward; assuming established circuits no longer work. >> > Mar 18 05:53:51.000 [warn] Your computer is too slow to handle this >> many >> > circuit creation requests! Please consider using the >> > MaxAdvertisedBandwidth config option or choosing a more restricted >> exit >> > policy. [1055 similar message(s) suppressed in last 60 seconds] >> > Mar 18 05:55:14.000 [warn] Your computer is too slow to handle this >> many >> > circuit creation requests! Please consider using the >> > MaxAdvertisedBandwidth config option or choosing a more restricted >> exit >> > policy. [329 similar message(s) suppressed in last 60 seconds] >> > >> > I'd like to figure out just how much the Raspberry Pi is capable of, >> > because it could be a cheap way to build out the relay network by >> people >> > who want to donate bandwidth - but of course it needs to be stable, >> and >> > something about my setup is not. >> > >> > Also: >> > >> > Mar 16 20:55:33.000 [notice] No AES engine found; using AES_* >> functions. >> > >> > I have no idea if the Broadcom BCM2835 SoC (ARM1176JZF-S CPU) in the >> Pi >> > has any AES capability, but it'd be great to find out. > > >

4 3

Running Obfsproxy on a Raspberry Pi
by Richard Budd 02 Jun '13

02 Jun '13

Don't know how common this is but I've had a Pi running for 35 days 6 hours (so far). With over 80GB transferred on my half assed comcast cable connection. Not bad for $25 a credit card sized board sitting in a cardboard box in my broom closet. I think I'm going to give one to everyone of my family members preloaded with Tor. Plug it into their cable router and let it run. That would be another 4 bridges added to the total. If we could get another 100 people to do this it it might be a good way to add capacity with very little cost or power use. BTW, I have nothing to do with the Pi foundation, I just like playing around with them. Richard

2 5