[tor-relays] Raspberry Pi Relay Node Performance and future Plans on Documentation and more
Michael Berlin
mail at mberlin.org
Sat Aug 3 15:05:05 UTC 2013
Hi Gordon and Matthias,
I've split your discussion from the original thread "Running exit-node in
Germany" and created a new one.
I fully agree with you that the Raspberry Pi is the perfect device to let
others run a Tor Relay Node very easily. What follows is a long mail about
my experiences and more thoughts about the Pi as relay.
On Thu, Aug 1, 2013 at 5:29 PM, Gordon Morehouse <gordon at morehouse.me>wrote:
> Matthias Redies:
> > Ok that is good to know. Right know I will probably run it on 1-1.5 Mbps
> > and later on 3-4 Mbps. What is the maximum your raspberry is capable to
> > do? Please let me know if you publish your tutorial.
>
> I had it pushing about 1.5Mbps and crashing only about once a week
> before I started having TCP connect floods and had to take it offline
> until I could pay attention for a while. I'm still tuning it. It
> crashed much, much more often before some basic tuning, though.
>
I'm running a relay node on a Raspberry Pi at my VDSL home connection for
several weeks now (provider "Deutsche Telekom", connection is 50mbps down,
10 up) and I didn't run into such issues so far.
There're already several howtos out there which describe how to setup a
relay node on the Raspberry Pi. I used this German one as starting point:
http://blog.weezerle.de/2013/01/11/raspberry-pi-als-tor-server/
My current configuration has only the following two non-default options:
DisableDebuggerAttachment 0
AvoidDiskWrites 1
(Plus, I run it as relay only by disabling the socks proxy with "SocksPort
0".)
Here's my node:
https://atlas.torproject.org/#details/5AB2F49A1B79148E3A8F52808A00451DA00841B1
http://globe.rndm.de/#/relay/5AB2F49A1B79148E3A8F52808A00451DA00841B1
As you can see, it forwards on average between 2-4mbps (256-512 kB/sec) and
there're peaks as high as 5.5mbps (~700 kB/s). The throughput varies
depending on the day of week and time of day and therefore I believe the
low average isn't the Raspberry's fault. I guess, if I set a higher
advertised value, then I would see a more constant throughput closer to the
peak value. Currently, it's set to 10mbps:
RelayBandwidthRate 1024 KB
RelayBandwidthBurst 1024 KB
(Note that the Pi is definitely not able to forward 10mbps of Tor traffic,
but it would be great to max it out 24/7.)
The logged throughput is also consistent with what I saw with the console
traffic monitor "nload" (suggested command line options for nicer units and
less refreshes: nload -u K -U G -t 3000). I guess, I saw even higher peaks
there. Monitoring everything with "arm" is also nice but far too CPU
intensive.
More information about my setup:
- I have the B revision with 512 MB memory
- running stock Raspbian
- installed "tor" package 0.2.3.25-1 from the Raspbian repository
- CPU is overclocked using "raspi-config" to 950 MHz
Before I overclocked the Pi, I saw a similar average throughput.
When you search for "rasp" at Tor Atlas or Globe (e.g.
http://globe.rndm.de/#/search/query=rasp) , you can see that there're
already more than 40 devices running. My node performs quite well compared
to others and I would argue that it's in the top 5 among all Raspberry Pi
nodes so far.
To track down possible CPU problems, we should also compare the openssl
benchmark results. The Raspberry Pi wiki has some reference values:
http://elinux.org/RPi_Performance#Results_3
I've attached the summary output of "openssl speed" for my overclocked Pi
(950 MHz) to this mail. As you can see, the numbers are higher than the
reference one (I guess probably due to overclocking and different OpenSSL
version).
I've also attached the log of notices of the last 4 days (unfortunately no
longer logs available). Within four days there was only one "Your computer
is too slow to handle this many circuit creation requests!" warning and one
"Failed to hand off onionskin.". In general I would say, I haven't seen any
serious issues so far.
>
> And my plan is to publish my results to the entire list, because at $35,
> Raspberry Pis can make *great* relays for slower home broadband, but
> they need a little tender loving care first. :)
>
>
I totally agree with you. Recently, there has been quite some buzz about
the OnionPi howto (http://learn.adafruit.com/onion-pi/overview) but its
main goal wasn't providing a long-running relay node. In my opinion, this
should get addressed separately and made as simple as possible for novices.
My guess is that many people want to contribute bandwidth, but they do not
want to deal with Linux specifics (and definitely do not want to get into
any trouble with the police). They need a plug-and-forget solution: attach
the Pi to the router and leave it running in the closet for months. If it
has to be restarted, Tor Weather can send them an email.
What do you think about documenting the setup of the relay in the Tor wiki?
https://trac.torproject.org/projects/tor/wiki
I'm not very familiar with the Tor Project yet, but to me it looks like the
wiki will be the best place. Other people can contribute as well and it
will be visible enough. It would be good if there's an always up-to-date
wiki page which could become the reference for new users.
The idea of the Raspberry Pi as relay node could be even further expanded:
- Matthias already mentioned that there should be a specific image for it.
I agree!
- New users would prefer a simple webinterface to configure the relay
(basically a Vidalia as webinterface :-)
- Such a webinterface should also show statistics e.g., "Your node has
forwarded 1 TB in the last 4 weeks." This will show people that their
contribution made a difference and it will make them happy and more
confident about it. Eventually, they'll convince others to run a relay as
well.
I won't have the time to realize these ideas, but maybe others want to jump
in. Further work and testing could be coordinated on this mailing list?
I hope to have something up in a week or two, I need to watch it for a
> while and continue to tweak, and maybe develop a solution for the TCP
> storms that can bring down a lot of consumer routers, before publishing
> for all.
>
I run a regular Linux PC as router which forwards all the traffic of the
Pi. The machine has more than enough CPU power and therefore I haven't seen
any issues there. Since the two months that I run the relay node on the Pi,
my Linux PC router has frozen twice (which didn't happen before). But I'm
not sure that it can be related to the forwarding of the relay node traffic.
For a broader adaption it's definitely necessary to look into the behavior
of the most popular routers e.g., the ones given out by the major
providers. These things could be documented on the Tor Wiki as well.
Best regards,
Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20130803/4633b363/attachment-0001.html>
-------------- next part --------------
OpenSSL 1.0.1e 11 Feb 2013
built on: Sun Mar 24 12:44:00 UTC 2013
options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) blowfish(ptr)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wa,--noexecstack -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
md2 0.00 0.00 0.00 0.00 0.00
mdc2 0.00 0.00 0.00 0.00 0.00
md4 2600.95k 9691.50k 30354.32k 64472.75k 96227.23k
md5 1909.94k 7068.47k 22301.54k 48309.51k 73045.79k
hmac(md5) 3494.14k 11837.23k 32889.41k 59205.69k 76712.62k
sha1 2125.37k 7237.09k 19115.72k 32521.76k 41017.92k
rmd160 1601.52k 5205.79k 13228.20k 21736.54k 26989.76k
rc4 47299.41k 54870.05k 57123.19k 57508.52k 57981.77k
des cbc 8557.03k 9046.09k 9171.39k 9199.22k 9189.29k
des ede3 3121.73k 3230.74k 3268.98k 3265.16k 3260.36k
idea cbc 0.00 0.00 0.00 0.00 0.00
seed cbc 9102.76k 11680.00k 12610.30k 12982.46k 13038.48k
rc2 cbc 8312.11k 8763.62k 8817.49k 8903.99k 8850.09k
rc5-32/12 cbc 0.00 0.00 0.00 0.00 0.00
blowfish cbc 16177.60k 17847.18k 18252.20k 18493.51k 18395.02k
cast cbc 13416.19k 15583.59k 16299.15k 16472.02k 16430.58k
aes-128 cbc 17918.61k 20484.20k 21281.82k 21499.00k 21562.22k
aes-192 cbc 15864.90k 17859.27k 18461.02k 18613.51k 18731.64k
aes-256 cbc 14200.77k 15928.39k 16346.46k 16412.43k 16577.08k
camellia-128 cbc 9985.35k 13278.78k 14391.91k 14761.70k 14805.67k
camellia-192 cbc 8591.10k 10699.11k 11420.34k 11568.38k 11744.62k
camellia-256 cbc 8608.10k 10741.85k 11419.23k 11601.68k 11618.99k
sha256 4872.42k 11548.39k 20366.52k 25060.60k 26904.83k
sha512 1484.95k 5915.80k 8530.88k 11663.67k 13030.74k
whirlpool 536.90k 1064.52k 1742.51k 2122.66k 2250.07k
aes-128 ige 15971.43k 18931.14k 19790.00k 20034.44k 19496.41k
aes-192 ige 14274.86k 16714.82k 17315.02k 17476.38k 17085.39k
aes-256 ige 13018.78k 14776.81k 15436.80k 15485.27k 15259.66k
ghash 21814.75k 23936.17k 24688.67k 24773.97k 24955.36k
sign verify sign/s verify/s
rsa 512 bits 0.001635s 0.000167s 611.5 5981.2
rsa 1024 bits 0.008259s 0.000472s 121.1 2117.3
rsa 2048 bits 0.053641s 0.001664s 18.6 601.0
rsa 4096 bits 0.390385s 0.006317s 2.6 158.3
sign verify sign/s verify/s
dsa 512 bits 0.001649s 0.001692s 606.6 590.9
dsa 1024 bits 0.004628s 0.005217s 216.1 191.7
dsa 2048 bits 0.016175s 0.018778s 61.8 53.3
sign verify sign/s verify/s
160 bit ecdsa (secp160r1) 0.0013s 0.0047s 743.7 211.5
192 bit ecdsa (nistp192) 0.0016s 0.0059s 619.8 168.3
224 bit ecdsa (nistp224) 0.0020s 0.0077s 498.4 129.4
256 bit ecdsa (nistp256) 0.0025s 0.0103s 404.9 96.9
384 bit ecdsa (nistp384) 0.0049s 0.0231s 204.6 43.3
521 bit ecdsa (nistp521) 0.0097s 0.0523s 103.6 19.1
163 bit ecdsa (nistk163) 0.0036s 0.0122s 280.7 82.2
233 bit ecdsa (nistk233) 0.0070s 0.0219s 143.5 45.6
283 bit ecdsa (nistk283) 0.0104s 0.0403s 96.6 24.8
409 bit ecdsa (nistk409) 0.0257s 0.0884s 38.9 11.3
571 bit ecdsa (nistk571) 0.0619s 0.2076s 16.1 4.8
163 bit ecdsa (nistb163) 0.0035s 0.0132s 289.1 75.6
233 bit ecdsa (nistb233) 0.0068s 0.0241s 146.4 41.6
283 bit ecdsa (nistb283) 0.0103s 0.0450s 96.7 22.2
409 bit ecdsa (nistb409) 0.0258s 0.1017s 38.8 9.8
571 bit ecdsa (nistb571) 0.0620s 0.2381s 16.1 4.2
op op/s
160 bit ecdh (secp160r1) 0.0036s 280.8
192 bit ecdh (nistp192) 0.0047s 212.4
224 bit ecdh (nistp224) 0.0062s 160.8
256 bit ecdh (nistp256) 0.0081s 123.0
384 bit ecdh (nistp384) 0.0189s 52.8
521 bit ecdh (nistp521) 0.0418s 23.9
163 bit ecdh (nistk163) 0.0059s 169.2
233 bit ecdh (nistk233) 0.0107s 93.5
283 bit ecdh (nistk283) 0.0199s 50.3
409 bit ecdh (nistk409) 0.0441s 22.7
571 bit ecdh (nistk571) 0.1030s 9.7
163 bit ecdh (nistb163) 0.0064s 157.1
233 bit ecdh (nistb233) 0.0118s 84.4
283 bit ecdh (nistb283) 0.0221s 45.2
409 bit ecdh (nistb409) 0.0501s 20.0
571 bit ecdh (nistb571) 0.1185s 8.4
-------------- next part --------------
# zcat /var/log/tor/notices.log*.gz | sort | uniq
Jul 27 06:25:07.000 [notice] Tor 0.2.3.25 (git-3fed5eb096d2d187) opening new log file.
Jul 27 07:15:27.000 [notice] Heartbeat: Tor's uptime is 7 days 15:17 hours, with 638 circuits open. I've sent 407.08 GB and received 399.69 GB.
Jul 27 13:15:27.000 [notice] Heartbeat: Tor's uptime is 7 days 21:17 hours, with 1160 circuits open. I've sent 415.67 GB and received 408.17 GB.
Jul 27 19:15:27.000 [notice] Heartbeat: Tor's uptime is 8 days 3:17 hours, with 571 circuits open. I've sent 423.10 GB and received 415.48 GB.
Jul 28 01:15:27.000 [notice] Heartbeat: Tor's uptime is 8 days 9:17 hours, with 401 circuits open. I've sent 432.89 GB and received 425.16 GB.
Jul 28 06:25:09.000 [notice] Read configuration file "/etc/tor/torrc".
Jul 28 06:25:09.000 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 28 06:25:09.000 [notice] Received reload signal (hup). Reloading config and resetting internal state.
Jul 28 06:25:09.000 [notice] Tor 0.2.3.25 (git-3fed5eb096d2d187) opening new log file.
Jul 28 07:15:27.000 [notice] Heartbeat: Tor's uptime is 8 days 15:17 hours, with 428 circuits open. I've sent 439.80 GB and received 432.00 GB.
Jul 28 13:15:27.000 [notice] Heartbeat: Tor's uptime is 8 days 21:17 hours, with 794 circuits open. I've sent 445.19 GB and received 437.28 GB.
Jul 28 19:15:27.000 [notice] Heartbeat: Tor's uptime is 9 days 3:17 hours, with 1079 circuits open. I've sent 452.40 GB and received 444.36 GB.
Jul 29 01:15:27.000 [notice] Heartbeat: Tor's uptime is 9 days 9:17 hours, with 664 circuits open. I've sent 461.49 GB and received 453.30 GB.
Jul 29 06:25:08.000 [notice] Read configuration file "/etc/tor/torrc".
Jul 29 06:25:08.000 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 29 06:25:08.000 [notice] Received reload signal (hup). Reloading config and resetting internal state.
Jul 29 06:25:08.000 [notice] Tor 0.2.3.25 (git-3fed5eb096d2d187) opening new log file.
Jul 29 07:15:27.000 [notice] Heartbeat: Tor's uptime is 9 days 15:17 hours, with 779 circuits open. I've sent 468.98 GB and received 460.71 GB.
Jul 29 13:15:27.000 [notice] Heartbeat: Tor's uptime is 9 days 21:17 hours, with 1549 circuits open. I've sent 475.90 GB and received 467.46 GB.
Jul 29 19:15:27.000 [notice] Heartbeat: Tor's uptime is 10 days 3:17 hours, with 1344 circuits open. I've sent 484.08 GB and received 475.44 GB.
Jul 29 19:21:37.000 [warn] Failed to hand off onionskin. Closing. [33164 similar message(s) suppressed in last 21600 seconds]
Jul 29 19:21:37.000 [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [68 similar message(s) suppressed in last 60 seconds]
Jul 30 01:15:27.000 [notice] Heartbeat: Tor's uptime is 10 days 9:17 hours, with 892 circuits open. I've sent 493.42 GB and received 484.64 GB.
Jul 30 06:25:09.000 [notice] Read configuration file "/etc/tor/torrc".
Jul 30 06:25:09.000 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 30 06:25:09.000 [notice] Received reload signal (hup). Reloading config and resetting internal state.
Jul 30 06:25:09.000 [notice] Tor 0.2.3.25 (git-3fed5eb096d2d187) opening new log file.
Jul 30 07:15:27.000 [notice] Heartbeat: Tor's uptime is 10 days 15:17 hours, with 1107 circuits open. I've sent 501.62 GB and received 492.74 GB.
Jul 30 13:15:27.000 [notice] Heartbeat: Tor's uptime is 10 days 21:17 hours, with 1225 circuits open. I've sent 510.17 GB and received 501.15 GB.
Jul 30 19:15:27.000 [notice] Heartbeat: Tor's uptime is 11 days 3:17 hours, with 1000 circuits open. I've sent 518.06 GB and received 508.89 GB.
Jul 31 01:15:27.000 [notice] Heartbeat: Tor's uptime is 11 days 9:17 hours, with 594 circuits open. I've sent 525.04 GB and received 515.74 GB.
Jul 31 06:25:07.000 [notice] Read configuration file "/etc/tor/torrc".
Jul 31 06:25:07.000 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 31 06:25:07.000 [notice] Received reload signal (hup). Reloading config and resetting internal state.
More information about the tor-relays
mailing list