300mbps FreeBSD Tor relay on HPE MicroServer Gen10 (AMD X3421)
Hi tor-relays@, I have a Tor middle relay NeelTorRelay2 hosted on a 50 megabit symmetrical Verizon FiOS (FTTH/GPON) connection. The server used is a HPE MicroServer Gen10 (AMD X3421 quad-core version, 8GB DDR4 RAM). This relay can be seen here: https://metrics.torproject.org/rs.html#details/D5B8C38539C509380767D4DE20DE8... My relay runs FreeBSD 11.2 and Tor runs in a "jail". I am using AESNI and Tor is configured to use OpenSSL cryptodev. Here's the situation: I will be moving apartments in a few days, and Verizon is upgrading my broadband speed to 300 megabits symmetrical. I plan to use this extra bandwidth for Tor. Right now, I set my RelayBandwidthRate to my line speed (yes really!), and plan to increase this setting according to my new speed. I know that Tor is not optimized for multicore CPUs, and that's the reason why I am posting here. My question is that can Tor work on the HPE MicroServer Gen10 with the AMD X3421 (or one with a similar computer of any brand with a similar performance CPU, whether desktop or server, Intel or AMD) with all 300 megabits to a single instance or would I need two instances (each at 150 megabits each)? Looking at my top usage, I average at about 20-30% CPU usage on my 50 megabit relay. Also keep in mind that: * I am using my own router instead of Verizon's and I plan to keep doing so * I want to keep using FreeBSD on my server and do not want to run Linux * I would prefer to have a single instance, but can use multiple if I have to * When I move, I will upgrade my server to FreeBSD 12.0 * My server supports hardware accelerated AES and SHA. I am using this on FreeBSD with the aesni kernel module and Tor with "HardwareAccel 1" and "AccelName cryptodev" Thank You, Neel Chauhan === https://www.neelc.org/
On Fri, 28 Dec 2018 14:13:03 +0000 "Neel Chauhan" <neel@neelc.org> wrote:
Here's the situation: I will be moving apartments in a few days, and Verizon is upgrading my broadband speed to 300 megabits symmetrical. I plan to use this extra bandwidth for Tor. Right now, I set my RelayBandwidthRate to my line speed (yes really!), and plan to increase this setting according to my new speed.
You could just remove that line altogether. Without it, Tor will use as much as it can, not wasting time on pointless bandwidth housekeeping. That line could be useful to limit bandwidth in case you notice Tor interfers with your normal Internet browsing, but since you just set it to 100% of line speed currently, it seems like you're not using it for that.
I know that Tor is not optimized for multicore CPUs, and that's the reason why I am posting here.
My question is that can Tor work on the HPE MicroServer Gen10 with the AMD X3421 (or one with a similar computer of any brand with a similar performance CPU, whether desktop or server, Intel or AMD) with all 300 megabits to a single instance or would I need two instances (each at 150 megabits each)? Looking at my top usage, I average at about 20-30% CPU usage on my 50 megabit relay.
It is hard to tell, but that shouldn't be a very important question, just run one for a while, see if it constantly bumps into 100% CPU, if it does, add a 2nd one. The CPU is a bit peculiar, the base frequency is 2.1 Ghz, but it turboes up to a whopping 3.4 Ghz. One could imagine it does that only as long as not all of its cores are utilized, so maybe adding a second instance will be somewhat detrimental to overall performance. On the other hand, if you want to use your network connection to its fullest, then running two instances is advisable, I'd say one instance will use at most 200-250 Mbit of your 300, but with two you can actually get to 2x140 or so. But of course the former case is actually preferable if the connection is also used for other tasks aside from Tor. -- With respect, Roman
Roman Mamedov:
On Fri, 28 Dec 2018 14:13:03 +0000 "Neel Chauhan" <neel@neelc.org> wrote:
Here's the situation: I will be moving apartments in a few days, and Verizon is upgrading my broadband speed to 300 megabits symmetrical. I plan to use this extra bandwidth for Tor. Right now, I set my RelayBandwidthRate to my line speed (yes really!), and plan to increase this setting according to my new speed.
You could just remove that line altogether. Without it, Tor will use as much as it can, not wasting time on pointless bandwidth housekeeping.
That line could be useful to limit bandwidth in case you notice Tor interfers with your normal Internet browsing, but since you just set it to 100% of line speed currently, it seems like you're not using it for that.
I know that Tor is not optimized for multicore CPUs, and that's the reason why I am posting here.
My question is that can Tor work on the HPE MicroServer Gen10 with the AMD X3421 (or one with a similar computer of any brand with a similar performance CPU, whether desktop or server, Intel or AMD) with all 300 megabits to a single instance or would I need two instances (each at 150 megabits each)? Looking at my top usage, I average at about 20-30% CPU usage on my 50 megabit relay.
It is hard to tell, but that shouldn't be a very important question, just run one for a while, see if it constantly bumps into 100% CPU, if it does, add a 2nd one.
The CPU is a bit peculiar, the base frequency is 2.1 Ghz, but it turboes up to a whopping 3.4 Ghz. One could imagine it does that only as long as not all of its cores are utilized, so maybe adding a second instance will be somewhat detrimental to overall performance.
On the other hand, if you want to use your network connection to its fullest, then running two instances is advisable, I'd say one instance will use at most 200-250 Mbit of your 300, but with two you can actually get to 2x140 or so. But of course the former case is actually preferable if the connection is also used for other tasks aside from Tor.
Neel: At some point, I want to get a few network-heavy FreeBSD involved in optimizing Tor on FreeBSD. It should not take a lot to do, since the networking stack is optimized out of the box, but my FreeBSD nodes never hit much more than 10mbps. One of those devs lives close to both you and I :) Keep us in the loop on the relay and any customizations you're doing. g -- 34A6 0A1F F8EF B465 866F F0C5 5D92 1FD1 ECF6 1682
George:
At some point, I want to get a few network-heavy FreeBSD involved in optimizing Tor on FreeBSD. It should not take a lot to do, since the networking stack is optimized out of the box, but my FreeBSD nodes never hit much more than 10mbps.
I doubt you need any particular tuning unless you aim for >500 Mbit/s for a single core -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
Hi George,
At some point, I want to get a few network-heavy FreeBSD involved in optimizing Tor on FreeBSD. It should not take a lot to do, since the networking stack is optimized out of the box, but my FreeBSD nodes never hit much more than 10mbps.
I hope you get to optimize high-bandwidth Tor on FreeBSD as well. I would love to have this as well. I can also help as well. About the slow relays, looking at your company website (http://queair.net/hardware.html), you appear to be a fan of low-power hardware like Alix or ARM boards (RPI, BeagleBone) and believe you run relays on these. I could be wrong, as it could also be your ISP. If the cause is low-power hardware, I'm not against low power development boards, I just feel that for Tor they're more for low-bandwidth relays (e.g. bridges or relays on slower connections).
One of those devs lives close to both you and I :)
Sounds great.
Keep us in the loop on the relay and any customizations you're doing.
OK, I will. When I get to setting up the server, I will post an article to my website (https://www.neelc.org) and a copy of the article here (@tor-relays). Thanks, Neel Chauhan === https://www.neelc.org/
My question is that can Tor work on the HPE MicroServer Gen10 with the AMD X3421 (or one with a similar computer of any brand with a similar performance CPU, whether desktop or server, Intel or AMD) with all 300 megabits to a single instance or would I need two instances (each at 150 megabits each)? Looking at my top usage, I average at about 20-30% CPU usage on my 50 megabit relay.
based on the cpubenchmarks I found for your CPU I estimate that 300 Mbit/s are doable with that CPU on a single core.
* When I move, I will upgrade my server to FreeBSD 12.0
beware of the incompatibility of Tor with OpenSSL 1.1.1a [1] (used by default on FreeBSD 12.0). The workaround is easy: recompile with the older openssl version available via ports [1] https://trac.torproject.org/projects/tor/ticket/28616 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
Hi Neel
My relay runs FreeBSD 11.2 and Tor runs in a "jail".
Jails are perfect for that! I observed the host Freebsd tcp stack is strong enough for more than 500Mbit/s in AND out.
I am using AESNI and Tor is configured to use OpenSSL cryptodev.
Does crypto run? On log info you should find the following entry during start: [info] crypto_openssl_init_engines: Initializing dynamic OpenSSL engine "dynamic" acceleration support. [info] crypto_openssl_init_engines: Loaded dynamic OpenSSL engine "dynamic". After finding this message you can switch to notice and restart.
* I want to keep using FreeBSD on my server and do not want to run Linux
+1
* I would prefer to have a single instance, but can use multiple if I have to
It's BSD, so may-be consider to go for libressl from ports (which does not support the crypto engine). And then use 2 instances per ip. Better for diversity ;)
* My server supports hardware accelerated AES and SHA. I am using this on FreeBSD with the aesni kernel module and Tor with "HardwareAccel 1" and "AccelName cryptodev"
A toorc can look like: RelayBandwidthRate 0 RelayBandwidthBurst 0 HardwareAccel 1 AccelName dynamic Log info file /var/log/tor/info -- Cheers from 35c3 , Felix
Felix:
Hi Neel
My relay runs FreeBSD 11.2 and Tor runs in a "jail".
Jails are perfect for that! I observed the host Freebsd tcp stack is strong enough for more than 500Mbit/s in AND out.
Yes, jails are a perfect fit in many ways. I haven't been a jail user since FreeBSD 7.x or 8.x, but one thing I'd like to do at some point is sort out a bare minimum jail for a Tor node. Not that usual full-base system jail, but something that would look like a chroot from the birds-eye view. I think it should be very doable with EZjail, but I always prefer base tools with shell scripts. I should mention that I'm not a fan of virtualization solutions for many use-cases, but FreeBSD jails aren't about bloat and just adding more lines of code with more bugs. They are a tight solution that can really mitigate compromises when used properly. For those interested, go look up there original usage by phk@ as a web site hosting solution. It was an instance where some Danish www hosting company kept getting their site hacked, so he had a cron job which diff'd the contents of the www-serving jail, and overwrote it if there was change, or something like that. I can't find the actual link but this helps: http://phk.freebsd.dk/sagas/jails.html
I am using AESNI and Tor is configured to use OpenSSL cryptodev.
Does crypto run? On log info you should find the following entry during start:
[info] crypto_openssl_init_engines: Initializing dynamic OpenSSL engine "dynamic" acceleration support. [info] crypto_openssl_init_engines: Loaded dynamic OpenSSL engine "dynamic".
After finding this message you can switch to notice and restart.
* I want to keep using FreeBSD on my server and do not want to run Linux
+1
Addressing the general audience here... I'm a long-time BSD person and have fought long and hard for OS diversity in Tor, but everyone should stick to OSs they are most comfortable with. The only thing I fear more than OS monocultures is anyone running OSs they can't admin systems which are public-facing and providing a vital service. A misconfigured BSD relay doesn't help anyone.
* I would prefer to have a single instance, but can use multiple if I have to
It's BSD, so may-be consider to go for libressl from ports (which does not support the crypto engine). And then use 2 instances per ip. Better for diversity ;)
Yes, !OpenSSL should be considered, and LibreSSL is a good start. I know LibreSSL doesn't support crypto engine, but not sure of the consequences outside of the basics with it.
* My server supports hardware accelerated AES and SHA. I am using this on FreeBSD with the aesni kernel module and Tor with "HardwareAccel 1" and "AccelName cryptodev"
A toorc can look like: RelayBandwidthRate 0 RelayBandwidthBurst 0 HardwareAccel 1 AccelName dynamic Log info file /var/log/tor/info
On that note, a lot of the Tor BSD docs have been migrated to the TPO documentation, and we need to finish migrating the https://wiki.torbsd.org there also. But there continues to be a need for more, plus additional translations. The BSDs have particularly large footprints in some countries that also happen to lack many Tor relays such as Japan and the Balkan countries. The "gateway" drug for most people running anything new is FAQs, how-tos and documentation. A good target might be optimizing BSD relays beyond the obvious. g -- 34A6 0A1F F8EF B465 866F F0C5 5D92 1FD1 ECF6 1682
FreeBSD jails are light, effective, fast, and detailed chroots... not bloated VM / HW / Hyper or emulation instances that eat RAM and CPU.
sort out a bare minimum jail for a Tor node.
minimum = static tor (1 file) + devfs (kernel managed fs)
company kept getting their site hacked, so he had a cron job
Disposable instances of legacy dependencies, many do that ;)
Thank you all for your feedback. I have already finished the moving process and the upgraded relay is already set up. My server now runs FreeBSD 12.0 as a host, but with Tor in a FBSD 11.2 jail. I will upgrade the jail to 12.0 when FreeBSD unbreaks Tor relays on OpenSSL 1.1.1. I am starting with a single instance to see if it handles 300 mbps. If not, I will switch to two 150mbps instances. I sadly am using OpenSSL, but that is so I can use the crypto engine and pre-built packages. Thank You, Neel Chauhan === https://www.neelc.org/ December 28, 2018 9:13 AM, "Neel Chauhan" <neel@neelc.org> wrote:
Hi tor-relays@,
I have a Tor middle relay NeelTorRelay2 hosted on a 50 megabit symmetrical Verizon FiOS (FTTH/GPON) connection. The server used is a HPE MicroServer Gen10 (AMD X3421 quad-core version, 8GB DDR4 RAM). This relay can be seen here:
https://metrics.torproject.org/rs.html#details/D5B8C38539C509380767D4DE20DE8...
My relay runs FreeBSD 11.2 and Tor runs in a "jail". I am using AESNI and Tor is configured to use OpenSSL cryptodev.
Here's the situation: I will be moving apartments in a few days, and Verizon is upgrading my broadband speed to 300 megabits symmetrical. I plan to use this extra bandwidth for Tor. Right now, I set my RelayBandwidthRate to my line speed (yes really!), and plan to increase this setting according to my new speed.
I know that Tor is not optimized for multicore CPUs, and that's the reason why I am posting here.
My question is that can Tor work on the HPE MicroServer Gen10 with the AMD X3421 (or one with a similar computer of any brand with a similar performance CPU, whether desktop or server, Intel or AMD) with all 300 megabits to a single instance or would I need two instances (each at 150 megabits each)? Looking at my top usage, I average at about 20-30% CPU usage on my 50 megabit relay.
Also keep in mind that:
* I am using my own router instead of Verizon's and I plan to keep doing so * I want to keep using FreeBSD on my server and do not want to run Linux * I would prefer to have a single instance, but can use multiple if I have to * When I move, I will upgrade my server to FreeBSD 12.0 * My server supports hardware accelerated AES and SHA. I am using this on FreeBSD with the aesni kernel module and Tor with "HardwareAccel 1" and "AccelName cryptodev"
Thank You,
Neel Chauhan
===
https://www.neelc.org _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
participants (7)
-
Felix -
George -
grarpamp -
Neel Chauhan -
neel＠neelc.org
-
nusenu -
Roman Mamedov