Why can't I see more traffic? (is my banana too weak?)
Hello, I have been running a middle relay for one year in a few days. I would expect the relay to push more traffic than it does. https://atlas.torproject.org/#details/875F74A3DC14737BECA06F8B500022154D1A29... What is the reason for it not handling more traffic? The Internet connection is 250 Mbit/s downstream and 100 Mbit/s upstream. There is nothing else using this connection. The "server" is a Banana Pro: http://www.lemaker.org/product-bananapro-specification.html The Banana is running Bananian Linux: https://www.bananian.org/ I know that the Banana Pro is not that powerful, but still I think that it is able to handle more traffic than it seems to do. According to top the CPU hovers around 10-20% most of the time. I have also been running nload and tload in order to see the load over a longer time. Does anyone have an idea of why I'm not seeing more traffic? Could it be that it is due to the quite slow hardware, even though I know that it is able to push more traffic? What kind of relay speeds can you expect from a Raspberry Pi for example? Thanks.
Could it be that it is due to the quite slow hardware, even though I know that it is able to push more traffic?
Yep, surely. You currently push 3Mbps of traffic, which is correct for this kind of hardware. All "cheap" hardware (raspi, banana, olimex, pine…) suffer of the fact they don’t have crypto hardware acceleration and do software encryption. And so is very slow (10-100× factor) even compared to low end amd64 CPU with AES-NI extension. Generally speaking, the bottleneck of a Tor relay is CPU, not bandwidth. Even high end Intel CPU is not able to deliver more than 300Mbps (one core of 2Ghz Xeon is ~ 100Mbits, standard CPU is ~ 300Mbps, best known Tor relay currently handle 500Mbps). <3, -- Aeris Individual crypto-terrorist group self-radicalized on the digital Internet https://imirhil.fr/ Protect your privacy, encrypt your communications GPG : EFB74277 ECE4E222 OTR : 5769616D 2D3DAC72 https://café-vie-privée.fr/
Could it be that it is due to the quite slow hardware, even though I know that it is able to push more traffic?
I notice too your relay got the guard flag very recently. So your relay is currently probably not at full capacity and rather at a low one (relay usage drops at guard flag assignment). You have to wait 60 days from your guard flag to reach this point. See https://blog.torproject.org/blog/lifecycle-of-a-new-relay, you’re currently at the beginning of the 3rd phase. <3, -- Aeris Individual crypto-terrorist group self-radicalized on the digital Internet https://imirhil.fr/ Protect your privacy, encrypt your communications GPG : EFB74277 ECE4E222 OTR : 5769616D 2D3DAC72 https://café-vie-privée.fr/
On Sat, 03 Sep 2016 16:53:25 +0200 Aeris <aeris+tor@imirhil.fr> wrote:
Could it be that it is due to the quite slow hardware, even though I know that it is able to push more traffic?
Yep, surely.
You currently push 3Mbps of traffic, which is correct for this kind of hardware. All "cheap" hardware (raspi, banana, olimex, pine…) suffer of the fact they don’t have crypto hardware acceleration and do software encryption. And so is very slow (10-100× factor) even compared to low end amd64 CPU with AES-NI extension.
According to 'openssl speed aes-128-cbc' the Allwinner A20 CPU in Banana Pro is capable of about 25 MBytes/sec in AES performance. While that won't translate 1:1 into Tor performance, as Farid noted in his case the CPU isn't being a bottleneck, with only 10-20% CPU load observed. @Farid,
According to top the CPU hovers around 10-20% most of the time.
I wonder is it 20% across both cores, which could be 40% of one core (since Tor is not multithreaded enough), and at least somewhat closer to not being practically idle. Can you launch 'top' and press '1' there to check? Also seems unclear why it didn't get the guard flag for so long, does your public IP address change from time to time? Or do you turn the relay off and on for whatever reason. -- With respect, Roman
Looking at Atlas your relay advertises 2.45 MB/s which is quite low for a 100Mbit connection: 2.45 MByte x 8 = 19.6 MbitWhat value do you have in your torrc? For a 100mbit connection it should be at least: BandwidthRate 12 MB ------ Originalmeddelande ------ Från: "Roman Mamedov" <rm@romanrm.net> Till: "Aeris" <aeris+tor@imirhil.fr> Kopia: tor-relays@lists.torproject.org; "Farid Joubbi" <joubbi@kth.se> Skickat: 2016-09-03 17:14:08 Ämne: Re: [tor-relays] Why can't I see more traffic? (is my banana too weak?)
On Sat, 03 Sep 2016 16:53:25 +0200 Aeris <aeris+tor@imirhil.fr> wrote:
Could it be that it is due to the quite slow hardware, even though I know that it is able to push more traffic?
Yep, surely.
You currently push 3Mbps of traffic, which is correct for this kind of hardware. All "cheap" hardware (raspi, banana, olimex, pine…) suffer of the fact they don’t have crypto hardware acceleration and do software encryption. And so is very slow (10-100× factor) even compared to low end amd64 CPU with AES-NI extension.
According to 'openssl speed aes-128-cbc' the Allwinner A20 CPU in Banana Pro is capable of about 25 MBytes/sec in AES performance. While that won't translate 1:1 into Tor performance, as Farid noted in his case the CPU isn't being a bottleneck, with only 10-20% CPU load observed.
@Farid,
According to top the CPU hovers around 10-20% most of the time.
I wonder is it 20% across both cores, which could be 40% of one core (since Tor is not multithreaded enough), and at least somewhat closer to not being practically idle. Can you launch 'top' and press '1' there to check?
Also seems unclear why it didn't get the guard flag for so long, does your public IP address change from time to time? Or do you turn the relay off and on for whatever reason.
-- With respect, Roman
According to 'openssl speed aes-128-cbc' the Allwinner A20 CPU in Banana Pro is capable of about 25 MBytes/sec in AES performance. While that won't translate 1:1 into Tor performance, as Farid noted in his case the CPU isn't being a bottleneck, with only 10-20% CPU load observed.
I don’t understand very well this fact, but CPU can be the bottleneck even if load or CPU usage not at full capacity. One of my Tor guard relay have the same CPU behaviour (see screenshot enclosed). 2 instances at 50-60% CPU usage (as reported by htop), load around 0.70-0.80 (4 cores), RAM at 20% (400MB/2GB) but bandwidth not saturated (20Mbps only on a 200Mbps line). Perhaps because of the multiple changes of context (AES crypto, Tor software logic, network IO…) and so a lot of wait/IRQ (as visible on the screen) and not a fully used CPU.
Also seems unclear why it didn't get the guard flag for so long, does your public IP address change from time to time? Or do you turn the relay off and on for whatever reason.
Perhaps a low bandwidth ? Babylonian seems to be on the lower part of guard relay (2146/2313), possible it hadn’t enough bandwith before end of august to get guard flags ? Only the 25% fastest relays can get the guard flag. Today it’s around 2.5 MBps advertised / 1MBps measured. Babylonian is just at the limit (2.45MBps advertised, 600kBps measured). <3, -- Aeris Individual crypto-terrorist group self-radicalized on the digital Internet https://imirhil.fr/ Protect your privacy, encrypt your communications GPG : EFB74277 ECE4E222 OTR : 5769616D 2D3DAC72 https://café-vie-privée.fr/
The USB connector on the power cord is a bit flaky. Babylonian had not received any power for almost three days without me noticing. Does anyone have an explanation why it pushes more traffic now, directly after being offline for three days? I would have believed that it would receive less traffic now since it has a track record of being less stable. According to Atlas it is pushing about twice as much now after being out for a while. https://atlas.torproject.org/#details/875F74A3DC14737BECA06F8B500022154D1A29... -- Farid ________________________________________ From: Aeris <aeris+tor@imirhil.fr> Sent: 03 September 2016 18:06 To: Roman Mamedov Cc: Farid Joubbi; tor-relays@lists.torproject.org Subject: Re: [tor-relays] Why can't I see more traffic? (is my banana too weak?)
According to 'openssl speed aes-128-cbc' the Allwinner A20 CPU in Banana Pro is capable of about 25 MBytes/sec in AES performance. While that won't translate 1:1 into Tor performance, as Farid noted in his case the CPU isn't being a bottleneck, with only 10-20% CPU load observed.
I don’t understand very well this fact, but CPU can be the bottleneck even if load or CPU usage not at full capacity. One of my Tor guard relay have the same CPU behaviour (see screenshot enclosed). 2 instances at 50-60% CPU usage (as reported by htop), load around 0.70-0.80 (4 cores), RAM at 20% (400MB/2GB) but bandwidth not saturated (20Mbps only on a 200Mbps line). Perhaps because of the multiple changes of context (AES crypto, Tor software logic, network IO…) and so a lot of wait/IRQ (as visible on the screen) and not a fully used CPU.
Also seems unclear why it didn't get the guard flag for so long, does your public IP address change from time to time? Or do you turn the relay off and on for whatever reason.
Perhaps a low bandwidth ? Babylonian seems to be on the lower part of guard relay (2146/2313), possible it hadn’t enough bandwith before end of august to get guard flags ? Only the 25% fastest relays can get the guard flag. Today it’s around 2.5 MBps advertised / 1MBps measured. Babylonian is just at the limit (2.45MBps advertised, 600kBps measured). <3, -- Aeris Individual crypto-terrorist group self-radicalized on the digital Internet https://imirhil.fr/ Protect your privacy, encrypt your communications GPG : EFB74277 ECE4E222 OTR : 5769616D 2D3DAC72 https://café-vie-privée.fr/
It seems as if Cpu1 is almost idle most of the time. Cpu0 is somewhere between 5 and 20. This is a rather high snapshot: %Cpu0 : 17.2 us, 2.0 sy, 0.0 ni, 79.5 id, 0.0 wa, 0.0 hi, 1.3 si, 0.0 st %Cpu1 : 2.4 us, 0.3 sy, 0.0 ni, 97.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st I have had the guard flag before. Maybe I lost it since I rebooted twice and changed ISP a bit more than a week ago. The IP address does not change unless I reboot or release the current address manually I used to have 250Mbit/s downstream and 10Mbit/s up with my old ISP. How does the algorithm that checks the bandwidth work? Does anyone else reading this run a Banana Pro, Raspberry or similar hardware with better results than me? Regards, Farid ________________________________________ From: Roman Mamedov <rm@romanrm.net> Sent: 03 September 2016 17:14 To: Aeris Cc: Farid Joubbi; tor-relays@lists.torproject.org Subject: Re: [tor-relays] Why can't I see more traffic? (is my banana too weak?) On Sat, 03 Sep 2016 16:53:25 +0200 Aeris <aeris+tor@imirhil.fr> wrote:
Could it be that it is due to the quite slow hardware, even though I know that it is able to push more traffic?
Yep, surely.
You currently push 3Mbps of traffic, which is correct for this kind of hardware. All "cheap" hardware (raspi, banana, olimex, pine…) suffer of the fact they don’t have crypto hardware acceleration and do software encryption. And so is very slow (10-100× factor) even compared to low end amd64 CPU with AES-NI extension.
According to 'openssl speed aes-128-cbc' the Allwinner A20 CPU in Banana Pro is capable of about 25 MBytes/sec in AES performance. While that won't translate 1:1 into Tor performance, as Farid noted in his case the CPU isn't being a bottleneck, with only 10-20% CPU load observed. @Farid,
According to top the CPU hovers around 10-20% most of the time.
I wonder is it 20% across both cores, which could be 40% of one core (since Tor is not multithreaded enough), and at least somewhat closer to not being practically idle. Can you launch 'top' and press '1' there to check? Also seems unclear why it didn't get the guard flag for so long, does your public IP address change from time to time? Or do you turn the relay off and on for whatever reason. -- With respect, Roman
On 4 Sep 2016, at 04:35, Farid Joubbi <joubbi@kth.se> wrote:
It seems as if Cpu1 is almost idle most of the time. Cpu0 is somewhere between 5 and 20. This is a rather high snapshot:
%Cpu0 : 17.2 us, 2.0 sy, 0.0 ni, 79.5 id, 0.0 wa, 0.0 hi, 1.3 si, 0.0 st %Cpu1 : 2.4 us, 0.3 sy, 0.0 ni, 97.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
I have had the guard flag before. Maybe I lost it since I rebooted twice and changed ISP a bit more than a week ago.
The IP address does not change unless I reboot or release the current address manually
I used to have 250Mbit/s downstream and 10Mbit/s up with my old ISP.
How does the algorithm that checks the bandwidth work?
Your relay reports a bandwidth based on the amount of traffic it has sustained in any 10 second period over the past day. You can also set a maximum advertised bandwidth on your relay. (Don't do this if you're trying to pick up more traffic.) Five bandwidth authorities measure each relay each week, and report how fast it is. Each of these factors can restrict the amount of bandwidth that the network assigns to your relay. Here's one way of testing what your relay is capable of: Run a Tor client as close to your relay as possible: tor DataDirectory /tmp/tor.$$ SOCKSPort [IPv4:]10000 EntryNodes your-relay-name Then download a large file using port 10000 as a socks proxy. That will give you some idea of how much traffic your relay can sustain, but it's worth noting that each client is limited to about 1 Mbps (I think - I can't find the manual page entry). Tim
Does anyone else reading this run a Banana Pro, Raspberry or similar hardware with better results than me?
Regards, Farid
________________________________________ From: Roman Mamedov <rm@romanrm.net> Sent: 03 September 2016 17:14 To: Aeris Cc: Farid Joubbi; tor-relays@lists.torproject.org Subject: Re: [tor-relays] Why can't I see more traffic? (is my banana too weak?)
On Sat, 03 Sep 2016 16:53:25 +0200 Aeris <aeris+tor@imirhil.fr> wrote:
Could it be that it is due to the quite slow hardware, even though I know that it is able to push more traffic?
Yep, surely.
You currently push 3Mbps of traffic, which is correct for this kind of hardware. All "cheap" hardware (raspi, banana, olimex, pine…) suffer of the fact they don’t have crypto hardware acceleration and do software encryption. And so is very slow (10-100× factor) even compared to low end amd64 CPU with AES-NI extension.
According to 'openssl speed aes-128-cbc' the Allwinner A20 CPU in Banana Pro is capable of about 25 MBytes/sec in AES performance. While that won't translate 1:1 into Tor performance, as Farid noted in his case the CPU isn't being a bottleneck, with only 10-20% CPU load observed.
@Farid,
According to top the CPU hovers around 10-20% most of the time.
I wonder is it 20% across both cores, which could be 40% of one core (since Tor is not multithreaded enough), and at least somewhat closer to not being practically idle. Can you launch 'top' and press '1' there to check?
Also seems unclear why it didn't get the guard flag for so long, does your public IP address change from time to time? Or do you turn the relay off and on for whatever reason.
-- With respect, Roman _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org
participants (5)
-
Aeris -
Farid Joubbi -
Logforme -
Roman Mamedov -
teor