Consensus weight drops continuously 28h after getting guard flag
Hi, approx. 28 hours after getting the guard flag, my relay <https://metrics.torproject.org/rs.html#details/A99B0D9E5FD5BC3C8F2D006EC91F8C7F40E09CB8>'s consensus weight hit a peak and has declined rather rapidly since then (again 28 hours ago). Did I do something wrong? Is it because I restarted the relay 52 hours ago? Is there actually any detailed documentation on how consensus weight is calculated? Also, my bandwidth (as well as my cpu, etc.) is only used to a fraction of its capacity, even though my relay is three weeks old now. The /RelayBandwidthRate/ option does not limit here... Regards, Ilka
https://blog.torproject.org/comment/54651 <https://blog.torproject.org/comment/54651> Everything will be alright :)
On 11. Jan 2019, at 16:44, Ilka Schulz <ilka@schulz.com.de> wrote:
Hi,
approx. 28 hours after getting the guard flag, my relay <https://metrics.torproject.org/rs.html#details/A99B0D9E5FD5BC3C8F2D006EC91F8C7F40E09CB8>'s consensus weight hit a peak and has declined rather rapidly since then (again 28 hours ago).
Did I do something wrong? Is it because I restarted the relay 52 hours ago? Is there actually any detailed documentation on how consensus weight is calculated?
Also, my bandwidth (as well as my cpu, etc.) is only used to a fraction of its capacity, even though my relay is three weeks old now. The RelayBandwidthRate option does not limit here...
Regards, Ilka _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hi,
On 12 Jan 2019, at 01:44, Ilka Schulz <ilka@schulz.com.de> wrote:
Is there actually any detailed documentation on how consensus weight is calculated?
Consensus weight is calculated using a relay's self-reported peak bandwidth usage, and measurements from ~6 bandwidth authorities around the world. The process is slightly different on some of the bandwidth authorities, because we are migrating to a newer system. There is detailed documentation here: 5/6 bandwidth authorities run torflow: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/R... 1/6 bandwidth authorities run sbws: https://github.com/juga0/sbws/blob/master/docs/source/specification.rst#simp... https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt#n656
Also, my bandwidth (as well as my cpu, etc.) is only used to a fraction of its capacity, even though my relay is three weeks old now. The RelayBandwidthRate option does not limit here...
Tor is a connection-oriented, reliable-transport, low-latency network, so it will never use your full bandwidth capacity. If it did, latency would suffer. Here are some initial steps for troubleshooting: https://trac.torproject.org/projects/tor/wiki/doc/MyRelayIsSlow T
On Jan 12, 2019, at 18:32, teor <teor@riseup.net> wrote:
Here are some initial steps for troubleshooting: https://trac.torproject.org/projects/tor/wiki/doc/MyRelayIsSlow
Hi! I have been running a couple of relays for 4-5 years. In spite of following the advice in the link above, I still only average about 8Mb/s on both the relays, as reported by Nyx. D76E1FDC7A3D899282BB882F74111B36A6D14B64 56DCA89A6B41ADA30E891EF65FDCC071DC05079B Both are on 100Mb/s fiber optic lines, otherwise lightly loaded. Both have well over 99.99% uptime. They are on separate autonomous systems. One relay rarely has CPU usage above 5%. The other (on much lighter-weight hardware) approaches 40% CPU occasionally, but both have about the same reported bandwidth stats. The lines are both low-latency. Both relay boxes can sustain file transfers at close to the full 100Mb/s (or more) for extended periods of time in testing. Packet loss between the boxes is nearly zero. Testing with iperf shows full bandwidth to even disparate parts of the internet. One of the boxes is behind NAT, but with appropriate port forwarding. The other box sits directly on the net with a dedicated static IP. Both boxes are running Linux Devuan ASCII, though I saw similar numbers with vanilla Debian and Ubuntu. I haven't seen anything in the logs that indicates any problems. I don't have any bandwidth limits set in torrc. The metrics.torproject.org page shows "Advertised Bandwidth" of 7.45 MiB/s and 5.11 MiB/s respectively, but the bandwidth graphs are only rarely above 1 MiB/s. Both relays are reachable net-wide on their IPv4 addresses. I have IPv6 disabled for now. Correct addresses are showing on the directory authorities. They are the only relays on their respective IP addresses (indeed, on their respective AS's). They have seven flags on the consensus health page with what I think are reasonable bandwidth numbers (I'm not 100% sure how to interpret the "bw=4600" lines, but the number is over 4000 wherever it's reported for my relays.) So why do my relays seem to only be using about 8% of their available bandwidth? Thanks for any insight you can provide... --Ron
D76E1FDC D76E1FDC7A3D899282BB882F74111B36A6D14B64 2propstor Relay Search <https://metrics.torproject.org/rs.html#details/D76E1FDC7A3D899282BB882F74111B36A6D14B64> | ⇜ <https://consensus-health.torproject.org/consensus-health-2019-01-13-18-00.html#D76E1FDC7A3D899282BB882F74111B36A6D14B64> Fast Guard HSDir Running Stable V2Dir Valid bw=7800 Fast Guard HSDir Running Stable V2Dir Valid Fast Guard HSDir Running Stable V2Dir Valid bw=9000 Fast Guard HSDir Running Stable V2Dir Valid Fast Guard HSDir Running Stable V2Dir Valid bw=13100 Fast !Guard HSDir Running Stable V2Dir Valid Fast Guard HSDir Running Stable V2Dir Valid bw=4430 Fast Guard HSDir Running Stable V2Dir Valid Fast Guard HSDir Running Stable V2Dir Valid bw=10500 Fast Guard HSDir Running Stable V2Dir Valid bw=9000 bwauth=longclaw Looking at https://consensus-health.torproject.org/consensus-health.html <https://consensus-health.torproject.org/consensus-health.html> my best guess would be that you are too far away from the Authority Servers. So your delay is too hight and the bw measurement is too low. Thats why the most high speed relays cluster in some countries / providers. I am sure Teor could this explain much better than I do. Markus
On 13. Jan 2019, at 19:23, ronqtorrelays@risley.net wrote:
On Jan 12, 2019, at 18:32, teor <teor@riseup.net> wrote:
Here are some initial steps for troubleshooting: https://trac.torproject.org/projects/tor/wiki/doc/MyRelayIsSlow
Hi!
I have been running a couple of relays for 4-5 years. In spite of following the advice in the link above, I still only average about 8Mb/s on both the relays, as reported by Nyx.
D76E1FDC7A3D899282BB882F74111B36A6D14B64 56DCA89A6B41ADA30E891EF65FDCC071DC05079B
Both are on 100Mb/s fiber optic lines, otherwise lightly loaded. Both have well over 99.99% uptime. They are on separate autonomous systems. One relay rarely has CPU usage above 5%. The other (on much lighter-weight hardware) approaches 40% CPU occasionally, but both have about the same reported bandwidth stats. The lines are both low-latency.
Both relay boxes can sustain file transfers at close to the full 100Mb/s (or more) for extended periods of time in testing. Packet loss between the boxes is nearly zero. Testing with iperf shows full bandwidth to even disparate parts of the internet.
One of the boxes is behind NAT, but with appropriate port forwarding. The other box sits directly on the net with a dedicated static IP.
Both boxes are running Linux Devuan ASCII, though I saw similar numbers with vanilla Debian and Ubuntu.
I haven't seen anything in the logs that indicates any problems.
I don't have any bandwidth limits set in torrc. The metrics.torproject.org page shows "Advertised Bandwidth" of 7.45 MiB/s and 5.11 MiB/s respectively, but the bandwidth graphs are only rarely above 1 MiB/s.
Both relays are reachable net-wide on their IPv4 addresses. I have IPv6 disabled for now. Correct addresses are showing on the directory authorities. They are the only relays on their respective IP addresses (indeed, on their respective AS's). They have seven flags on the consensus health page with what I think are reasonable bandwidth numbers (I'm not 100% sure how to interpret the "bw=4600" lines, but the number is over 4000 wherever it's reported for my relays.)
So why do my relays seem to only be using about 8% of their available bandwidth?
Thanks for any insight you can provide...
--Ron _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hi! Thanks. I'm curious what, in the consensus, suggests that I'm too far from the Authority Servers? I don't know how to read that page; I can't even figure out what units they're using to report bandwidth. One of the relays is one hop away (via a lightly-loaded terabit switch) from the (formerly known as) Level3 tier 1 network, so should have excellent peering worldwide unless CenturyLink has degraded it since their acquisition last year. The other sits two or three hops (depending, apparently, on the phase of the moon) from the tier 1 network run by Telia. So, at least with my limited understanding of internet topography, they should both be topologically close to most hosts worldwide. But I will admit that there is much that I don't understand about routing at this level. Again, thanks... --Ron
On Jan 13, 2019, at 11:58, niftybunny <abuse@to-surf-and-protect.net> wrote:
Looking at https://consensus-health.torproject.org/consensus-health.html my best guess would be that you are too far away from the Authority Servers. So your delay is too hight and the bw measurement is too low. Thats why the most high speed relays cluster in some countries / providers. I am sure Teor could this explain much better than I do.
Markus
Hard to tell. A few years ago I had an ISP with a fat shiny direct line to the DE-CIX. So in theory everything was wonderful, it was not. Rule of thumb: Get as near as possible to the auth servers, same data center would be perfect :) Having all relays in one data center would make the state actors very very happy. I am still a fan of more auth servers all over the world. But who am I to tell what to do. The bw is normally displayed in kilobytes. So you have 9000 kilobytes thats around 9 megabyte and Tor will use (in best case scenario) 4,5 megabyte. ATM Tor is using 1 megabyte :( If you really want to know how much Tor will give you, run it as an Exit. Tor will love you and gives you every bit of traffic it has. Please don’t do this from home or if you are not sure what you are doing etc . (insert big fat disclaimer) Markus
On 14. Jan 2019, at 00:32, ronqtorrelays@risley.net wrote:
Hi!
Thanks. I'm curious what, in the consensus, suggests that I'm too far from the Authority Servers? I don't know how to read that page; I can't even figure out what units they're using to report bandwidth.
One of the relays is one hop away (via a lightly-loaded terabit switch) from the (formerly known as) Level3 tier 1 network, so should have excellent peering worldwide unless CenturyLink has degraded it since their acquisition last year. The other sits two or three hops (depending, apparently, on the phase of the moon) from the tier 1 network run by Telia. So, at least with my limited understanding of internet topography, they should both be topologically close to most hosts worldwide.
But I will admit that there is much that I don't understand about routing at this level.
Again, thanks...
--Ron
On Jan 13, 2019, at 11:58, niftybunny <abuse@to-surf-and-protect.net> wrote:
Looking at https://consensus-health.torproject.org/consensus-health.html my best guess would be that you are too far away from the Authority Servers. So your delay is too hight and the bw measurement is too low. Thats why the most high speed relays cluster in some countries / providers. I am sure Teor could this explain much better than I do.
Markus
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On 14 Jan 2019, at 09:32, ronqtorrelays@risley.net wrote:
Thanks. I'm curious what, in the consensus, suggests that I'm too far from the Authority Servers? I don't know how to read that page; I can't even figure out what units they're using to report bandwidth.
It's a unitless amount due to scaling, but it starts as kilobytes per second.
One of the relays is one hop away (via a lightly-loaded terabit switch) from the (formerly known as) Level3 tier 1 network, so should have excellent peering worldwide unless CenturyLink has degraded it since their acquisition last year. The other sits two or three hops (depending, apparently, on the phase of the moon) from the tier 1 network run by Telia. So, at least with my limited understanding of internet topography, they should both be topologically close to most hosts worldwide.
Your relays are on the south and west coasts of the US, which means they're further away from the Tor bandwidth authorities in northern North America and north western Europe. Tor load-balances for client latency and bandwidth capacity. Relays with higher latency or lower bandwidth are only partly used. But this reserve capacity helps during peak times. Tor also load-balances according to relay position in the circuit. Tor guards currently have about 200 Gbps capacity, and clients are currently using 75 Gbps, or 37%: https://metrics.torproject.org/bandwidth-flags.html So your guards have slightly lower than average utilisation: 1 Mbps / 5.1 Mbps = 20% 1.2 Mbps / 7.4 Mbps = 16% I wouldn't worry about it too much.
On 14 Jan 2019, at 09:49, niftybunny <abuse@to-surf-and-protect.net> wrote:
Hard to tell. A few years ago I had an ISP with a fat shiny direct line to the DE-CIX. So in theory everything was wonderful, it was not. Rule of thumb: Get as near as possible to the auth servers, same data center would be perfect :)
Having all relays in one data center would make the state actors very very happy. I am still a fan of more auth servers all over the world. But who am I to tell what to do.
We're focused on migrating to a stable bandwidth measurement system for the next year or so. A failed bandwidth measurement system is even worse: then relays can just claim to be as big as they want to be. After that, we'll look at geographical dispersion. But if we spread the relay load out too far, client performance will suffer. (And users wont see reliable, consistent performance, which is even worse.)
If you really want to know how much Tor will give you, run it as an Exit. Tor will love you and gives you every bit of traffic it has. Please don’t do this from home or if you are not sure what you are doing etc . (insert big fat disclaimer)
Yes, Tor needs more exits, their utilisation is often close to 75%. T
On Jan 13, 2019, at 18:03, teor <teor@riseup.net> wrote:
On 14 Jan 2019, at 09:32, ronqtorrelays@risley.net wrote:
Thanks. I'm curious what, in the consensus, suggests that I'm too far from the Authority Servers? I don't know how to read that page; I can't even figure out what units they're using to report bandwidth.
It's a unitless amount due to scaling, but it starts as kilobytes per second.
Good to know.
One of the relays is one hop away (via a lightly-loaded terabit switch) from the (formerly known as) Level3 tier 1 network, so should have excellent peering worldwide unless CenturyLink has degraded it since their acquisition last year. The other sits two or three hops (depending, apparently, on the phase of the moon) from the tier 1 network run by Telia. So, at least with my limited understanding of internet topography, they should both be topologically close to most hosts worldwide.
So your guards have slightly lower than average utilisation: 1 Mbps / 5.1 Mbps = 20% 1.2 Mbps / 7.4 Mbps = 16%
I wouldn't worry about it too much.
Okay, I'll try to lose the inferiority complex around my bandwidth usage. Mostly trying to make sure I wasn't doing anything silly that was causing my surplus bandwidth to go to waste.
On 14 Jan 2019, at 09:49, niftybunny <abuse@to-surf-and-protect.net> wrote:
If you really want to know how much Tor will give you, run it as an Exit. Tor will love you and gives you every bit of traffic it has. Please don’t do this from home or if you are not sure what you are doing etc . (insert big fat disclaimer)
Unfortunately, I can't do an exit at either of these sites. I'm actively working to line up a site that will support an exit. Thanks to you both for the information. --r
Hi Ron Am 14.01.2019 um 04:49 schrieb ronqtorrelays@risley.net:
So your guards have slightly lower than average utilisation: 1 Mbps / 5.1 Mbps = 20% 1.2 Mbps / 7.4 Mbps = 16%
I wouldn't worry about it too much.
Okay, I'll try to lose the inferiority complex around my bandwidth usage. Mostly trying to make sure I wasn't doing anything silly that was causing my surplus bandwidth to go to waste.
Just a guess: Your cpu idles and your bandwidth is underused? Put a second relay to your ip. You have more ips? Put more relays to them. Some underused relays might end up in a well used server :) -- Cheers, Felix
On 01/15/2019 03:23 PM, ronqtorrelays@risley.net wrote:
On Jan 15, 2019, at 11:15, Felix <zwiebel@quantentunnel.de> wrote:
Just a guess: Your cpu idles and your bandwidth is underused? Put a second relay to your ip.
Fine idea! Will do.
--Ron
You can use tor-instance-create. With systemd, you can easily manage each instance. There's a default torrc, plus supplementary ones for the instances. You can only have two relays per IP address, but you can also have multiple IP addresses.
participants (6)
-
Felix -
Ilka Schulz -
Mirimir -
niftybunny -
ronqtorrelays＠risley.net -
teor