Diversifying Bandwidth Measurement
Hi Rob and bandwidth authority operators, I've just been contacted by an Exit operator whose Exits have all seen a 60% drop in traffic over the last few weeks. This drop has been caused by a drop in bandwidth authority measurements, which seems to affect Exit connections that transit to bwauth.torproject.org (cymru) through Cogent. For example, from Australia, I get about 3 MB/s to bwauth.torproject.org [0]. A relay operator in France gets 500 KB/s. This affects at least Online S.a.ss, Feralhosting, and trabia. Do we want Exit traffic concentrating in the US? (It seems all the affected networks are UK/EU.) Is there any way that we can fix this? It also has a social impact: there are a number of disappointed exit operators out there. Can someone run another bwauth HTTPS download server on another (non-North American?) network? The required files are listed here[1]. If they did, would the bandwidth authority operators apply a patch to use it? Or could it go on 38.229.70.2 ? (That's the alternate bwauth address in the source code[2], but it's down.) [0]: https://bwauth.torproject.org/bwauth.torproject.org/64M [1]: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/a... [2]: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/b... T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Dear teor, First, my apologies! We've had some issues with Cogent as a peer of late, though I didn't realize it was impacting our Tor efforts. Ugh. Second, I'm going to turn this over to the Engineering team here. We will shift things about for the bwauth.torproject.org server to provide better connectivity. That may take a bit, but it will be a priority. Third, we can easily stand up something on 38.229.70.2. Was this one of your servers, or one of ours? Do you recall? I am about to jump on a plane to London, so I'll be offline for a few hours. I'll ensure this is handed-off to someone who remains in contact with you, teor. Again my apologies, and we're on it! Thank you! Rob. On 2/26/17 8:34 PM, teor wrote:
Hi Rob and bandwidth authority operators,
I've just been contacted by an Exit operator whose Exits have all seen a 60% drop in traffic over the last few weeks.
This drop has been caused by a drop in bandwidth authority measurements, which seems to affect Exit connections that transit to bwauth.torproject.org (cymru) through Cogent.
For example, from Australia, I get about 3 MB/s to bwauth.torproject.org [0]. A relay operator in France gets 500 KB/s.
This affects at least Online S.a.ss, Feralhosting, and trabia. Do we want Exit traffic concentrating in the US? (It seems all the affected networks are UK/EU.)
Is there any way that we can fix this? It also has a social impact: there are a number of disappointed exit operators out there.
Can someone run another bwauth HTTPS download server on another (non-North American?) network? The required files are listed here[1].
If they did, would the bandwidth authority operators apply a patch to use it?
Or could it go on 38.229.70.2 ? (That's the alternate bwauth address in the source code[2], but it's down.)
[0]: https://bwauth.torproject.org/bwauth.torproject.org/64M [1]: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAutho rity/aggregate.py#n323
[2]: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthori ty/bwauthority_child.py#n51
T
-- Tim Wilson-Brown (teor)
teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ----------------------------------------------------------------------
- --
- -- Rabbi Rob Thomas Team Cymru "It is easy to believe in freedom of speech for those with whom we agree." - Leo McKern -----BEGIN PGP SIGNATURE----- iQIcBAEBCAAGBQJYs4vhAAoJEEPoYWL6hfKNnPEQAKeeUcrg03c7b1G/yP79Oex5 TcO6AMcWsW3znYifv3SdbC5WDg/I1mAMVk/f7BFHnXvnJGtB/XafgSxQRBxmQ6I7 l6q6ELplMBaxaouRz+R/b6OGh41JSwjtbTTIU8/7Ujzq3KzIteBigE3xBx3bxnTB u/oj1jHDs5cP1JKtLSW/F9MbfHI64Dict6I1952uKCF54sSAciuvUrDVJwbNuYwu pMuj2ZDE+PvpYqO4cRUsT9KfkfIvpLf8ry6USss+IExOTQHGdZ+ahnpqrsBDovSr K6zSzuHk3V8hzzTK8eH1En2KomLyZwr7Fs4ZJd+fieLmcO8NNCyVc/yH03sKCIsr rlAD//2mNEyRDTO4v2cb1DSSLwnWYRMIC50jVxumkqbgSz3UCZq4kwEGeqJSeC6K DExY1JTklrRLx2yuLtzA0B6MA2nDitzixODdDgi9w9CDwJx+R5+gRdSvWSxAgWdS E37+Qz8QPyBtuFI6i3Zl8bQTbodjP6O4AHuEIy35xlZhxeyJoTeSCN1v0gupD/eQ DUTmUWMPbEX2fRKVV4YiPnKJ+E6MI1057ljz2F03bIgkKpOqrgphaoIUHzgPCD0h MX8iRzCXp/CN3l1jbLC8jZGknWWfPdelkJa/46DWrlW2SFbDfI7p6EOn2fKxA6ml /QrK/x7f2YKH5kBzls0w =zF4C -----END PGP SIGNATURE-----
On 27 Feb 2017, at 13:16, Rabbi Rob Thomas <robt@cymru.com> wrote:
First, my apologies! We've had some issues with Cogent as a peer of late, though I didn't realize it was impacting our Tor efforts. Ugh.
Second, I'm going to turn this over to the Engineering team here. We will shift things about for the bwauth.torproject.org server to provide better connectivity. That may take a bit, but it will be a priority.
Thank you, Rob, I really appreciate this.
Third, we can easily stand up something on 38.229.70.2. Was this one of your servers, or one of ours? Do you recall?
I'm not sure: it was set up back in 2011, before my time. Here's the original ticket: https://trac.torproject.org/projects/tor/ticket/2014 We should check how many bandwidth authority operators are configured to use this IP address before we do anything with it. For reference, here are the setup instructions for the HTTPS server.[0]
On 27 Feb 2017, at 13:23, Sebastian Hahn <tor@sebastianhahn.net> wrote:
I am and always have been operating my own https server for the bw files that sits directly next to the dirauth/bwauth server. I totally hoped everoyne else was doing the same thing or at least not share the same server for more than one bwauth instance. Is that incorrect?
I don't know: but it seems that the issue with Cogent affects a majority of the bandwidth authorities. Perhaps it's more than just cymru that's affected? If you send me the address of your HTTPS server, I can ask a relay operator to do a traceroute and speed test. Or you can do one to 51.15.50.10 (which used to get multiple megabytes a second, but has dropped to 500 KB/s).
As for the social aspect, life's tough is the best answer I have. The implemented algorithm uses median, so if we fix this we probably produce a bunch of unhappy US relay operators. Transatlantic bw sucks, and if your hosting provider doesn't pay $$$ to have a decent connection across the pond, you'll make people who use your relay sad also. I would be quite willing to guess that the cheap hosters have good links to the closest speed measurement website and nothing else…
Trans-pacific is worse. (Except in this case.) My typical answer is: "we prioritise clients".
PS: What is the status on bwauth? Can I finally upgrade my bwauth? It broke again twice recently, and it's keeping my dirauth on an older OS version that I must migrate away from soon.
I am not aware of anything that's ready right now. I would hope we could test it on the test network before deployment. [0]: https://gitweb.torproject.org/torflow.git/blob_plain/master:/NetworkScanners... T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
On 27 Feb 2017, at 13:46, teor <teor2345@gmail.com> wrote:
On 27 Feb 2017, at 13:23, Sebastian Hahn <tor@sebastianhahn.net> wrote:
I am and always have been operating my own https server for the bw files that sits directly next to the dirauth/bwauth server. I totally hoped everoyne else was doing the same thing or at least not share the same server for more than one bwauth instance. Is that incorrect?
I don't know: but it seems that the issue with Cogent affects a majority of the bandwidth authorities. Perhaps it's more than just cymru that's affected?
If you send me the address of your HTTPS server, I can ask a relay operator to do a traceroute and speed test. Or you can do one to 51.15.50.10 (which used to get multiple megabytes a second, but has dropped to 500 KB/s).
The operator gets 194 KB/s over a 14-hop path from Paris to a bandwidth authority server in Erlangen, Germany. (Paris, Amsterdam, Hamburg, Erlangen.) This looks like it's a problem for any AS that transits via Cogent. It's certainly not cmryu-specific, or even specific to North America. It appears that the Tor bandwidth authorities really should be down-rating Exits on these networks. It would still be great to make sure we don't have a single point of failure for the bandwidth authority HTTPS server. But I think we're ok otherwise. T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
Hi there,
On 27 Feb 2017, at 02:34, teor <teor2345@gmail.com> wrote:
Hi Rob and bandwidth authority operators,
I've just been contacted by an Exit operator whose Exits have all seen a 60% drop in traffic over the last few weeks.
This drop has been caused by a drop in bandwidth authority measurements, which seems to affect Exit connections that transit to bwauth.torproject.org (cymru) through Cogent.
For example, from Australia, I get about 3 MB/s to bwauth.torproject.org [0]. A relay operator in France gets 500 KB/s.
This affects at least Online S.a.ss, Feralhosting, and trabia. Do we want Exit traffic concentrating in the US? (It seems all the affected networks are UK/EU.)
Is there any way that we can fix this? It also has a social impact: there are a number of disappointed exit operators out there.
Can someone run another bwauth HTTPS download server on another (non-North American?) network? The required files are listed here[1].
If they did, would the bandwidth authority operators apply a patch to use it?
Or could it go on 38.229.70.2 ? (That's the alternate bwauth address in the source code[2], but it's down.)
[0]: https://bwauth.torproject.org/bwauth.torproject.org/64M [1]: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/a... [2]: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/b...
I am and always have been operating my own https server for the bw files that sits directly next to the dirauth/bwauth server. I totally hoped everoyne else was doing the same thing or at least not share the same server for more than one bwauth instance. Is that incorrect? As for the social aspect, life's tough is the best answer I have. The implemented algorithm uses median, so if we fix this we probably produce a bunch of unhappy US relay operators. Transatlantic bw sucks, and if your hosting provider doesn't pay $$$ to have a decent connection across the pond, you'll make people who use your relay sad also. I would be quite willing to guess that the cheap hosters have good links to the closest speed measurement website and nothing else... Cheers Sebastian PS: What is the status on bwauth? Can I finally upgrade my bwauth? It broke again twice recently, and it's keeping my dirauth on an older OS version that I must migrate away from soon.
Sebastian Hahn <tor@sebastianhahn.net> wrote Mon, 27 Feb 2017 03:23:50 +0100:
I am and always have been operating my own https server for the bw files that sits directly next to the dirauth/bwauth server. I totally hoped everoyne else was doing the same thing or at least not share the same server for more than one bwauth instance. Is that incorrect?
FWIW, I did that too when I was running a bwauth. If we conclude that this is not the case for one or more bwauth operators and that the reason is that they lack the resources for running their own source, I'm happy to set one up at Sunet/NORDUnet. We have decent presence in both Europe and the U.S., caring about quality across at least that pond. http://stats.nordu.net/ http://stats.nordu.net/stat-q/load-map/ndn-map,,traffic,peak
Linus Nordberg <linus@nordberg.se> writes:
Sebastian Hahn <tor@sebastianhahn.net> wrote Mon, 27 Feb 2017 03:23:50 +0100:
I am and always have been operating my own https server for the bw files that sits directly next to the dirauth/bwauth server. I totally hoped everoyne else was doing the same thing or at least not share the same server for more than one bwauth instance. Is that incorrect?
FWIW, I did that too when I was running a bwauth.
Also pulling from own https server for bw files, but its not pulling the files from a server that sits directly next to the bwauth server.
On 27 February 2017 at 10:35, micah <micah@riseup.net> wrote:
Linus Nordberg <linus@nordberg.se> writes:
Sebastian Hahn <tor@sebastianhahn.net> wrote Mon, 27 Feb 2017 03:23:50 +0100:
I am and always have been operating my own https server for the bw files that sits directly next to the dirauth/bwauth server. I totally hoped everoyne else was doing the same thing or at least not share the same server for more than one bwauth instance. Is that incorrect?
FWIW, I did that too when I was running a bwauth.
Also pulling from own https server for bw files, but its not pulling the files from a server that sits directly next to the bwauth server.
That's 2 of the 5 not using Cymru's server. I'm the shmuck who is but has no reason to. I think it was mentioned in the early set-up days as something I could do but didn't have to, and I forgot about it. I've switched it to use it's own HTTPS server, they run on the same box. -tom
participants (6)
-
Linus Nordberg -
micah -
Rabbi Rob Thomas -
Sebastian Hahn -
teor -
Tom Ritter