Consensus weight dropped
All, In December consensus weight of both my "nosurveillance" Tor exits dropped: https://atlas.torproject.org/#details/7C3AE76BB9E9E6E4F2AE9270FD824DF54A9441... https://atlas.torproject.org/#details/E6D740ABFFAAAD8052EDF95B2C8DC4059763F3... I assumed this to be related to the directory authorities tweaking consensus weight in response to the Lizard Squad annoyance around that time. However, traffic has not yet picked up yet. Testing download and upload speeds of my server both maxed out the 100 Mbps line. I have even done a complete reinstall from a fresh Ubuntu image a week ago (while keeping the old keys, as not to restart building reputation from scratch) but that doesn't seem to help either. I am renting this dedicated server from my own private money, in my spare time as I hate surveillance and spying by governments. But right now I starting to feel silly spending that much money for a Tor exit of which only 50kbps bandwidth is used. AFAIK nothing has changed on my server, so I am puzzled why consensus has dropped and was never restored again. Please advice, Thanks, Bram de Boer
Hello, You are not alone with this issue ( https://lists.torproject.org/pipermail/tor-relays/2015-January/006055.html ). The weirdest part is, that consensus is fixed to exactly 20 and on Jan 06, on both nodes yours and mine the weight spiked up for a short amount and then dropped back to 20. So far there is no helpful response regarding this subject. https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26... https://atlas.torproject.org/#details/6911888F83565892FE23F1B03EB501D80E1E87... On 18.01.2015 03:53 PM, Bram de Boer wrote:
All,
In December consensus weight of both my "nosurveillance" Tor exits dropped:
https://atlas.torproject.org/#details/7C3AE76BB9E9E6E4F2AE9270FD824DF54A9441... https://atlas.torproject.org/#details/E6D740ABFFAAAD8052EDF95B2C8DC4059763F3...
I assumed this to be related to the directory authorities tweaking consensus weight in response to the Lizard Squad annoyance around that time. However, traffic has not yet picked up yet.
Testing download and upload speeds of my server both maxed out the 100 Mbps line. I have even done a complete reinstall from a fresh Ubuntu image a week ago (while keeping the old keys, as not to restart building reputation from scratch) but that doesn't seem to help either.
I am renting this dedicated server from my own private money, in my spare time as I hate surveillance and spying by governments. But right now I starting to feel silly spending that much money for a Tor exit of which only 50kbps bandwidth is used.
AFAIK nothing has changed on my server, so I am puzzled why consensus has dropped and was never restored again.
Please advice,
Thanks, Bram de Boer
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
There is a similar issue with some other relays: https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26... https://atlas.torproject.org/#details/6911888F83565892FE23F1B03EB501D80E1E87... There was a thread about it but nobody found out why https://lists.torproject.org/pipermail/tor-relays/2015-January/006055.html On Sun, Jan 18, 2015 at 9:53 AM, Bram de Boer <list-tor-relays@nosur.com> wrote:
All,
In December consensus weight of both my "nosurveillance" Tor exits dropped:
https://atlas.torproject.org/#details/7C3AE76BB9E9E6E4F2AE9270FD824DF54A9441...
https://atlas.torproject.org/#details/E6D740ABFFAAAD8052EDF95B2C8DC4059763F3...
I assumed this to be related to the directory authorities tweaking consensus weight in response to the Lizard Squad annoyance around that time. However, traffic has not yet picked up yet.
Testing download and upload speeds of my server both maxed out the 100 Mbps line. I have even done a complete reinstall from a fresh Ubuntu image a week ago (while keeping the old keys, as not to restart building reputation from scratch) but that doesn't seem to help either.
I am renting this dedicated server from my own private money, in my spare time as I hate surveillance and spying by governments. But right now I starting to feel silly spending that much money for a Tor exit of which only 50kbps bandwidth is used.
AFAIK nothing has changed on my server, so I am puzzled why consensus has dropped and was never restored again.
Please advice,
Thanks, Bram de Boer
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Indeed, as Nicholas pointed out, this happened to many relays! Using the onionoo database I am calculating the average consensus weight during 18-25 october 2014, and the average consensus weight during 11-18 january 2015 of all relays that exist longer than half a year. Below is the top 20 of relays that dropped the most (so far; the list is still being generated because I didn't want to "DoS" onionoo so I am taking it slow). The full table can be found here: http://nosur.com/consensus.txt change october january days fingerprint -100.00% 124.32 0.00 283 D542940134156DCA0B872A9AFA43EECDF8E64A50 -100.00% 13549.74 0.00 1119 49095052C6668FB6BB9A2C9CEDB0A79B6193870B -100.00% 16028.09 0.00 275 28F04B33154C6EFF497D793C8E65AF09A7B1427C -100.00% 16740.50 0.00 519 61685F33CB798A87CEA3756CA7649EDB4FD314D4 -100.00% 21390.41 0.00 549 1ECD73B936CB6E6B3CD647CC204F108D9DF2C9F7 -100.00% 34109.42 0.00 209 7C3AE76BB9E9E6E4F2AE9270FD824DF54A944127 -100.00% 41042.60 0.00 327 84F8007D6E33E26EB689E40A65183CBDD5538955 -99.88% 13568.03 16.17 199 7109376DCD7F9A9F707B3138056408F2D786F079 -99.87% 9300.44 12.41 549 1FC841137C1F4C525D869B908E007D1D35727EF2 -99.85% 2446.56 3.64 1829 19D0F028BADD11B79DC5846C1D75497672E85511 -99.80% 9381.42 18.92 439 75DF14AA364152CE23EBF542152BF1B3155A2FD0 -99.68% 31269.41 101.42 189 5AE72C34DAA2EF6A511AF82FAF68AA08BE9802D8 -99.65% 5587.80 19.49 1829 721E494B0E06554D8F9A169BDBF17282C07B2588 -99.62% 1377.67 5.27 215 955AF68665FE0FE26DD79894F9684345CB9FF86F -99.53% 3561.96 16.70 227 CD8C1CF1901ADF4E0C6DA87EE8F2981854423C2A -99.41% 2998.52 17.76 327 27AEFB49809A921119821D4A32F0F3795FD46887 -99.11% 2171.73 19.30 339 24835DBD5492E21F7987DFC3FACB39016BEE59A8 -98.98% 6880.00 70.34 699 D779CB52D412F0DCAA296C9539C4ED03E2DF21A5 -98.96% 1416.23 14.78 259 81E06DAAE3ECB1DD0FFB78EC8A5FE23AB4B13DDB -98.78% 2066.60 25.23 233 C8FB6DB9BE327A05F5B29A2FFC240DACDB3C2967 Almost all of them show the same strong drop end of december and a short spike around january 5th! I wasn't able to find any common features among the relays. It hits both exits and non-exits, fast and slow relays, etc. Something is definitely wrong here! Thanks, Bram de Boer
Update: generation of the http://nosur.com/consensus.txt list has completed now, and contains 3683 relays that exist half a year or more. Again the relays at the top of the list show the sharp drop in consensus weight end of december and a short spike around January 6th.
This is roughly consistent with what I've been seeing on my own node. Weird. On Mon, Jan 19, 2015 at 1:31 AM, Bram de Boer <list-tor-relays@nosur.com> wrote:
Update: generation of the http://nosur.com/consensus.txt list has completed now, and contains 3683 relays that exist half a year or more. Again the relays at the top of the list show the sharp drop in consensus weight end of december and a short spike around January 6th.
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hey there, On 19 Jan 2015, at 10:03, eric gisse <jowr.pi@gmail.com> wrote:
This is roughly consistent with what I've been seeing on my own node.
Weird.
On Mon, Jan 19, 2015 at 1:31 AM, Bram de Boer <list-tor-relays@nosur.com> wrote:
Update: generation of the http://nosur.com/consensus.txt list has completed now, and contains 3683 relays that exist half a year or more. Again the relays at the top of the list show the sharp drop in consensus weight end of december and a short spike around January 6th.
figuring out what happened will likely involve looking at the directory authority votes to see if anything specific happened. One theory might be that the addition of a new bwauth has shifted which vote gets picked for the consensus. It's conceivable that bwauths rate relays which are placed topologically close higher than others. I'm looking forward to any analysis someone might do on this. Cheers Sebastian
Sebastian wrote:
One theory might be that the addition of a new bwauth has shifted which vote gets picked for the consensus. It's conceivable that bwauths rate relays which are placed topologically close higher than others.
Thank you for your suggestion. I hope that is not the case and the drop in consensus weight is just a temporary glitch. I will post to the development mailing list to see if the techies can comment on this. Paying hundreds of dollars of my hard earned money for a relay that is not being used by the network is not something I will keep doing for long. I support the goals and ideology of Tor, but the project will lose me as a volunteer if consensus weight does not restore soon. Thanks, Bram
Hey there,
On 19 Jan 2015, at 10:03, eric gisse <jowr.pi@gmail.com> wrote:
This is roughly consistent with what I've been seeing on my own node.
Weird.
On Mon, Jan 19, 2015 at 1:31 AM, Bram de Boer <list-tor-relays@nosur.com> wrote:
Update: generation of the http://nosur.com/consensus.txt list has completed now, and contains 3683 relays that exist half a year or more. Again the relays at the top of the list show the sharp drop in consensus weight end of december and a short spike around January 6th.
figuring out what happened will likely involve looking at the directory authority votes to see if anything specific happened. One theory might be that the addition of a new bwauth has shifted which vote gets picked for the consensus. It's conceivable that bwauths rate relays which are placed topologically close higher than others. I'm looking forward to any analysis someone might do on this.
Cheers Sebastian _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
I concur. Maybe it's worth to also post to the bugtracker? On 19.01.2015 08:14 PM, Bram de Boer wrote:
Sebastian wrote:
One theory might be that the addition of a new bwauth has shifted which vote gets picked for the consensus. It's conceivable that bwauths rate relays which are placed topologically close higher than others.
Thank you for your suggestion. I hope that is not the case and the drop in consensus weight is just a temporary glitch. I will post to the development mailing list to see if the techies can comment on this.
Paying hundreds of dollars of my hard earned money for a relay that is not being used by the network is not something I will keep doing for long. I support the goals and ideology of Tor, but the project will lose me as a volunteer if consensus weight does not restore soon.
Thanks, Bram
Hey there,
On 19 Jan 2015, at 10:03, eric gisse <jowr.pi@gmail.com> wrote:
This is roughly consistent with what I've been seeing on my own node.
Weird.
On Mon, Jan 19, 2015 at 1:31 AM, Bram de Boer <list-tor-relays@nosur.com> wrote:
Update: generation of the http://nosur.com/consensus.txt list has completed now, and contains 3683 relays that exist half a year or more. Again the relays at the top of the list show the sharp drop in consensus weight end of december and a short spike around January 6th.
figuring out what happened will likely involve looking at the directory authority votes to see if anything specific happened. One theory might be that the addition of a new bwauth has shifted which vote gets picked for the consensus. It's conceivable that bwauths rate relays which are placed topologically close higher than others. I'm looking forward to any analysis someone might do on this.
Cheers Sebastian _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 19/01/15 20:14, Bram de Boer wrote:
Sebastian wrote:
One theory might be that the addition of a new bwauth has shifted which vote gets picked for the consensus. It's conceivable that bwauths rate relays which are placed topologically close higher than others.
Thank you for your suggestion. I hope that is not the case and the drop in consensus weight is just a temporary glitch. I will post to the development mailing list to see if the techies can comment on this.
Paying hundreds of dollars of my hard earned money for a relay that is not being used by the network is not something I will keep doing for long. I support the goals and ideology of Tor, but the project will lose me as a volunteer if consensus weight does not restore soon.
Did you check whether the consensus weight *fraction* also dropped? If all consensus weights dropped by a certain factor, there's no change in the probability of clients choosing your relay at all. Take a look at Atlas' or Globe's consensus weight fraction graphs and see if they changed over the past weeks or months. All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJUvV7zAAoJEJd5OEYhk8hIvWgIAJi002cwqJbZv03t67QGCs1L DlNt50+i00zoIaYw7eVZV5F6DtkO1WRRR9VRBHgfufz2q5xj1YccR5a/mMdtZvVV T1PT0b1lXox7Hhogj7SKuvYKTbpgbTZh0WIq66ysoMBS8LqY0ZFcwiLZQs9fwo/J D1F/xsPZzjgm3GiBktmQH479LZT588Y8qzt3LGpKBpu2aXQF81YLv8plbJAo9Oh7 atD4xTZlSpA7MntJ7Rnn67aChaYDn2QgERWH+b04rloU9NGhmkBXuDZxBBb4/EWo ShxOshZRNRlcicbbm8dcaHdEVpnxi1Pl7ztidkoK2jYxOSnAqVfkayWpW7BjiIs= =4doV -----END PGP SIGNATURE-----
Yes, fraction dropped from 0,2% to 0.000072% On 19.01.2015 08:45 PM, Karsten Loesing wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 19/01/15 20:14, Bram de Boer wrote:
Sebastian wrote:
One theory might be that the addition of a new bwauth has shifted which vote gets picked for the consensus. It's conceivable that bwauths rate relays which are placed topologically close higher than others.
Thank you for your suggestion. I hope that is not the case and the drop in consensus weight is just a temporary glitch. I will post to the development mailing list to see if the techies can comment on this.
Paying hundreds of dollars of my hard earned money for a relay that is not being used by the network is not something I will keep doing for long. I support the goals and ideology of Tor, but the project will lose me as a volunteer if consensus weight does not restore soon.
Did you check whether the consensus weight *fraction* also dropped? If all consensus weights dropped by a certain factor, there's no change in the probability of clients choosing your relay at all.
Take a look at Atlas' or Globe's consensus weight fraction graphs and see if they changed over the past weeks or months.
All the best, Karsten
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org
iQEcBAEBAgAGBQJUvV7zAAoJEJd5OEYhk8hIvWgIAJi002cwqJbZv03t67QGCs1L DlNt50+i00zoIaYw7eVZV5F6DtkO1WRRR9VRBHgfufz2q5xj1YccR5a/mMdtZvVV T1PT0b1lXox7Hhogj7SKuvYKTbpgbTZh0WIq66ysoMBS8LqY0ZFcwiLZQs9fwo/J D1F/xsPZzjgm3GiBktmQH479LZT588Y8qzt3LGpKBpu2aXQF81YLv8plbJAo9Oh7 atD4xTZlSpA7MntJ7Rnn67aChaYDn2QgERWH+b04rloU9NGhmkBXuDZxBBb4/EWo ShxOshZRNRlcicbbm8dcaHdEVpnxi1Pl7ztidkoK2jYxOSnAqVfkayWpW7BjiIs= =4doV -----END PGP SIGNATURE----- _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 19/01/15 21:04, Network Operations Center wrote:
Yes, fraction dropped from 0,2% to 0.000072%
Please post your relay fingerprint(s) here, and I'll investigate this. All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJUvg2BAAoJEJd5OEYhk8hIQisIAKk08twrq+CzzlXiZQFV5dKZ RhK9lYSjcjdvKk9pRiIN8DpM4dKRfGcrqR46OsChR6n8yFR2XqQWU7N0KEzJsegr aaMDnfxUVwAQwewyM0Vzjf7hUAC9UDWxH4/Cb/OtCqsEAQIF6T4yoikA67H/U5Wy UrwjpHJhiSOpqYSNN4bXTjwPWBZbGlcptfaOkZyeEZbFa85SVMcJxSGdIwxw4Mpk S2XQgDlf1230OI6I4PbbBHdaI/4PKbBAMttP2Ij/tlheH1PohJXuGkTlI7Ayroam 3dXWjUMZkmj0FzUxHORKUhwPQkUIJk5ezUY66z9BZzaGOaNK5eUef0UYkcd57F0= =LXBV -----END PGP SIGNATURE-----
Thank you! https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26... On 20.01.2015 09:10 AM, Karsten Loesing wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 19/01/15 21:04, Network Operations Center wrote:
Yes, fraction dropped from 0,2% to 0.000072%
Please post your relay fingerprint(s) here, and I'll investigate this.
All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org
iQEcBAEBAgAGBQJUvg2BAAoJEJd5OEYhk8hIQisIAKk08twrq+CzzlXiZQFV5dKZ RhK9lYSjcjdvKk9pRiIN8DpM4dKRfGcrqR46OsChR6n8yFR2XqQWU7N0KEzJsegr aaMDnfxUVwAQwewyM0Vzjf7hUAC9UDWxH4/Cb/OtCqsEAQIF6T4yoikA67H/U5Wy UrwjpHJhiSOpqYSNN4bXTjwPWBZbGlcptfaOkZyeEZbFa85SVMcJxSGdIwxw4Mpk S2XQgDlf1230OI6I4PbbBHdaI/4PKbBAMttP2Ij/tlheH1PohJXuGkTlI7Ayroam 3dXWjUMZkmj0FzUxHORKUhwPQkUIJk5ezUY66z9BZzaGOaNK5eUef0UYkcd57F0= =LXBV -----END PGP SIGNATURE----- _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On Tue, Jan 20, 2015 at 11:44:46AM +0100, Network Operations Center wrote:
Thank you!
https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26...
The votes from the directory authorities for the last consensus period are here: http://freehaven.net/~arma/moria1-v3-status-votes In this case it looks like schokomilch has these votes for the w line: w Bandwidth=2525 Measured=1600 [moria1] w Bandwidth=2525 [dizum] w Bandwidth=2525 [Faravahar] w Bandwidth=2525 [gabelmoo] w Bandwidth=2525 [dannenberg] w Bandwidth=2525 [urras] w Bandwidth=2525 [longclaw] w Bandwidth=2525 Measured=674 [tor26] w Bandwidth=2525 [maatuska] So since only two directory authorities vote a Measured value for it, and the design calls for three opinions, it ends up unmeasured, and thus with a consensus weight of 20. You can read about the reasoning for requiring Measured votes here: https://trac.torproject.org/projects/tor/ticket/2286 In theory gabelmoo and longclaw are supposed to have opinions about your relay too: https://consensus-health.torproject.org/consensus-health.html#bwauthstatus But they don't, so here we are. The problem is likely that the bwauth (bandwidth authority) scripts are old and buggy and unmaintained. See https://trac.torproject.org/projects/tor/query?status=!closed&component=Torf... especially the tickets towards the bottom. We've already known about this in the context of "the bandwidth authority scripts are very poorly tuned for the changes that have happened in the Tor network since the scripts were written, so they vote wildly varying numbers for relays". But I don't think that we'd realized the "some relays don't get three votes at all, so they basically get zeroed out" issue. Hm. (Ultimately I am hoping for the bwauth scripts to get phased out, in favor of one of the secure bandwidth measurement schemes that various research groups have been working on lately. Those other designs also will have the advantage that it's harder to game the system by lying about your bandwidth. But it will be some months at least until we have one of those designs to evaluate.) --Roger
Very thorough explanation, thanks. I assume that there is nothing I can do except wait until a.) a new BWauth script is being introduced or b.) hope that a third node rediscovers me and once I have 3 votes in the bag I'm back on track. What still confuses me is why several nodes were being dropped by the BWauths all on Dec 28. Then on Jan 6th all of the affected nodes have been rediscovered for a day. I tracerouted all of the BWauths and I don't have trouble sending ICMP packets to said hosts, so it doesn't seem routing related. On 20.01.2015 10:58 PM, Roger Dingledine wrote:
On Tue, Jan 20, 2015 at 11:44:46AM +0100, Network Operations Center wrote:
Thank you!
https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26...
The votes from the directory authorities for the last consensus period are here: http://freehaven.net/~arma/moria1-v3-status-votes
In this case it looks like schokomilch has these votes for the w line:
w Bandwidth=2525 Measured=1600 [moria1] w Bandwidth=2525 [dizum] w Bandwidth=2525 [Faravahar] w Bandwidth=2525 [gabelmoo] w Bandwidth=2525 [dannenberg] w Bandwidth=2525 [urras] w Bandwidth=2525 [longclaw] w Bandwidth=2525 Measured=674 [tor26] w Bandwidth=2525 [maatuska]
So since only two directory authorities vote a Measured value for it, and the design calls for three opinions, it ends up unmeasured, and thus with a consensus weight of 20.
You can read about the reasoning for requiring Measured votes here: https://trac.torproject.org/projects/tor/ticket/2286
In theory gabelmoo and longclaw are supposed to have opinions about your relay too: https://consensus-health.torproject.org/consensus-health.html#bwauthstatus
But they don't, so here we are.
The problem is likely that the bwauth (bandwidth authority) scripts are old and buggy and unmaintained. See https://trac.torproject.org/projects/tor/query?status=!closed&component=Torf... especially the tickets towards the bottom.
We've already known about this in the context of "the bandwidth authority scripts are very poorly tuned for the changes that have happened in the Tor network since the scripts were written, so they vote wildly varying numbers for relays". But I don't think that we'd realized the "some relays don't get three votes at all, so they basically get zeroed out" issue. Hm.
(Ultimately I am hoping for the bwauth scripts to get phased out, in favor of one of the secure bandwidth measurement schemes that various research groups have been working on lately. Those other designs also will have the advantage that it's harder to game the system by lying about your bandwidth. But it will be some months at least until we have one of those designs to evaluate.)
--Roger
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On 20.01.2015 23:38, Network Operations Center wrote:
Very thorough explanation, thanks. I assume that there is nothing I can do except wait until a.) a new BWauth script is being introduced or b.) hope that a third node rediscovers me and once I have 3 votes in the bag I'm back on track. What about c.): Clearing out the relay keys to recreate the nodes' identity? BR Felix
On 20 Jan 2015, at 22:58, Roger Dingledine <arma@mit.edu> wrote:
We've already known about this in the context of "the bandwidth authority scripts are very poorly tuned for the changes that have happened in the Tor network since the scripts were written, so they vote wildly varying numbers for relays". But I don't think that we'd realized the "some relays don't get three votes at all, so they basically get zeroed out" issue. Hm.
Yeah we knew about it actually, ot was discussed extensively but in a different context. We knew that each dirauth misses up to 40% of all relays in absolute amount, which means it misses an unknown amount of bandwidth. Grr.
Holy crap, 40%? And that's been historically acceptable? On Tue, Jan 20, 2015 at 5:54 PM, Sebastian Hahn <mail@sebastianhahn.net> wrote:
On 20 Jan 2015, at 22:58, Roger Dingledine <arma@mit.edu> wrote:
We've already known about this in the context of "the bandwidth authority scripts are very poorly tuned for the changes that have happened in the Tor network since the scripts were written, so they vote wildly varying numbers for relays". But I don't think that we'd realized the "some relays don't get three votes at all, so they basically get zeroed out" issue. Hm.
Yeah we knew about it actually, ot was discussed extensively but in a different context. We knew that each dirauth misses up to 40% of all relays in absolute amount, which means it misses an unknown amount of bandwidth. Grr. _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 21/01/15 06:03, Sebastian Hahn wrote:
On 21 Jan 2015, at 05:10, eric gisse <jowr.pi@gmail.com> wrote:
Holy crap, 40%? And that's been historically acceptable?
I don't think it was historically like that.
Actually, it's not that bad: https://people.torproject.org/~karsten/volatile/bwauths-2015-01-21.png That graph shows that most relays have been measured by either 4 or 5 bandwidth authorities in the past weeks. Only relays with 0, 1, or 2 measurements had their consensus weight fraction set to almost 0. But it's far less than 40% of relays. I assume that's natural churn in the network. Seems like the two relays mentioned on this list have some other issue. Ideas, anyone? All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJUv4C4AAoJEJd5OEYhk8hICKYH/21kTHfZ0pG0L/OiFBrbTFy3 bNAPYeTa1AAJb0PHpweKn7gX9pBheKwCDzd36Nk8cWhkYJ/QmrumE2IXxoFTGT3L X++MCTxqtnN+XDqNlNdgyAfYVAk/jG7RtqxSzxDFTl3BSW18t8KwbOGokuWluAI+ Zp7Oo33Rmvk3/Jmgc4Ht364esrLXyFpO2SBdGCzSLLtSkPATIMrnhBx5ruDpWGcg 4wD5tNzztfBfrc7vSVwJXLTfAmJOZmaH7nBRS8CRhOlQ9x6/FBW8unSf8bD75+O7 mMep1/k2QJlTwbU9ydySgx4crwO1b5bLKOoD8kYme3TDM6qjZ5cpcETpiR01vUM= =CefK -----END PGP SIGNATURE-----
My dropping consensus overlaps exactly with the blue line on that graph time-wise. The 1 Month Graph shows this pretty well. https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26... It feels as if I am almost completely dependent on that blue node, although since one needs multiple measures, it shouldn't be possible. On 21.01.2015 11:34 AM, Karsten Loesing wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 21/01/15 06:03, Sebastian Hahn wrote:
On 21 Jan 2015, at 05:10, eric gisse <jowr.pi@gmail.com> wrote:
Holy crap, 40%? And that's been historically acceptable?
I don't think it was historically like that.
Actually, it's not that bad:
https://people.torproject.org/~karsten/volatile/bwauths-2015-01-21.png
That graph shows that most relays have been measured by either 4 or 5 bandwidth authorities in the past weeks. Only relays with 0, 1, or 2 measurements had their consensus weight fraction set to almost 0. But it's far less than 40% of relays. I assume that's natural churn in the network.
Seems like the two relays mentioned on this list have some other issue. Ideas, anyone?
All the best, Karsten
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org
iQEcBAEBAgAGBQJUv4C4AAoJEJd5OEYhk8hICKYH/21kTHfZ0pG0L/OiFBrbTFy3 bNAPYeTa1AAJb0PHpweKn7gX9pBheKwCDzd36Nk8cWhkYJ/QmrumE2IXxoFTGT3L X++MCTxqtnN+XDqNlNdgyAfYVAk/jG7RtqxSzxDFTl3BSW18t8KwbOGokuWluAI+ Zp7Oo33Rmvk3/Jmgc4Ht364esrLXyFpO2SBdGCzSLLtSkPATIMrnhBx5ruDpWGcg 4wD5tNzztfBfrc7vSVwJXLTfAmJOZmaH7nBRS8CRhOlQ9x6/FBW8unSf8bD75+O7 mMep1/k2QJlTwbU9ydySgx4crwO1b5bLKOoD8kYme3TDM6qjZ5cpcETpiR01vUM= =CefK -----END PGP SIGNATURE----- _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 21/01/15 11:54, Network Operations Center wrote:
My dropping consensus overlaps exactly with the blue line on that graph time-wise. The 1 Month Graph shows this pretty well.
https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26...
It feels as if I am almost completely dependent on that blue node, although since one needs multiple measures, it shouldn't be possible.
Ah, sorry for being unclear. The blue line is not bandwidth authority number 4, it's the number of relays for which there are measurements from 4 bandwidth authorities. But here's another graph, specifically for your relay schokomilch: https://people.torproject.org/~karsten/volatile/schokomilch-cw-2015-01-21.pn... 14C1 is tor26, 4901 is maatuska, and D586 is moria1. It looks like maatuska stopped measuring your relay between December 29 and January 5, which is a general problem with maatuska's scanner, not specific to your relay, AFAIK. And unrelated to this, neither gabelmoo nor longclaw ever measured your relay. I'd say this is something to investigate and then fix. All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJUv5oBAAoJEJd5OEYhk8hIqgkIAKE82nkthE5OEIGxBtgtqKdd cB8Kq2BdgGSq6kYi7CF6EclJuc7mhEBHkfhYiWTyCE/yhXyg7hFkyiL3rr9qGVtE gBxoJqX8OgmdoV9c74Ao83qU130SQ4BArYgFwqmC64tvSpe4jl6tiFddndF5MomC 4YH0Nuw6VnxUTPcG2ZiRjab5sZcpJ4YLJzBmbjB1oXRy2UaRQOTT2o1Cz65fBIt3 4lCW9lMTfxn4G3JUEO2Pj1rTgyoKM3U7spv/IdyrmQmK+wwzKCDRIxkWAIDbM2Rw lOEwNz3rodsIksgyZk/oOPA/NiJUaKEWKX7DR2D2rBhCIuRamV6np1b5oE2uRjU= =BhVW -----END PGP SIGNATURE-----
Ah I see, thanks for taking the time investigating this. If there is something I need to do to help, please let me know. On 21.01.2015 01:22 PM, Karsten Loesing wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 21/01/15 11:54, Network Operations Center wrote:
My dropping consensus overlaps exactly with the blue line on that graph time-wise. The 1 Month Graph shows this pretty well.
https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26...
It feels as if I am almost completely dependent on that blue node, although since one needs multiple measures, it shouldn't be possible.
Ah, sorry for being unclear. The blue line is not bandwidth authority number 4, it's the number of relays for which there are measurements from 4 bandwidth authorities.
But here's another graph, specifically for your relay schokomilch:
https://people.torproject.org/~karsten/volatile/schokomilch-cw-2015-01-21.pn...
14C1 is tor26, 4901 is maatuska, and D586 is moria1. It looks like maatuska stopped measuring your relay between December 29 and January 5, which is a general problem with maatuska's scanner, not specific to your relay, AFAIK.
And unrelated to this, neither gabelmoo nor longclaw ever measured your relay. I'd say this is something to investigate and then fix.
All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org
iQEcBAEBAgAGBQJUv5oBAAoJEJd5OEYhk8hIqgkIAKE82nkthE5OEIGxBtgtqKdd cB8Kq2BdgGSq6kYi7CF6EclJuc7mhEBHkfhYiWTyCE/yhXyg7hFkyiL3rr9qGVtE gBxoJqX8OgmdoV9c74Ao83qU130SQ4BArYgFwqmC64tvSpe4jl6tiFddndF5MomC 4YH0Nuw6VnxUTPcG2ZiRjab5sZcpJ4YLJzBmbjB1oXRy2UaRQOTT2o1Cz65fBIt3 4lCW9lMTfxn4G3JUEO2Pj1rTgyoKM3U7spv/IdyrmQmK+wwzKCDRIxkWAIDbM2Rw lOEwNz3rodsIksgyZk/oOPA/NiJUaKEWKX7DR2D2rBhCIuRamV6np1b5oE2uRjU= =BhVW -----END PGP SIGNATURE----- _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 21/01/15 13:35, Network Operations Center wrote:
Ah I see, thanks for taking the time investigating this. If there is something I need to do to help, please let me know.
You could start a second relay on the same physical machine on a different port and see whether the bandwidth scanners pick that up. Give it a day or two, and see if only tor26 and moria1 measure it. You can find out the latter by fetching the latest votes from all directory authorities and searching for your nickname and subsequent "w" lines in them: https://collector.torproject.org/recent/relay-descriptors/votes/ And of course you'll notice on Atlas whether the consensus weight fraction of your new relay will be 0.00000% or not. All the best, Karsten
On 21.01.2015 01:22 PM, Karsten Loesing wrote: On 21/01/15 11:54, Network Operations Center wrote:
My dropping consensus overlaps exactly with the blue line on that graph time-wise. The 1 Month Graph shows this pretty well.
https://atlas.torproject.org/#details/3D7E274A87D9A89AF064C13D1EE4CA1F184F26...
It feels as if I am almost completely dependent on that blue node,
although since one needs multiple measures, it shouldn't be possible.
Ah, sorry for being unclear. The blue line is not bandwidth authority number 4, it's the number of relays for which there are measurements from 4 bandwidth authorities.
But here's another graph, specifically for your relay schokomilch:
https://people.torproject.org/~karsten/volatile/schokomilch-cw-2015-01-21.pn...
14C1 is tor26, 4901 is maatuska, and D586 is moria1. It looks like maatuska stopped measuring your relay between December 29 and January 5, which is a general problem with maatuska's scanner, not specific to your relay, AFAIK.
And unrelated to this, neither gabelmoo nor longclaw ever measured your relay. I'd say this is something to investigate and then fix.
All the best, Karsten
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJUv5+CAAoJEJd5OEYhk8hIqTwH/0AeNxhZu7zNlkdN3W2F9TT1 k1yAeZrZtf4qfb+rAkTD21njzrFo3VUaO7QhBBZPOuVsLgfWGc9YAVVAINnRQPOq WzgE7lr/123xbYRZV5if+tcODhrj9WHmnOK7UT+iA3EvekkNU7S4Z1iJh/klxix5 31q36RvDbo/OQIWRV8p+pirwJ5fPes5bDr9B3JbKAHw10XeqG1VlGZVoPaLXR05W AwQBhU80UsRiZzxH434KwAXY/5iaCY92cLoux+N/B+nAMZ33NGAjCHb530FghpB8 O0KZHawlV+XFMoaEM+il2I1aDoKC+gtmBerdUk93ZD55PxVAjfOBMLrgVWgtbVI= =j/m5 -----END PGP SIGNATURE-----
Thank you all for looking into this. Karsten wrote:
You could start a second relay on the same physical machine on a different port and see whether the bandwidth scanners pick that up. Give it a day or two, and see if only tor26 and moria1 measure it.
In fact, both the 7C3AE76BB9E9E6E4F2AE9270FD824DF54A944127 and E6D740ABFFAAAD8052EDF95B2C8DC4059763F365 relays operate on the same IP address. Both dropped to 0.000000%. However, other nodes in the same AS16265 are doing fine (e.g. B144DC5C08AF1FB3ABD729AFC2CF938CF63F78AC). This seems to suggest that the route between the bwauths and the relay is irrelevant and connectivity is not an issue. I can imagine that an overloaded bwauth occasionally skips a few relays. But wouldn't that be corrected automatically during the measurement the next day? Given that the relays are missing votes consistently during many consecutive days, some other mechanism must be causing this. Would a quick-fix be to randomize the order in which relays are measured? That way, if a bwauth has trouble processing the entire list in 24h, every day other relays are given a chance? Thanks, Bram
All, Consensus weight of my relays and those of others is still near zero, and not improving. For a network that attempts to break censorship, it is peculiar that this is getting so little attention. Apparently a few malfunctioning bwauth systems is enough to censor specific Tor relays. Endless research and development effort is put in tweaking and optimizing the relay-to-relay communication, but having only a few systems in the world that determine the consensus weight of the entire network does not seem to trouble anyone. Wierd. I hope the bwauth operators can find a way to correct the problem. I am feeling silly spending good money on a high-end server with unmetered bandwidth that has now been relaying a whopping 300 Kb/s on average during the last five weeks. Thanks, Bram
Thank you all for looking into this.
Karsten wrote:
You could start a second relay on the same physical machine on a different port and see whether the bandwidth scanners pick that up. Give it a day or two, and see if only tor26 and moria1 measure it.
In fact, both the 7C3AE76BB9E9E6E4F2AE9270FD824DF54A944127 and E6D740ABFFAAAD8052EDF95B2C8DC4059763F365 relays operate on the same IP address. Both dropped to 0.000000%.
However, other nodes in the same AS16265 are doing fine (e.g. B144DC5C08AF1FB3ABD729AFC2CF938CF63F78AC). This seems to suggest that the route between the bwauths and the relay is irrelevant and connectivity is not an issue.
I can imagine that an overloaded bwauth occasionally skips a few relays. But wouldn't that be corrected automatically during the measurement the next day? Given that the relays are missing votes consistently during many consecutive days, some other mechanism must be causing this.
Would a quick-fix be to randomize the order in which relays are measured? That way, if a bwauth has trouble processing the entire list in 24h, every day other relays are given a chance?
Thanks, Bram
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Karsten Loesing <karsten@torproject.org> wrote Wed, 21 Jan 2015 13:22:25 +0100: | But here's another graph, specifically for your relay schokomilch: | | https://people.torproject.org/~karsten/volatile/schokomilch-cw-2015-01-21.pn... | | 14C1 is tor26, 4901 is maatuska, and D586 is moria1. It looks like | maatuska stopped measuring your relay between December 29 and January | 5, which is a general problem with maatuska's scanner, not specific to | your relay, AFAIK. I am the operator of the bandwidth authority reporting to maatuska. This bandwidth authority has had multiple issues since late December but is now making progress towards serving maatuska with measurement data again. This should not have been a big deal, but since two other bw auths apparently have (had) trouble measuring some relays it hurt more than anticipated. Thanks for your patience and time put into digging into this. Thanks for running relays!
Karsten wrote:
Did you check whether the consensus weight *fraction* also dropped?
Yes, it dropped from 0.193553% to 0.000000%
If all consensus weights dropped by a certain factor, there's no change in the probability of clients choosing your relay at all.
My relay used to push 80 Mbps, now it is doing 0.05 Mbps, so I am very certain the probability of clients choosing my relay has changed! Thanks, Bram
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 19/01/15 21:54, Bram de Boer wrote:
Karsten wrote:
Did you check whether the consensus weight *fraction* also dropped?
Yes, it dropped from 0.193553% to 0.000000%
If all consensus weights dropped by a certain factor, there's no change in the probability of clients choosing your relay at all.
My relay used to push 80 Mbps, now it is doing 0.05 Mbps, so I am very certain the probability of clients choosing my relay has changed!
Please post your relay fingerprint(s) here, and I'll investigate this. All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJUvg2UAAoJEJd5OEYhk8hId+oIAIjM8ckPEilaGvvvI8hN66uE WMmUZNilvC6WgD9DG39Setl1PWLpzcLvIEHj9DwOeFJloV0ivQ8YH7QfzrLmXYvj Ht7zUcmjubRw/kLqcaqFLqfOyMqctQnREo3g5DumCk3TlaHGlwvmM6Hk2lQ6Nkna ay5uXwSEWtvoAr8XevvM1ITuqYVYsYjeDYLAs+6ounTSVvj6xXqMlCBybJY4CKeO 7tAQsgqLrMzRAzWg6UImx86hWUcQGGnRQ9Uq7tO5FhNSMmO/D+gN6zl4qWmynCaO ppSradCIrMkLD89EjbHfF/9n82FmPbvBx/rIOUAYccRKkP1aE8dtM8+RcNUrvr8= =DPyj -----END PGP SIGNATURE-----
Karsten wrote:
Did you check whether the consensus weight *fraction* also dropped?
Yes, it dropped from 0.193553% to 0.000000%
Please post your relay fingerprint(s) here, and I'll investigate this.
These are the fingerprints of the relays I operate: 7C3AE76BB9E9E6E4F2AE9270FD824DF54A944127 E6D740ABFFAAAD8052EDF95B2C8DC4059763F365 Thanks, Bram
participants (10)
-
Bram de Boer -
eric gisse -
Felix Buedenhoelzer -
Karsten Loesing -
Linus Nordberg -
Network Operations Center -
Nicholas Suan -
Roger Dingledine -
Sebastian Hahn -
Sebastian Hahn