Relay suddenly stopped relaying traffic but is still in the consensus
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello. Yesterday, one of my relays suddenly (within minutes) went from passing about 5 Mbps to passing nothing. The traffic rate dropped off as rapidly as if it went into hibernation. The consensus still says it is up. Nyx does not show any indication of failures and all directory authorities are able to reach its ORPort. There's no network censorship occurring as far as I can tell (my other relays can reach its ORPort). Nothing is out of place in the logs, but Nyx still reports an average rate of 1 KB/sec. Why is this? The relay is 0EF2A357140FCBDAE2F948CC2228909695B81B35. Regards, forest -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQtr8ZXhq/o01Qf/pow+TRLM+X4xgUCasLlCAAKCRAw+TRLM+X4 xkrfAQClP3aXODcrcvS9KOgj7tPCUeB9holpR1kpSrpF99qAFQD9EjjMBr3lKZgF F3RLEbRjTQDULtmqSqMbkJaA/oW4FQg= =6Xg7 -----END PGP SIGNATURE-----
On Sun, Oct 04, 2026 at 11:46:17PM +0000, forest via tor-relays wrote:
Yesterday, one of my relays suddenly (within minutes) went from passing about 5 Mbps to passing nothing. The traffic rate dropped off as rapidly as if it went into hibernation. The consensus still says it is up. Nyx does not show any indication of failures and all directory authorities are able to reach its ORPort. There's no network censorship occurring as far as I can tell (my other relays can reach its ORPort). Nothing is out of place in the logs, but Nyx still reports an average rate of 1 KB/sec. Why is this? The relay is 0EF2A357140FCBDAE2F948CC2228909695B81B35.
That relay currently has only 4 votes for the Fast flag, out of the 9 directory authority votes: $ curl -s https://www.freehaven.net/~arma/moria1-v3-status-votes|grep -A2 103.174.51.65 |grep ^s s Running Stable V2Dir Valid s Fast Running Stable V2Dir Valid s Fast Running Stable V2Dir Valid s Fast Running Stable V2Dir Valid s Running Stable V2Dir Valid s Fast Running Stable V2Dir Valid s Running Stable V2Dir Valid s Running Stable V2Dir Valid s Running Stable V2Dir Valid (Normally I would point you at https://consensus-health.torproject.org/#relayinfo for a web interface way of exploring the votes, but with our gitlab problems lately that page is having problems keeping updated: https://gitlab.torproject.org/tpo/tpa/team/-/work_items/42828) Not getting the Fast flag is because your consensus weight is quite low, which in turn is because the bwauths think you are underperforming compared to other relays that advertise a similar speed: $ grep -A5 103.174.51.65 v3-status-votes|grep ^w w Bandwidth=698 Measured=230 w Bandwidth=698 w Bandwidth=698 w Bandwidth=698 w Bandwidth=698 Measured=81 w Bandwidth=698 w Bandwidth=698 Measured=120 w Bandwidth=698 Measured=64 w Bandwidth=698 Measured=72 So, I don't know why it might have dropped off a cliff all at once, but that would certainly explain a drop over the course of an hour. Since you're right around the threshold for Fast, I wouldn't be surprised if you wobble back and forth between being Fast (and getting some traffic) and not being Fast (and getting basically no traffic). --Roger
Used Roger's guidance to help automate the analysis (and fix some bugs) in https://metrics.1aeo.com (all open source!) Below are inline Roger's comments and the rendered pages for all relays without having to know which strings to grep.
Normally I would point you at https://consensus-health.torproject.org/#relayinfo Same vote data, per relay, as deep links below.
$ curl -s .../moria1-v3-status-votes|grep -A2 103.174.51.65 |grep ^s https://metrics.1aeo.com/relay/0EF2A357140FCBDAE2F948CC2228909695B81B35/#col... Flags column of the per-authority table: each authority's s line for this relay (green = all agree, yellow = some).
https://metrics.1aeo.com/relay/0EF2A357140FCBDAE2F948CC2228909695B81B35/#fla... Fast row: Fast vote count, the bandwidth each authority judged by (measured / relay-reported / not published), and fast-speed.
$ grep -A5 103.174.51.65 v3-status-votes|grep ^w https://metrics.1aeo.com/relay/0EF2A357140FCBDAE2F948CC2228909695B81B35/#col... Cons Wt column: each vote's w value, Measured= (M) or Bandwidth= (R); R is grayed out once 3+ bwauths measured the relay.
https://metrics.1aeo.com/relay/0EF2A357140FCBDAE2F948CC2228909695B81B35/#bw-... Consensus Weight: low median of the votes' Measured= values, i.e. the consensus w Bandwidth=. https://metrics.1aeo.com/relay/0EF2A357140FCBDAE2F948CC2228909695B81B35/#bw-... Measured By: how many bwauths have a Measured= value for this relay, and which don't. https://metrics.1aeo.com/relay/0EF2A357140FCBDAE2F948CC2228909695B81B35/#col... Fast (BW|T) column: each authority's bandwidth for this relay vs its own fast-speed, green where it qualifies.
wobble back and forth https://metrics.1aeo.com/relay/0EF2A357140FCBDAE2F948CC2228909695B81B35/#fla... Flag Uptime: share of the last 1M/6M the relay held its primary flag, here Fast (Onionoo). Fast votes: Oct 8 4/9, Oct 10 5/9, Oct 11 20:00 8/9 (moria1: Measured=810 < fast-speed 1048). Oct 11 20:00: Measured=140-810, consensus w Bandwidth=270. (M) = Measured=, (R) = Bandwidth=. Page shows bit/s (Measured=270 -> 2.2 Mbit/s); raw KB/s on hover.
Roger's commands caught our KB/s, per-authority Fast/Guard and median bugs; fixed in https://github.com/1aeo/allium/pull/234 On Sunday, October 4th, 2026 at 8:13 PM, Roger Dingledine via tor-relays <tor-relays@lists.torproject.org> wrote:
On Sun, Oct 04, 2026 at 11:46:17PM +0000, forest via tor-relays wrote:
Yesterday, one of my relays suddenly (within minutes) went from passing about 5 Mbps to passing nothing. The traffic rate dropped off as rapidly as if it went into hibernation. The consensus still says it is up. Nyx does not show any indication of failures and all directory authorities are able to reach its ORPort. There's no network censorship occurring as far as I can tell (my other relays can reach its ORPort). Nothing is out of place in the logs, but Nyx still reports an average rate of 1 KB/sec. Why is this? The relay is 0EF2A357140FCBDAE2F948CC2228909695B81B35.
That relay currently has only 4 votes for the Fast flag, out of the 9 directory authority votes:
$ curl -s https://www.freehaven.net/~arma/moria1-v3-status-votes|grep -A2 103.174.51.65 |grep ^s s Running Stable V2Dir Valid s Fast Running Stable V2Dir Valid s Fast Running Stable V2Dir Valid s Fast Running Stable V2Dir Valid s Running Stable V2Dir Valid s Fast Running Stable V2Dir Valid s Running Stable V2Dir Valid s Running Stable V2Dir Valid s Running Stable V2Dir Valid
(Normally I would point you at https://consensus-health.torproject.org/#relayinfo for a web interface way of exploring the votes, but with our gitlab problems lately that page is having problems keeping updated: https://gitlab.torproject.org/tpo/tpa/team/-/work_items/42828)
Not getting the Fast flag is because your consensus weight is quite low, which in turn is because the bwauths think you are underperforming compared to other relays that advertise a similar speed:
$ grep -A5 103.174.51.65 v3-status-votes|grep ^w w Bandwidth=698 Measured=230 w Bandwidth=698 w Bandwidth=698 w Bandwidth=698 w Bandwidth=698 Measured=81 w Bandwidth=698 w Bandwidth=698 Measured=120 w Bandwidth=698 Measured=64 w Bandwidth=698 Measured=72
So, I don't know why it might have dropped off a cliff all at once, but that would certainly explain a drop over the course of an hour.
Since you're right around the threshold for Fast, I wouldn't be surprised if you wobble back and forth between being Fast (and getting some traffic) and not being Fast (and getting basically no traffic).
--Roger
_______________________________________________ tor-relays mailing list -- tor-relays@lists.torproject.org To unsubscribe send an email to tor-relays-leave@lists.torproject.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 2026-10-05 03:11, Roger Dingledine via tor-relays wrote:
Not getting the Fast flag is because your consensus weight is quite low, which in turn is because the bwauths think you are underperforming compared to other relays that advertise a similar speed:
$ grep -A5 103.174.51.65 v3-status-votes|grep ^w w Bandwidth=698 Measured=230 w Bandwidth=698 w Bandwidth=698 w Bandwidth=698 w Bandwidth=698 Measured=81 w Bandwidth=698 w Bandwidth=698 Measured=120 w Bandwidth=698 Measured=64 w Bandwidth=698 Measured=72
So, I don't know why it might have dropped off a cliff all at once, but that would certainly explain a drop over the course of an hour.
Since you're right around the threshold for Fast, I wouldn't be surprised if you wobble back and forth between being Fast (and getting some traffic) and not being Fast (and getting basically no traffic).
You're right, that was the issue. I know it's in SEA and that's pretty far from any bwauth, but it should not be estimating mere kB/s when the international connectivity is good: iperf3 Network Speed Tests (IPv4): - --------------------------------- Provider | Location (Link) | Send Speed | Recv Speed | Ping - ----- | ----- | ---- | ---- | ---- Clouvider | London, UK (10G) | busy | busy | 172 ms Eranium | Amsterdam, NL (100G) | 86.8 Mbits/sec | 159 Mbits/sec | 175 ms Uztelecom | Tashkent, UZ (10G) | 109 Mbits/sec | 106 Mbits/sec | 170 ms Leaseweb | Singapore, SG (10G) | 169 Mbits/sec | 152 Mbits/sec | 47.6 ms Clouvider | Los Angeles, CA, US (10G) | 103 Mbits/sec | 112 Mbits/sec | 234 ms Leaseweb | NYC, NY, US (10G) | 74.2 Mbits/sec | 113 Mbits/sec | 286 ms Edgoo | Sao Paulo, BR (1G) | 57.4 Mbits/sec | 64.3 Mbits/sec | 314 ms What could be causing such a large discrepency between measurements from bwauths and any other measurement from the same location? I'm aware that the bwauths are mostly centered in Eastern USA and Western Europe and so will be biased against distant locations, but what the bwauths report is not even accurate, even when accounting for the distance. Why would each bwauth in Germany show under 3 Mbps but an iperf3 server just 20 ms away in the Netherlands shows 159 Mbps? The international connectivity is really not _that_ bad. So what's going wrong here and how to I fix it? Regards, forest -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQtr8ZXhq/o01Qf/pow+TRLM+X4xgUCasx1BQAKCRAw+TRLM+X4 xuYDAQCikkvYQ4Je79xCVSdOZj4ukD7UmLVngehH96gXqqd6LwD/VIkbo6z6OVT5 LveuLcZigbIn3Cdz4BaVNRucYBZulwA= =xpdJ -----END PGP SIGNATURE-----
participants (3)
-
forest -
Roger Dingledine -
Tor at 1AEO