Hi folks!
We're hunting down a mystery where two of our big university relays are having troubles reaching the Tor directory authorities: https://gitlab.torproject.org/tpo/network-health/analysis/-/issues/86
Can you check to see if your relay is in a similar situation?
In particular, the situation to look for is "Tor process is still running fine from your perspective, but, relay-search (https://atlas.torproject.org/) says you are no longer running."
If your relay is in this situation, the next step is to check your Tor logs, try to rule out other issues like firewall rules on your side, and then (if you're able) to start exploring traceroutes to the directory authority IP addresses vs other addresses. If you need more direct help, we can help you debug or answer other questions on #tor-relays on IRC.
Thanks, --Roger
Hi, my relay is showing no abnormal logs, publishes it's descriptor just fine and shows on Atlas / Metrics.
Thank you for the heads up though!
-GH
On Tuesday, October 22nd, 2024 at 10:48 AM, Roger Dingledine arma@torproject.org wrote:
Hi folks!
We're hunting down a mystery where two of our big university relays are having troubles reaching the Tor directory authorities: https://gitlab.torproject.org/tpo/network-health/analysis/-/issues/86
Can you check to see if your relay is in a similar situation?
In particular, the situation to look for is "Tor process is still running fine from your perspective, but, relay-search (https://atlas.torproject.org/) says you are no longer running."
If your relay is in this situation, the next step is to check your Tor logs, try to rule out other issues like firewall rules on your side, and then (if you're able) to start exploring traceroutes to the directory authority IP addresses vs other addresses. If you need more direct help, we can help you debug or answer other questions on #tor-relays on IRC.
Thanks, --Roger
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Tossing this kdea out there since it is more an attack on bitcoin style decentralization rather than Tor style decentralization. I do not know if it applies to Tor.
Could this be a form of an "Eclipse" attack?
"Eclipse attacks occur when a node is isolated from all honest peers but remains connected to at least one malicious peer." https://bitcoinops.org/en/topics/eclipse-attacks/
Could an ASN feasibly deny connections to all official directories besides a malicious one to serve a malicious consensus? Perhaps to be used to then provide malicious controlled circuits or other attacks.
I understand that there seems to be a signing of the consensus by directory authorities. Can an outdated, yet cryptographically valid, consesus be served by malicous DA's when others are eclipsed? Perhaps this could serve an older or more vulnurable consensus.
Tossing this idea out there since blocking just of directory authorities compared to all Tor relays came off as odd to me.
-------- Original Message -------- On 10/22/24 4:48 AM, Roger Dingledine - arma at torproject.org arma_at_torproject_org_dakfxbjzjp@simplelogin.co wrote:
Hi folks!
We're hunting down a mystery where two of our big university relays are having troubles reaching the Tor directory authorities: https://gitlab.torproject.org/tpo/network-health/analysis/-/issues/86
Can you check to see if your relay is in a similar situation?
In particular, the situation to look for is "Tor process is still running fine from your perspective, but, relay-search (https://atlas.torproject.org/) says you are no longer running."
If your relay is in this situation, the next step is to check your Tor logs, try to rule out other issues like firewall rules on your side, and then (if you're able) to start exploring traceroutes to the directory authority IP addresses vs other addresses. If you need more direct help, we can help you debug or answer other questions on #tor-relays on IRC.
Thanks, --Roger
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On Tue, Oct 29, 2024 at 04:35:35AM +0000, pasture_clubbed242--- via tor-relays wrote:
"Eclipse attacks occur when a node is isolated from all honest peers but remains connected to at least one malicious peer." https://bitcoinops.org/en/topics/eclipse-attacks/
Could an ASN feasibly deny connections to all official directories besides a malicious one to serve a malicious consensus? Perhaps to be used to then provide malicious controlled circuits or other attacks.
That style of attack is actually one of the reasons why Tor still has the directory authority design. Some of the other 'distributed trust' anonymity designs out there (no need to name names, that's not the point) have less control over who the relays are, but much more importantly they have less control over which pieces of the network clients learn about, which can lead to all sorts of attacks and surprises.
I understand that there seems to be a signing of the consensus by directory authorities. Can an outdated, yet cryptographically valid, consesus be served by malicous DA's when others are eclipsed? Perhaps this could serve an older or more vulnurable consensus.
No, those sorts of attacks should not be possible, and if you find some way to break it, please let us know!
For a distant-past background paper on the topic of learning things about the user if they only know about a subset of the network, check out https://www.freehaven.net/anonbib/full/date.html#danezis2006rfa and then for a slightly more recent analysis showing how hard it is to protect against these attacks in "structured" scalable peer-to-peer anonymity networks, see https://www.freehaven.net/anonbib/#wpes09-dht-attack
All of this said, Tor's directory authority design is not perfect. It might benefit from using some of the innovations from the consensus / blockchain worlds over the past decade, in terms of fast robust agreement among peers about the state of the network. For a concrete edge case where we could do better, see this paper from this year's Oakland conference: https://www.freehaven.net/anonbib/#directory-oakland2024
So, for sure the directory design isn't quite as good as we might like it to be; but one of its big strengths, which has come in handy over the years, is that it is really simple.
--Roger
NOTE: this email has been written around 2024-10-30 19:00 UTC, about 2.5 hours ago. As I was testing stuff, suddenly I got flags and *some* traffic on my node. It also appeared back in atlas as mysteriously as it did disappear. Nonetheless I’m sending the email, hoping it is helpful, and will report back in a week.
Can you check to see if your relay is in a similar situation?
In particular, the situation to look for is "Tor process is still running fine from your perspective, but, relay-search (https://atlas.torproject.org/) says you are no longer running."
Hello, I see the same with my node.
Prompted by a fall in traffic I checked relay-search a few days ago and found the relay is no longer listed. Going through the logs it seems there is no new circuits being made since October 23. In Nyx I see at most a few incoming and outgoing connections.
The node is mpan11 92A125B9AB491AFC084E4257B552D0FB56090CB3, and this is the first time in about 10 years I experience anything like that.
If your relay is in this situation, the next step is to check your Tor logs, try to rule out other issues like firewall rules on your side, and then (if you're able) to start exploring traceroutes to the directory authority IP addresses vs other addresses. If you need more direct help, we can help you debug or answer other questions on #tor-relays on IRC.
Traceroutes from the node listed in [1]. From two other hosts (French, German VPS) all addresses except maatuska are accessible, and the final hops are similar. I asked a friend (same ISP, city) to ping all the addresses that don’t respond and they confirmed no ping responses. However, maatuska seems to respond to ICMP pings.
If that helps, on October 23 I see some entries[2] I don’t recall ever seeing. This did happen when my network connection died for about 10 minutes around 01:20. At 22 I had to reboot the system and since then the node reports 0 circuits. But that may be just a coincidence and the responses might’ve been ISP’s modem interfering in some way.
____ [1] Traceroutes, to 30 hops. ‘…’ indicates no more replies: ----------------------------------------------------------------------- # moria1 $ tracepath -b 128.31.0.39 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.499ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 15.434ms 3: no reply …
# tor26 $ tracepath -b 217.196.147.77 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.359ms 1: _gateway (192.168.1.1) 3.782ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 3.833ms 3: war-r21.tpnet.pl (80.50.19.25) 2.383ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 12.699ms 5: as33891-ic-331887.ip.twelve99-cust.net (62.115.13.142) 16.619ms asymm 6 6: ae5-2058.slz10.core-backbone.com (81.95.2.50) 19.534ms asymm 9 7: core-backbone.conova.at (5.56.17.86) 17.129ms asymm 12 8: 149.3.164.138 (149.3.164.138) 18.783ms asymm 13 9: reliant-gw.noreply.org (185.69.162.222) 27.474ms asymm 13 10: tor.cypherpunks.eu (217.196.147.77) 30.815ms !H
# dizum $ tracepath -b 45.66.35.11 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.739ms 1: _gateway (192.168.1.1) 0.716ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.480ms 3: war-r21.tpnet.pl (80.50.19.25) 4.162ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 15.974ms 5: win-bb2-link.ip.twelve99.net (62.115.114.182) 22.121ms asymm 6 6: ffm-bb2-link.ip.twelve99.net (62.115.138.22) 26.017ms asymm 8 7: adm-bb2-link.ip.twelve99.net (62.115.137.222) 30.844ms asymm 9 8: no reply 9: novoserve-ic-354347.ip.twelve99-cust.net (62.115.188.167) 31.944ms asymm 8 10: no reply 11: no reply 12: no reply 13: tor.dizum.com (45.66.35.11) 34.349ms reached
# gabelmoo $ tracepath -b 131.188.40.189 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 2.812ms 1: _gateway (192.168.1.1) 1.120ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 11.639ms 3: no reply …
# dannenberg $ tracepath -b 193.23.244.244 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.563ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.746ms 3: war-r22.tpnet.pl (80.50.20.25) 6.182ms 4: win-b2-link.ip.twelve99.net (213.248.103.4) 13.986ms 5: win-bb1-link.ip.twelve99.net (62.115.114.184) 16.536ms asymm 6 6: mcn-b1-link.ip.twelve99.net (62.115.136.41) 20.223ms asymm 7 7: mcn-b3-link.ip.twelve99.net (62.115.127.30) 19.945ms 8: interlinkgmbh-ic-381330.ip.twelve99-cust.net (62.115.186.123) 18.466ms asymm 9 9: r1-str1-de.as5405.net (94.103.180.27) 62.334ms (This broken router returned corrupted payload) asymm 15 10: r4-fra2-de.as5405.net (94.103.180.11) 38.532ms (This broken router returned corrupted payload) asymm 14 11: r4-fra1-de.as5405.net (94.103.180.7) 36.582ms (This broken router returned corrupted payload) asymm 12 12: r3-ber1-de.as5405.net (94.103.180.2) 37.920ms asymm 11 13: cust-syseleven.r3-ber1-de.as5405.net (45.153.82.11) 38.488ms asymm 12 14: ae0-0.blu1-r2.de.syseleven.net (109.68.226.26) 39.453ms asymm 16 15: ae2-0.bak1-r1.syseleven.net (109.68.226.23) 41.234ms asymm 14 16: no reply 17: dannenberg.torauth.de (193.23.244.244) 34.575ms reached
# maatuska $ tracepath -b 171.25.193.9 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 3.200ms 1: _gateway (192.168.1.1) 0.674ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 8.933ms 3: war-r21.tpnet.pl (80.50.19.25) 5.954ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 12.257ms 5: win-bb2-link.ip.twelve99.net (62.115.114.182) 13.424ms asymm 6 6: ffm-bb2-link.ip.twelve99.net (62.115.138.22) 24.160ms asymm 9 7: kbn-bb6-link.ip.twelve99.net (62.115.114.94) 40.453ms asymm 10 8: kbn-b1-link.ip.twelve99.net (62.115.143.7) 35.856ms 9: globalconnect-ic-368141.ip.twelve99-cust.net (62.115.185.221) 37.018ms 10: 146.247.201.152 (146.247.201.152) 44.090ms asymm 9 11: no reply 12: no reply 13: 195.225.184.150 (195.225.184.150) 48.578ms asymm 15 14: rs2.dfri.net (171.25.193.225) 46.928ms asymm 16 15: rs2.dfri.net (171.25.193.225) 48.011ms pmtu 1420 15: no reply …
# longclaw $ tracepath -b 199.58.81.140 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.679ms 1: _gateway (192.168.1.1) 1.245ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.817ms 3: no reply …
# bastet $ tracepath -b 204.13.164.118 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 1.010ms 1: _gateway (192.168.1.1) 0.293ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 6.126ms 3: war-r21.tpnet.pl (80.50.19.25) 4.701ms asymm 4 4: win-b2-link.ip.twelve99.net (213.248.103.4) 14.664ms 5: be1300.ccr51.vie01.atlas.cogentco.com (130.117.14.181) 26.007ms asymm 7 6: be2974.ccr21.muc03.atlas.cogentco.com (154.54.58.5) 27.318ms asymm 9 7: be5516.ccr41.fra05.atlas.cogentco.com (154.54.62.121) 26.305ms asymm 8 8: be5340.ccr42.ams03.atlas.cogentco.com (154.54.62.210) 30.990ms asymm 9 9: be12194.ccr41.lon13.atlas.cogentco.com (154.54.56.93) 122.662ms asymm 13 10: be3042.ccr21.ymq01.atlas.cogentco.com (154.54.44.162) 129.524ms asymm 11 11: be3393.ccr31.bos01.atlas.cogentco.com (154.54.47.141) 125.119ms 12: be3600.ccr22.alb02.atlas.cogentco.com (154.54.0.221) 125.220ms asymm 10 13: be2718.ccr42.ord01.atlas.cogentco.com (154.54.7.129) 137.118ms asymm 10 14: be2717.ccr41.ord01.atlas.cogentco.com (154.54.6.221) 130.040ms asymm 10 15: be3802.ccr21.den01.atlas.cogentco.com (154.54.165.77) 144.294ms asymm 10 16: be5456.ccr32.slc01.atlas.cogentco.com (154.54.45.166) 185.868ms asymm 11 17: be5823.ccr21.sea02.atlas.cogentco.com (154.54.167.146) 199.296ms asymm 11 18: be5823.ccr21.sea02.atlas.cogentco.com (154.54.167.146) 194.850ms asymm 11 19: bastet.readthefinemanual.net (204.13.164.118) 200.905ms reached
# faravahar $ tracepath -b 216.218.219.41 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 1.493ms 1: _gateway (192.168.1.1) 0.774ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 7.927ms 3: no reply …
-----------------------------------------------------------------------
[2] Logs entries before the issue started: ----------------------------------------------------------------------- Oct 23 01:27:03 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 217.196.147.77:80 while fetching consensus directory. Oct 23 01:27:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 204.13.164.118:80 while fetching consensus directory. Oct 23 01:27:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 204.13.164.118:80 while fetching consensus directory. Oct 23 01:28:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 217.196.147.77:80 while fetching consensus directory. -----------------------------------------------------------------------
On 30/10/24 18:35, mpan wrote:
NOTE: this email has been written around 2024-10-30 19:00 UTC, about 2.5 hours ago. As I was testing stuff, suddenly I got flags and *some* traffic on my node. It also appeared back in atlas as mysteriously as it did disappear. Nonetheless I’m sending the email, hoping it is helpful, and will report back in a week.
Can you check to see if your relay is in a similar situation?
In particular, the situation to look for is "Tor process is still running fine from your perspective, but, relay-search (https://atlas.torproject.org/) says you are no longer running."
Hello, I see the same with my node.
Prompted by a fall in traffic I checked relay-search a few days ago and found the relay is no longer listed. Going through the logs it seems there is no new circuits being made since October 23. In Nyx I see at most a few incoming and outgoing connections.
The node is mpan11 92A125B9AB491AFC084E4257B552D0FB56090CB3, and this is the first time in about 10 years I experience anything like that.
If your relay is in this situation, the next step is to check your Tor logs, try to rule out other issues like firewall rules on your side, and then (if you're able) to start exploring traceroutes to the directory authority IP addresses vs other addresses. If you need more direct help, we can help you debug or answer other questions on #tor-relays on IRC.
Traceroutes from the node listed in [1]. From two other hosts (French, German VPS) all addresses except maatuska are accessible, and the final hops are similar. I asked a friend (same ISP, city) to ping all the addresses that don’t respond and they confirmed no ping responses. However, maatuska seems to respond to ICMP pings.
Based on Ooni results, it seems that your ISP (AS5617) began blocking the Tor directory authorities on October 22.
https://explorer.ooni.org/search?since=2024-10-01&until=2024-11-01&f...
Other relays hosted by the same ISP appear to have also lost consensus:
https://metrics.torproject.org/rs.html#search/as:AS5617
If that helps, on October 23 I see some entries[2] I don’t recall ever seeing. This did happen when my network connection died for about 10 minutes around 01:20. At 22 I had to reboot the system and since then the node reports 0 circuits. But that may be just a coincidence and the responses might’ve been ISP’s modem interfering in some way.
[1] Traceroutes, to 30 hops. ‘…’ indicates no more replies:
# moria1 $ tracepath -b 128.31.0.39 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.499ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 15.434ms 3: no reply …
# tor26 $ tracepath -b 217.196.147.77 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.359ms 1: _gateway (192.168.1.1) 3.782ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 3.833ms 3: war-r21.tpnet.pl (80.50.19.25) 2.383ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 12.699ms 5: as33891-ic-331887.ip.twelve99-cust.net (62.115.13.142) 16.619ms asymm 6 6: ae5-2058.slz10.core-backbone.com (81.95.2.50) 19.534ms asymm 9 7: core-backbone.conova.at (5.56.17.86) 17.129ms asymm 12 8: 149.3.164.138 (149.3.164.138) 18.783ms asymm 13 9: reliant-gw.noreply.org (185.69.162.222) 27.474ms asymm 13 10: tor.cypherpunks.eu (217.196.147.77) 30.815ms !H
# dizum $ tracepath -b 45.66.35.11 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.739ms 1: _gateway (192.168.1.1) 0.716ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.480ms 3: war-r21.tpnet.pl (80.50.19.25) 4.162ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 15.974ms 5: win-bb2-link.ip.twelve99.net (62.115.114.182) 22.121ms asymm 6 6: ffm-bb2-link.ip.twelve99.net (62.115.138.22) 26.017ms asymm 8 7: adm-bb2-link.ip.twelve99.net (62.115.137.222) 30.844ms asymm 9 8: no reply 9: novoserve-ic-354347.ip.twelve99-cust.net (62.115.188.167) 31.944ms asymm 8 10: no reply 11: no reply 12: no reply 13: tor.dizum.com (45.66.35.11) 34.349ms reached
# gabelmoo $ tracepath -b 131.188.40.189 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 2.812ms 1: _gateway (192.168.1.1) 1.120ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 11.639ms 3: no reply …
# dannenberg $ tracepath -b 193.23.244.244 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.563ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.746ms 3: war-r22.tpnet.pl (80.50.20.25) 6.182ms 4: win-b2-link.ip.twelve99.net (213.248.103.4) 13.986ms 5: win-bb1-link.ip.twelve99.net (62.115.114.184) 16.536ms asymm 6 6: mcn-b1-link.ip.twelve99.net (62.115.136.41) 20.223ms asymm 7 7: mcn-b3-link.ip.twelve99.net (62.115.127.30) 19.945ms 8: interlinkgmbh-ic-381330.ip.twelve99-cust.net (62.115.186.123) 18.466ms asymm 9 9: r1-str1-de.as5405.net (94.103.180.27) 62.334ms (This broken router returned corrupted payload) asymm 15 10: r4-fra2-de.as5405.net (94.103.180.11) 38.532ms (This broken router returned corrupted payload) asymm 14 11: r4-fra1-de.as5405.net (94.103.180.7) 36.582ms (This broken router returned corrupted payload) asymm 12 12: r3-ber1-de.as5405.net (94.103.180.2) 37.920ms asymm 11 13: cust-syseleven.r3-ber1-de.as5405.net (45.153.82.11) 38.488ms asymm 12 14: ae0-0.blu1-r2.de.syseleven.net (109.68.226.26) 39.453ms asymm 16 15: ae2-0.bak1-r1.syseleven.net (109.68.226.23) 41.234ms asymm 14 16: no reply 17: dannenberg.torauth.de (193.23.244.244) 34.575ms reached
# maatuska $ tracepath -b 171.25.193.9 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 3.200ms 1: _gateway (192.168.1.1) 0.674ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 8.933ms 3: war-r21.tpnet.pl (80.50.19.25) 5.954ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 12.257ms 5: win-bb2-link.ip.twelve99.net (62.115.114.182) 13.424ms asymm 6 6: ffm-bb2-link.ip.twelve99.net (62.115.138.22) 24.160ms asymm 9 7: kbn-bb6-link.ip.twelve99.net (62.115.114.94) 40.453ms asymm 10 8: kbn-b1-link.ip.twelve99.net (62.115.143.7) 35.856ms 9: globalconnect-ic-368141.ip.twelve99-cust.net (62.115.185.221) 37.018ms 10: 146.247.201.152 (146.247.201.152) 44.090ms asymm 9 11: no reply 12: no reply 13: 195.225.184.150 (195.225.184.150) 48.578ms asymm 15 14: rs2.dfri.net (171.25.193.225) 46.928ms asymm 16 15: rs2.dfri.net (171.25.193.225) 48.011ms pmtu 1420 15: no reply …
# longclaw $ tracepath -b 199.58.81.140 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.679ms 1: _gateway (192.168.1.1) 1.245ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.817ms 3: no reply …
# bastet $ tracepath -b 204.13.164.118 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 1.010ms 1: _gateway (192.168.1.1) 0.293ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 6.126ms 3: war-r21.tpnet.pl (80.50.19.25) 4.701ms asymm 4 4: win-b2-link.ip.twelve99.net (213.248.103.4) 14.664ms 5: be1300.ccr51.vie01.atlas.cogentco.com (130.117.14.181) 26.007ms asymm 7 6: be2974.ccr21.muc03.atlas.cogentco.com (154.54.58.5) 27.318ms asymm 9 7: be5516.ccr41.fra05.atlas.cogentco.com (154.54.62.121) 26.305ms asymm 8 8: be5340.ccr42.ams03.atlas.cogentco.com (154.54.62.210) 30.990ms asymm 9 9: be12194.ccr41.lon13.atlas.cogentco.com (154.54.56.93) 122.662ms asymm 13 10: be3042.ccr21.ymq01.atlas.cogentco.com (154.54.44.162) 129.524ms asymm 11 11: be3393.ccr31.bos01.atlas.cogentco.com (154.54.47.141) 125.119ms 12: be3600.ccr22.alb02.atlas.cogentco.com (154.54.0.221) 125.220ms asymm 10 13: be2718.ccr42.ord01.atlas.cogentco.com (154.54.7.129) 137.118ms asymm 10 14: be2717.ccr41.ord01.atlas.cogentco.com (154.54.6.221) 130.040ms asymm 10 15: be3802.ccr21.den01.atlas.cogentco.com (154.54.165.77) 144.294ms asymm 10 16: be5456.ccr32.slc01.atlas.cogentco.com (154.54.45.166) 185.868ms asymm 11 17: be5823.ccr21.sea02.atlas.cogentco.com (154.54.167.146) 199.296ms asymm 11 18: be5823.ccr21.sea02.atlas.cogentco.com (154.54.167.146) 194.850ms asymm 11 19: bastet.readthefinemanual.net (204.13.164.118) 200.905ms reached
# faravahar $ tracepath -b 216.218.219.41 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 1.493ms 1: _gateway (192.168.1.1) 0.774ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 7.927ms 3: no reply …
[2] Logs entries before the issue started:
Oct 23 01:27:03 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 217.196.147.77:80 while fetching consensus directory. Oct 23 01:27:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 204.13.164.118:80 while fetching consensus directory. Oct 23 01:27:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 204.13.164.118:80 while fetching consensus directory. Oct 23 01:28:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 217.196.147.77:80 while fetching consensus directory.
Have you tried checking what happens when you access the directory's port using a web browser or curl?
curl -I http://217.196.147.77:80
Where do you get redirected?
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Have you tried checking what happens when you access the directory's port using a web browser or curl?
curl -I http://217.196.147.77:80
Where do you get redirected?
Back then, no. I noticed the redirect only when investigating the issue now. For completness, *now* both Firefox and curl return an empty response.
But please keep in mind the 302 may be a red harring. It likely happened around the moment ISP’s box was booting. It isn’t supposed to have any such feature, but it’s not hard to imagine it has an undisclosed service (a captive portal etc.) that gets activated by accident during boot and then is shut down, leading to a spurious 302. Hence I added it only as a side note.
For now the node seems to be running fine. The addresses I mentioned earlier to be inaccessible (moria1, gabelmoo, maatuska, longclaw, faravahar) remain unresponsive from my end, last hop (83.1.5.194) being inside ISP’s infra. maatuska still responds to ICMP pings, but not TCP (including tracepath or connections): however the issue seems to be on matuuska’s end.
Hello dear MPAN,
thank you for being so detailed with all the trace-routes.
Seems like your upstream provider is blocking traffic to:
moria1 gabelmoo longclaw faravahar
This should never happen, can you contact your provider, and the last online servers e-mail address (just WHOIS the IP-Address and there should be a NOC e-mail address), and politely ask why they drop your traffic?
On a bright side, your relay is back online on Atlas/Metrics, even though the graph shows that it was apparently "offline" for more than 6 days.
Thank you for reporting this.
-GH
On Wednesday, October 30th, 2024 at 10:35 PM, mpan tor-1qnuaylp@mpan.pl wrote:
NOTE: this email has been written around 2024-10-30 19:00 UTC, about 2.5 hours ago. As I was testing stuff, suddenly I got flags and some traffic on my node. It also appeared back in atlas as mysteriously as it did disappear. Nonetheless I’m sending the email, hoping it is helpful, and will report back in a week.
Can you check to see if your relay is in a similar situation?
In particular, the situation to look for is "Tor process is still running fine from your perspective, but, relay-search (https://atlas.torproject.org/) says you are no longer running."
Hello, I see the same with my node.
Prompted by a fall in traffic I checked relay-search a few days ago and found the relay is no longer listed. Going through the logs it seems there is no new circuits being made since October 23. In Nyx I see at most a few incoming and outgoing connections.
The node is mpan11 92A125B9AB491AFC084E4257B552D0FB56090CB3, and this is the first time in about 10 years I experience anything like that.
If your relay is in this situation, the next step is to check your Tor logs, try to rule out other issues like firewall rules on your side, and then (if you're able) to start exploring traceroutes to the directory authority IP addresses vs other addresses. If you need more direct help, we can help you debug or answer other questions on #tor-relays on IRC.
Traceroutes from the node listed in [1]. From two other hosts (French, German VPS) all addresses except maatuska are accessible, and the final hops are similar. I asked a friend (same ISP, city) to ping all the addresses that don’t respond and they confirmed no ping responses. However, maatuska seems to respond to ICMP pings.
If that helps, on October 23 I see some entries[2] I don’t recall ever seeing. This did happen when my network connection died for about 10 minutes around 01:20. At 22 I had to reboot the system and since then the node reports 0 circuits. But that may be just a coincidence and the responses might’ve been ISP’s modem interfering in some way.
[1] Traceroutes, to 30 hops. ‘…’ indicates no more replies:
# moria1 $ tracepath -b 128.31.0.39 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.499ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 15.434ms 3: no reply …
# tor26 $ tracepath -b 217.196.147.77 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.359ms 1: _gateway (192.168.1.1) 3.782ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 3.833ms 3: war-r21.tpnet.pl (80.50.19.25) 2.383ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 12.699ms 5: as33891-ic-331887.ip.twelve99-cust.net (62.115.13.142) 16.619ms asymm 6 6: ae5-2058.slz10.core-backbone.com (81.95.2.50) 19.534ms asymm 9 7: core-backbone.conova.at (5.56.17.86) 17.129ms asymm 12 8: 149.3.164.138 (149.3.164.138) 18.783ms asymm 13 9: reliant-gw.noreply.org (185.69.162.222) 27.474ms asymm 13 10: tor.cypherpunks.eu (217.196.147.77) 30.815ms !H
# dizum $ tracepath -b 45.66.35.11 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.739ms 1: _gateway (192.168.1.1) 0.716ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.480ms 3: war-r21.tpnet.pl (80.50.19.25) 4.162ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 15.974ms 5: win-bb2-link.ip.twelve99.net (62.115.114.182) 22.121ms asymm 6 6: ffm-bb2-link.ip.twelve99.net (62.115.138.22) 26.017ms asymm 8 7: adm-bb2-link.ip.twelve99.net (62.115.137.222) 30.844ms asymm 9 8: no reply 9: novoserve-ic-354347.ip.twelve99-cust.net (62.115.188.167) 31.944ms asymm 8 10: no reply 11: no reply 12: no reply 13: tor.dizum.com (45.66.35.11) 34.349ms reached
# gabelmoo $ tracepath -b 131.188.40.189 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 2.812ms 1: _gateway (192.168.1.1) 1.120ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 11.639ms 3: no reply …
# dannenberg $ tracepath -b 193.23.244.244 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.563ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.746ms 3: war-r22.tpnet.pl (80.50.20.25) 6.182ms 4: win-b2-link.ip.twelve99.net (213.248.103.4) 13.986ms 5: win-bb1-link.ip.twelve99.net (62.115.114.184) 16.536ms asymm 6 6: mcn-b1-link.ip.twelve99.net (62.115.136.41) 20.223ms asymm 7 7: mcn-b3-link.ip.twelve99.net (62.115.127.30) 19.945ms 8: interlinkgmbh-ic-381330.ip.twelve99-cust.net (62.115.186.123) 18.466ms asymm 9 9: r1-str1-de.as5405.net (94.103.180.27) 62.334ms (This broken router returned corrupted payload) asymm 15 10: r4-fra2-de.as5405.net (94.103.180.11) 38.532ms (This broken router returned corrupted payload) asymm 14 11: r4-fra1-de.as5405.net (94.103.180.7) 36.582ms (This broken router returned corrupted payload) asymm 12 12: r3-ber1-de.as5405.net (94.103.180.2) 37.920ms asymm 11 13: cust-syseleven.r3-ber1-de.as5405.net (45.153.82.11) 38.488ms asymm 12 14: ae0-0.blu1-r2.de.syseleven.net (109.68.226.26) 39.453ms asymm 16 15: ae2-0.bak1-r1.syseleven.net (109.68.226.23) 41.234ms asymm 14 16: no reply 17: dannenberg.torauth.de (193.23.244.244) 34.575ms reached
# maatuska $ tracepath -b 171.25.193.9 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 3.200ms 1: _gateway (192.168.1.1) 0.674ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 8.933ms 3: war-r21.tpnet.pl (80.50.19.25) 5.954ms asymm 4 4: win-b2-link.ip.twelve99.net (62.115.153.224) 12.257ms 5: win-bb2-link.ip.twelve99.net (62.115.114.182) 13.424ms asymm 6 6: ffm-bb2-link.ip.twelve99.net (62.115.138.22) 24.160ms asymm 9 7: kbn-bb6-link.ip.twelve99.net (62.115.114.94) 40.453ms asymm 10 8: kbn-b1-link.ip.twelve99.net (62.115.143.7) 35.856ms 9: globalconnect-ic-368141.ip.twelve99-cust.net (62.115.185.221) 37.018ms 10: 146.247.201.152 (146.247.201.152) 44.090ms asymm 9 11: no reply 12: no reply 13: 195.225.184.150 (195.225.184.150) 48.578ms asymm 15 14: rs2.dfri.net (171.25.193.225) 46.928ms asymm 16 15: rs2.dfri.net (171.25.193.225) 48.011ms pmtu 1420 15: no reply …
# longclaw $ tracepath -b 199.58.81.140 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 0.679ms 1: _gateway (192.168.1.1) 1.245ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 5.817ms 3: no reply …
# bastet $ tracepath -b 204.13.164.118 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 1.010ms 1: _gateway (192.168.1.1) 0.293ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 6.126ms 3: war-r21.tpnet.pl (80.50.19.25) 4.701ms asymm 4 4: win-b2-link.ip.twelve99.net (213.248.103.4) 14.664ms 5: be1300.ccr51.vie01.atlas.cogentco.com (130.117.14.181) 26.007ms asymm 7 6: be2974.ccr21.muc03.atlas.cogentco.com (154.54.58.5) 27.318ms asymm 9 7: be5516.ccr41.fra05.atlas.cogentco.com (154.54.62.121) 26.305ms asymm 8 8: be5340.ccr42.ams03.atlas.cogentco.com (154.54.62.210) 30.990ms asymm 9 9: be12194.ccr41.lon13.atlas.cogentco.com (154.54.56.93) 122.662ms asymm 13 10: be3042.ccr21.ymq01.atlas.cogentco.com (154.54.44.162) 129.524ms asymm 11 11: be3393.ccr31.bos01.atlas.cogentco.com (154.54.47.141) 125.119ms 12: be3600.ccr22.alb02.atlas.cogentco.com (154.54.0.221) 125.220ms asymm 10 13: be2718.ccr42.ord01.atlas.cogentco.com (154.54.7.129) 137.118ms asymm 10 14: be2717.ccr41.ord01.atlas.cogentco.com (154.54.6.221) 130.040ms asymm 10 15: be3802.ccr21.den01.atlas.cogentco.com (154.54.165.77) 144.294ms asymm 10 16: be5456.ccr32.slc01.atlas.cogentco.com (154.54.45.166) 185.868ms asymm 11 17: be5823.ccr21.sea02.atlas.cogentco.com (154.54.167.146) 199.296ms asymm 11 18: be5823.ccr21.sea02.atlas.cogentco.com (154.54.167.146) 194.850ms asymm 11 19: bastet.readthefinemanual.net (204.13.164.118) 200.905ms reached
# faravahar $ tracepath -b 216.218.219.41 1?: [LOCALHOST] pmtu 1500 1: _gateway (192.168.1.1) 1.493ms 1: _gateway (192.168.1.1) 0.774ms 2: war-bng10.neo.tpnet.pl (83.1.5.194) 7.927ms 3: no reply …
[2] Logs entries before the issue started:
Oct 23 01:27:03 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 217.196.147.77:80 while fetching consensus directory. Oct 23 01:27:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 204.13.164.118:80 while fetching consensus directory. Oct 23 01:27:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 204.13.164.118:80 while fetching consensus directory. Oct 23 01:28:55 null Tor[368865]: Received http status code 302 ("Moved Temporarily") from server 217.196.147.77:80 while fetching consensus directory.
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays@lists.torproject.org
-
George Hartley -
mpan -
pasture_clubbed242＠simplelogin.com -
Roger Dingledine -
tor-relays+tor-relays＠queer.cat