Update on attack in progress post 0.4.17 upgrade
I lost two HSDIR's to attacks pre-0.4.17 and one was under attack as 0.4.17 was released and upgraded. These are my observations: First of all, in the post BEGIN_DIR new wave of attacks first reported by Forest, there were apparently two types. I will call them: Type A: Disproportionate ingress - very high traffic reported with ~6:1 ingress to output ratio Type B: Disproportionate output - very high traffic reported with ~1:2 ingress to output ratio, very high instances of circuit creation "Your computer is too slow..." messages, often 400,000 or more per minute. Note that this attack poisons ipset-style tor firewall rules. I will report now a new type: Type C: Traffic shutdown - traffic falls to very low levels with 1-2 MiB/s ingress and sub 1MiB/s output, medium to low cpu usage, very high number of "relay unresponsive" messages, affects other parts of the OS (can see slowdown in ssh connections to the machine), but does not trigger connection-attempt firewall rules. Observations: - One relay was under a type B attack at the time of my upgrade to 0.4.17. This relay had no traffic limits before the upgrade, at time of upgrade 12MiB/s rate limits were added. - Pre-upgrade attack traffic was 18MiB/s ingress, ~40MiB/s output - Post-upgrade and with traffic limits in place, traffic was extremely variable, 8MiB/s ingress, 4-20 MiB/s output. The relay was clearly still under Type B attack, but the effectiveness was significantly reduced. - The attach SHIFTED approximately 10 minutes after upgrade to the newly seen type C - The Type C attack caused repeat "relay unresponsive" log messages, dropping all traffic precipitously. - Relay unresponsive / resolved messages increased in frequency until traffic dropped to zero, tor's cpu usage was not terribly high - spiking in little spurts, but often as low ar 30%. - After the shift to Type C, my ipset firewall rules were re-enabled to no effect - no rules were triggered - Relay unresponsiveness ultimately caused loss of HSDIR. Attack ceased immediately on loss of HSDIR. Analysis: - As I have never seen a type A attack, I cannot myself say A and B are not similar symptoms of the same vector. That said, there are two to three independent vectors, and the current mitigations in 0.4.17 are insufficient to protect HSDIR. - The attacks are AGILE, switching mid-attack to different vectors. - It is unknown how many more tor protocol weaknesses are currently queued by the attacker, but given the sophistication of the attacks, it should be considered as given there are more waiting in the wings. - While it cannot be discounted this is merely a DoS against hidden services, this attacker is sophisticated enough for this to be a deanonymizing and tracking attack on hidden service operators and users. Attacks against friendly HSDIR nodes should be considered to be likely only half the threat, in conjunction with the attacker's own compromised HSDIR nodes. Recommendations: 1) Get data now on what nodes are retaining HSDIR and not being attacked. These should be considered highly suspect nodes. 2) WARN WARN WARN - the responsible action is for the Tor project to officially warn that Tor Onion Hidden Services are currently AT RISK FOR DEANONYMIZATION AND CONNECTION TRACKING. 3) The Tor Project should consider relaxing rules for giving out HSDIR to keep the population of friendly HSDIR nodes very high. This will attract more attacks, which will spread the attack surface out more and tax the attacker. Currently the attacker is rewarded for a two hour attack by days of HSDIR loss in the target. Once granted, consider making HSDIR sticky for one week or more. 4) Rate limits on every single tor network transaction must be implemented and this should be considered to be an emergent threat. 5) When any attack mitigation is released, the mitigation code MUST LOG THE IP ADDRESS OF NODES THAT TRIGGER THE MITIGATIONS. BEGIN_DIR rate limit hitting should have triggered informational log entries with the node fingerprint and IP address. If you do not start treating this as a sustained, ongoing, coordinated attack and start taking more global affirmative mitigation measures, you are going to be permanently operating in catch-up mode. LOG IT! THE LACK OF ATTACK LOGGING IS IRRESPONSIBLE!
Am 14.11.26 um 08:49 schrieb Red Oaive via tor-relays:
Type C: Traffic shutdown - traffic falls to very low levels with 1-2 MiB/s ingress and sub 1MiB/s output, medium to low cpu usage, very high number of "relay unresponsive" messages, affects other parts of the OS (can see slowdown in ssh connections to the machine), but does not trigger connection-attempt firewall rules.
I have a situation where all 4 cores are 100% used and traffic falls from 1MByte/s to 30kByte/s. Now CPU is nearly nothing. Is that the same or another type of attack? -- Gruß Marco Muell und Spam bitte an abfalleimer2002@stinkedores.dorfdsl.de
Really weird. Both of my tor relay ssh are super laggy. Even lagging when writing but all the stats looks somewhat normal. Takes like 30secs to get vnstat readings on both. Also im transferring alot less than normal on both. Third vps with other services are fine.
_______________________________________________ tor-relays mailing list -- tor-relays@lists.torproject.org To unsubscribe send an email to tor-relays-leave@lists.torproject.org
Am 14.11.26 um 13:54 schrieb kullikovana via tor-relays:
Really weird. Both of my tor relay ssh are super laggy. Even lagging when writing but all the stats looks somewhat normal. Takes like 30secs to get vnstat readings on both. Also im transferring alot less than normal on both. Third vps with other services are fine.
Works fine for me. Machine is an Intel Atom with 2 GB RAM and Slackware Linux. -- Gruß Marco Junk-Mail bitte an trashcan@stinkedores.dorfdsl.de
On 2026-11-14 10:44, Marco Moock via tor-relays wrote:
Am 14.11.26 um 08:49 schrieb Red Oaive via tor-relays:
Type C: Traffic shutdown - traffic falls to very low levels with 1-2 MiB/s ingress and sub 1MiB/s output, medium to low cpu usage, very high number of "relay unresponsive" messages, affects other parts of the OS (can see slowdown in ssh connections to the machine), but does not trigger connection-attempt firewall rules.
I have a situation where all 4 cores are 100% used and traffic falls from 1MByte/s to 30kByte/s. Now CPU is nearly nothing.
Is that the same or another type of attack?
This sounds a lot like what I saw with my relay that had been under attack when I upgraded. When the attack shifted from B -> C. My report of medium-to-low CPU usage is somewhat incomplete. I saw moments some cores would spike, but then it would drop to almost nothing. I was too busy watching other dynamics to pay really close attention to the CPU, but my overall classification of this "Type C" attack as "medium to low cpu usage" is averaging what I saw over time. I think we should lump your report into Type C. A better overall description of what I call "Type C" is: Type C: Traffic shutdown - traffic falls to very low levels with 1-2 MiB/s ingress and sub 1MiB/s output, CPU usage averages as medium to low but with periods of pegged spikes, very high number of "relay unresponsive" messages, affects other parts of the OS (can see slowdown in ssh connections to the machine), but does not trigger connection-attempt firewall rules. The shift from BEGIN_DIR (maybe call this Type Zero) to the new Types A and B, then the immediate on-the-fly shift from B to the previously unseen type C in the middle of an attack as type B mitigations were installed suggests an adversary that is well prepared for mitigations with a stockpile of weaknesses, and likely an AI-guided attack. This has the hallmarks of Apex AI or similar technology.
On Nov 14, 2026, at 11:50 AM, Red Oaive via tor-relays <tor-relays@lists.torproject.org> wrote: The shift from BEGIN_DIR (maybe call this Type Zero) to the new Types A and B, then the immediate on-the-fly shift from B to the previously unseen type C in the middle of an attack as type B mitigations were installed suggests an adversary that is well prepared for mitigations with a stockpile of weaknesses, and likely an AI-guided attack.
As I've said before and told the Tor project: the problem is not the particular DOS vector (you can NEVER make DOS impossible for a public service against a well resourced adversary, you can only make it hard) but the attacker's objectives. This is HSDir shaping and until a router which loses its HSDir flag, even if it is because it is down, instead keeps its flag and place in the HSDir hash ring until 12:00 UTC the next day the attacks are going to continue. There are probably ~1/2 a dozen to a dozen hidden services where the attacker is attempting to deanonymize visitors.
Further to the below report, I had 0.4.17 machines come under further attach earlier today. The attack started with what I labelled type B (my type classifications are quoted at the end) which is: disproportionate-output attack that appears to have high volumes of circuit creations. My earlier report of 14 Nov described a machine coming under attack just before I ugraded it to 0.4.17, and post upgrade it seemed to be more resistant to the type B attack. This does not now seem to be the case. The implementation of RelayBandwithRate in real time during the earlier attack is likely what caused that attack to switch from Type B -> Type C. Today's attack came on significantly higher capacity machines that had no bandwidth limits and appear to be classic Type B in the beginning with newly encountered behaviour at the end that may represent a novel attack type. I also reiterate all recommendations made 14 Nov, also quoted below. Today's attack SoE: Minute 000: Beginning of attack, first circuit creation warning Minute 000-060: Circuit creation warnings slowly increase from 3000/s at start to >12000/s over next hour Minute 060-133: Circuit creation warnings slowly DECREASE from peak of
12000/s to 1100/s Minute 133: Bug warning appears in log, Minute 137: Last circuit creation warning. Log is mostly silent from now until... Minute 185: General overload -> Ntor dropped (45690340) fraction 60.7269% is above threshold of 0.5000% Sometime between minute 185 and 202 is when the relay lost all flags. Minute 202: Tor was manually restarted, exactly 40 more Bug warnings occur over two seconds as Tor shuts down.
At the peak of the attack were ~12000 circuit creation warning messages per second: [warn] Your computer is too slow to handle this many circuit creation requests! Please consider using the MaxAdvertisedBandwidth config option or choosing a more restricted exit policy. [738123 similar message(s) suppressed in last 60 seconds] Analysis: Whatever happened to finally erase HSDIR appeared to occur after minute 137+, after the classic Type B attack wound down. It is possible that the attack controller saw the machine still had RUNNING at the next hourly consensus after the Type B attack was at full capacity (perhaps owing to this machine having significantly higher CPU and bandwidth capabilities compared to others of mine that have come under attack) and switched to a previously unknown type of attack, triggering the Bug message in an otherwise mostly silent log. New to today's attack were the 41 BUG reports appearing in the log as below: Nov 16 XX:YY:13.000 [warn] tor_bug_occurred_(): Bug: ../src/core/or/dos.c:745: conn_update_on_close: Non-fatal assertion !(stats->concurrent_count == 0) failed. (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: Tor 0.4.9.17: Non-fatal assertion !(stats->concurrent_count == 0) failed in conn_update_on_close at ../src/core/or/dos.c:745. Stack trace: (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(log_backtrace_impl+0x5b) [0x557ff60e5d8b] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(tor_bug_occurred_+0x16a) [0x557ff60f118a] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(+0x1a3c31) [0x557ff6196c31] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(+0x72258) [0x557ff6065258] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /lib/x86_64-linux-gnu/libevent-2.1.so.7(+0x215a5) [0x7fa6bd8a25a5] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /lib/x86_64-linux-gnu/libevent-2.1.so.7(event_base_loop+0x49f) [0x7fa6bd8a2c3f] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(do_main_loop+0xf5) [0x557ff6066c75] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(tor_run_main+0x1e5) [0x557ff60620c5] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(tor_main+0x5d) [0x557ff605e2bd] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(main+0x1d) [0x557ff605de4d] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /lib/x86_64-linux-gnu/libc.so.6(+0x2724a) [0x7fa6bcf8924a] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x7fa6bcf89305] (on Tor 0.4.9.17 ) Nov 16 XX:YY:13.000 [warn] Bug: /usr/bin/tor(_start+0x21) [0x557ff605dea1] (on Tor 0.4.9.17 ) On 2026-11-14 03:49, Red Oaive via tor-relays wrote:
I lost two HSDIR's to attacks pre-0.4.17 and one was under attack as 0.4.17 was released and upgraded. These are my observations:
First of all, in the post BEGIN_DIR new wave of attacks first reported by Forest, there were apparently two types. I will call them: Type A: Disproportionate ingress - very high traffic reported with ~6:1 ingress to output ratio Type B: Disproportionate output - very high traffic reported with ~1:2 ingress to output ratio, very high instances of circuit creation "Your computer is too slow..." messages, often 400,000 or more per minute. Note that this attack poisons ipset-style tor firewall rules.
I will report now a new type: Type C: Traffic shutdown - traffic falls to very low levels with 1-2 MiB/s ingress and sub 1MiB/s output, medium to low cpu usage, very high number of "relay unresponsive" messages, affects other parts of the OS (can see slowdown in ssh connections to the machine), but does not trigger connection-attempt firewall rules.
Observations: - One relay was under a type B attack at the time of my upgrade to 0.4.17. This relay had no traffic limits before the upgrade, at time of upgrade 12MiB/s rate limits were added. - Pre-upgrade attack traffic was 18MiB/s ingress, ~40MiB/s output - Post-upgrade and with traffic limits in place, traffic was extremely variable, 8MiB/s ingress, 4-20 MiB/s output. The relay was clearly still under Type B attack, but the effectiveness was significantly reduced. - The attach SHIFTED approximately 10 minutes after upgrade to the newly seen type C - The Type C attack caused repeat "relay unresponsive" log messages, dropping all traffic precipitously. - Relay unresponsive / resolved messages increased in frequency until traffic dropped to zero, tor's cpu usage was not terribly high - spiking in little spurts, but often as low ar 30%. - After the shift to Type C, my ipset firewall rules were re-enabled to no effect - no rules were triggered - Relay unresponsiveness ultimately caused loss of HSDIR. Attack ceased immediately on loss of HSDIR.
Analysis: - As I have never seen a type A attack, I cannot myself say A and B are not similar symptoms of the same vector. That said, there are two to three independent vectors, and the current mitigations in 0.4.17 are insufficient to protect HSDIR. - The attacks are AGILE, switching mid-attack to different vectors. - It is unknown how many more tor protocol weaknesses are currently queued by the attacker, but given the sophistication of the attacks, it should be considered as given there are more waiting in the wings. - While it cannot be discounted this is merely a DoS against hidden services, this attacker is sophisticated enough for this to be a deanonymizing and tracking attack on hidden service operators and users. Attacks against friendly HSDIR nodes should be considered to be likely only half the threat, in conjunction with the attacker's own compromised HSDIR nodes.
Recommendations: 1) Get data now on what nodes are retaining HSDIR and not being attacked. These should be considered highly suspect nodes. 2) WARN WARN WARN - the responsible action is for the Tor project to officially warn that Tor Onion Hidden Services are currently AT RISK FOR DEANONYMIZATION AND CONNECTION TRACKING. 3) The Tor Project should consider relaxing rules for giving out HSDIR to keep the population of friendly HSDIR nodes very high. This will attract more attacks, which will spread the attack surface out more and tax the attacker. Currently the attacker is rewarded for a two hour attack by days of HSDIR loss in the target. Once granted, consider making HSDIR sticky for one week or more. 4) Rate limits on every single tor network transaction must be implemented and this should be considered to be an emergent threat. 5) When any attack mitigation is released, the mitigation code MUST LOG THE IP ADDRESS OF NODES THAT TRIGGER THE MITIGATIONS. BEGIN_DIR rate limit hitting should have triggered informational log entries with the node fingerprint and IP address. If you do not start treating this as a sustained, ongoing, coordinated attack and start taking more global affirmative mitigation measures, you are going to be permanently operating in catch-up mode. LOG IT! THE LACK OF ATTACK LOGGING IS IRRESPONSIBLE!
_______________________________________________ tor-relays mailing list -- tor-relays@lists.torproject.org To unsubscribe send an email to tor-relays-leave@lists.torproject.org
Am 16.11.26 um 16:23 schrieb Red Oaive via tor-relays:
0) failed in conn_update_on_close at ../src/core/or/dos.c:745. Stack trace: (on Tor 0.4.9.17 )
I can confirm this. Is there already a bug reported? I cannot find something if I search for that string "rc/core/or/dos.c:745". -- Gruß Marco Muell und Spam bitte an abfalleimer2002@stinkedores.dorfdsl.de
Marco Moock via tor-relays:
Am 16.11.26 um 16:23 schrieb Red Oaive via tor-relays:
0) failed in conn_update_on_close at ../src/core/or/dos.c:745. Stack trace: (on Tor 0.4.9.17 )
I can confirm this.
Is there already a bug reported?
I cannot find something if I search for that string "rc/core/or/dos.c:745".
Yes, it's https://gitlab.torproject.org/tpo/core/tor/-/work_items/41488, I believe. Georg
_______________________________________________ tor-relays mailing list -- tor-relays@lists.torproject.org To unsubscribe send an email to tor-relays-leave@lists.torproject.org
**Additional observations of `conn_update_on_close` BUG on Tor 0.4.9.17** I operate three non-exit Tor relays and can confirm the same non-fatal assertion discussed above: `../src/core/or/dos.c:745: conn_update_on_close: Non-fatal assertion !(stats->concurrent_count == 0) failed.` Here are my observations: - **Relay 1:** Oracle Cloud, AMD x86_64, Ubuntu 26.04.1 LTS, kernel 7.0.0-1016-oracle. No occurrences in the November 16–17 logs. - **Relay 2:** Hetzner, ARM64, Tor 0.4.9.17. **13 occurrences** on November 16, none observed in the November 17 log at the time of inspection. - **Relay 3:** Oracle Cloud, ARM64, Tor 0.4.9.17. **21 occurrences** on November 16 and another three on November 17. **A particularly interesting observation:** Relays 2 and 3 logged exactly the same assertion at **2026-11-16 14:22:25 UTC**, despite running on independent servers at different hosting providers. Both ARM64 relays maintained Tor uptime and continued handling traffic. Our independent monitoring did not reveal an obvious shared performance collapse in the historical five-minute snapshots surrounding that event. Higher-resolution telemetry is available, but has not yet been fully correlated. Both ARM64 relays receive IPv6 connections, whereas the x86_64 relay currently reports no IPv6 connections. This is a potentially relevant difference, not evidence of causation. The relays also differ in OS/kernel configuration, available resources and workload. Separately, the x86_64 relay experienced a BEGIN_DIR storm followed by a severe reduction in traffic and active circuits, broadly resembling the Type C symptoms discussed in this thread, **without generating this assertion**. I therefore think the assertion and the traffic-collapse symptoms should be investigated separately until a causal relationship can be demonstrated. I see that issue #41488 has been mentioned. Would the developers find the synchronized timestamps, complete stack traces, and time-aligned monitoring data from these independent relays useful for investigating it?
Am 17.11.26 um 16:43 schrieb khankrum359--- via tor-relays:
Both ARM64 relays receive IPv6 connections, whereas the x86_64 relay currently reports no IPv6 connections.
It is reachable via IPv6? Did you try to manually initiate a connection to it, e.g. by pinning it as an entry? -- Gruß Marco Muell und Spam bitte an abfalleimer2002@stinkedores.dorfdsl.de
participants (6)
-
Georg Koppen -
khankrum359@gmail.com -
kullikovana -
Marco Moock -
Nick Weaver -
Red Oaive