@Enkidu
As an user of your filtering script, I want to first say thank you for maintaining the script!
The idea that all relays must be able to connect to other relays any time
and in any shape or form they choose can not exist in real world of DDoS mitigation.
I totally agree, however I want to also add a note here regarding privacy: If an adversary can nudge a user to choose some particular circuit / relay and avoid other circuits, they can break the privacy guarantee of Tor network. Aggressive automated rate limiting might provide adversaries with this opportunity, as an adversary might find ways to "trigger" the filtering and cause more circuits between "good" relays to be blocked, and clients will automatically fall back to circults that go through two or more malicious relays operated by the adversary (leading to deanonymization).
This attack surface is a pure theoretical speculation and probably not happening right now. Still, at a high level we need to be careful when designing automated filtering mechanisms, and I suggest erring on the more conservative side. E.g., 5 incoming connections per relay instance, so if an IP has 4 or 8 relay instances it can have 4*5=20 or 8*5=40 connections. (Of course, the quota should depend on flags, so an attacker can't just spin up 8 new relays per IP.)
Also, it might be good to have a more conservative (higher) rate limit by default and let users turn it down as necessary. -- Danny
On Tue, Feb 7, 2023 at 8:34 AM Chris Enkidu-6 tor@wcbsecurity.com wrote:
@nusenu
Thank you very much for taking the time to help me understand things better. I can use all the help I can get.
> You can also not be sure whether it is an actual authenticatedrelay to relay > connection or a client to relay connection just by looking at the source IP.
> In a vanilla current tor version it is not possible to use a torclient > to connect via an exit to another relay's ORPort so this is very unlikely.
> An exit can share it's source IP with tor clients for examplebehind NAT
Sorry I'm confused here. Those statements seem to contradict each other or there's something here I misunderstood. Even if that happens, why would a client connect directly to an Exit and get the Exit to connect to another relay or Guard using the Exit's IP address? Wouldn't the same protection you mentioned kick in and stop a client?
> I was not able to follow you there. I was unable to find any exitrelay that > has more than one ORPort on IPv4 (I identify relays by fingerprint not by IP address).
Sorry, I should have been more careful in using my words. In the world of firewalls and iptables we identify connections by IPAddress:port pairs hence my inaccurate wording. A relay has only one ORPort but an IP address can have 4.
> I think there is a misunderstanding there. One important point totake away: > Relays do not chose the next hop in a circuit, tor clients do. > So if my tor clients picks a circuit with these relays: > client -> A -> B -> C -> example.com > and B can not reach C, B can not say "oh I can not reach C I'll pick D instead for you",
I understand that and I think it might be my fault again for not being clear. It doesn't matter who makes that choice. What matters is that the choice is being made. My understanding is that the moment you open your Tor Browser, it starts establishing connections and building multiple circuits and keeps them in a pool.
In the old days of vidalia you could actually see that in real time. Many established circuits were just sitting there waiting. Each time you try to browse a page, Tor doesn't say, oh let's find a relay. It picks a circuit from a pool of already Established ones. If Tor tries to build a circuit with a relay and it's unable to, it will try another one that works and keeps it in the pool for when it's needed. Again, I may be wrong. Feel free to tell me if I am.
If that wasn't the case, any time I rebooted, someone would scream for being disconnected from the Internet. Evey single relay is important but not that important.
> Blocking relay to relay communication should not be done > as it breaks a core assumption of the tor network (every relay cantalk > to all other relays ORPorts)
Okay, now let's be clear about DDoS mitigations. There's no such thing as DDoS mitigation without rate limiting. Both my rules and @toralf rules and frankly any other rules that anyone could come up with by themselves, includes rate limiting.This means that at any given time, someone, somewhere is not allowed to connect due to their behaviour.
The idea that all relays must be able to connect to other relays any time and in any shape or form they choose can not exist in real world of DDoS mitigation. Right at this moment I have about 1600 relays that have established connections to me and I probably have as many connections to other relays and my relay is happily operating at about its Max Advertised Bandwidth. Do I really need to keep 6000 relays in my allow list and let them do whatever they want? Giving 6000 servers a free reign on your system is unheard of in the security world.
DDoS mitigation like anything else in life is about compromise. Ideals are sacrificed in favour of "good to haves" and "good to haves" are sacrificed in favour of survival. Otherwise you'd be a part of the currently [over 2000 relays that are overloaded]( https://github.com/Enkidu-6/tor-relay-lists/blob/main/overloaded.txt) and passing it along.
When I'm attacked, it's not just about me. I'm relaying that attack to the next relay and they relay that to the next one. So the idea that I should accept the attack when it's coming from another relay is simply unacceptable.
Again I'd like to be very clear, this is not about blocking relays, it's about rate limiting. They do get to connect but at a reasonable rate. For example, there's no justifiable reason for a relay to try to connect to another relay at a rate of 10 times a second for 15 minutes straight.
So if for example we say a relay can have two Established connections to me, we're not blocking them. If they do need the connection, they use it. If they close a connection and at some point they need one, they can open another one. But they can't have 10 connections at the same time.
Again, My goal here with my questions is to find a way to keep the balance of that compromise in our favour. But thinking that we can go on without making those compromises would be naive.
Again thanks for reading and thank you for your time and response.
On 2/6/2023 6:17 PM, nusenu wrote:
Hi,
thanks for raising these questions and trying to understand before deploying/changes to filters.
A good understanding of how tor relays and connections work is important when trying to defend against overload attacks, without breaking functionality with packet filters that cause false positive blocks, especially when such a long standing limit like the relays per IPv4 address limit is changed.
- I have a few Exit relays as permanent residents in my block list, not
because I want them to be there but because, no matter how many times I remove them, day or night, they'll be back in seconds for making too many concurrent attempts.
As a relay operator you should allow all other relays to connect to your relay's ORPort no matter what flags or onionoo guard/middle/exit probability they might have. Place known relay IPs on an exception list so your filters don't block them and update that list at least every hour.
If you have a problem on your relay's ORPort with a source IP that is also used by an exit relay please try to contact the operator by looking at their contactinfo, if they don't have a contactinfo, join the 'require a usable contactinfo' lobby ;) for this very reason and maybe ask on this list if they can drop you an email.
You can also not be sure whether it is an actual authenticated relay to relay connection or a client to relay connection just by looking at the source IP. Some upcoming MetricsPort enhancement might help you there in the future not per source IP but as a general overview for your relay's ORPort connections. An exit can share it's source IP with tor clients for example behind NAT, but I don't think that is common and it is also discouraged. Exits should have dedicated IPs that is not shared for unrelated things.
Blocking relay to relay communication should not be done as it breaks a core assumption of the tor network (every relay can talk to all other relays ORPorts). Upcoming tooling that detects broken relay to relay links might also detect and flag your relay if your filters break relay to relay communication.
As an option of last resort - after verifying it is an authenticated relay to relay connection that is causing you trouble and not some tor client using the same source IP you might contact the bad-relays list. That is still better than blocking another relay from reaching your relay's ORPort.
I've seen other problematic filter practices for relay to relay connections and I'll write up some recommendations in a separate email so it doesn't get lost in this lengthy email.
I'm assuming this is due to the fact that Exits are being used to attack other relays
In a vanilla current tor version it is not possible to use a tor client to connect via an exit to another relay's ORPort so this is very unlikely. The background here is that tor does not allow such connections to prevent an attacker from reentering the tor network via an exit relay.
You can test that by opening this URL in tor browser, you will get a "Unable to connect" very fast:
https://185.220.102.242/ because it is this ORPort:
https://metrics.torproject.org/rs.html#details/0A2366980A2842D770EF8E136A7DA...
the answer is very fast and not a slow timeout because a tor client can predict that is inaccessible before even trying to create a stream because exits will not allow such connections to relay ORPorts.
I have 2 Established connections to two Or Ports of an exit relay
I was not able to follow you there. I was unable to find any exit relay that has more than one ORPort on IPv4 (I identify relays by fingerprint not by IP address). Maybe you can list the specific exit relay fingerprint and timestamp so I can cross check for bugs in my tooling/onionoo?
- Each relay has Established connections to many other relays and if
they're guard they will also have many connections to regular users and their Tor browsers until they have enough traffic to reach their MaxAdvertisedBandwidth. Obviously we don't Establish connections to all 6300 relays out there.
It is best to actually expect that.
So if we do not allow each IP more than two connections and they need 4, They'll have two from us and they'll move on to another relay and get the other two and get the job done and we will reach our Max Bandwidth anyway by accepting traffic from other relays. Diversity of relays as opposed to concentration of some relays. Am I correct in my assumption that this will have little to no effect on the health of the Tor network as a whole?
I think there is a misunderstanding there. One important point to take away: Relays do not chose the next hop in a circuit, tor clients do. So if my tor clients picks a circuit with these relays: client -> A -> B -> C -> example.com and B can not reach C, B can not say "oh I can not reach C I'll pick D instead for you", a relay has no say in that. I hope that helps to reinforce the importance of ubiquitous reachability between relays. Relays have to obey a tor client's orders and a tor client expects that all relays can talk to each other. A few years ago David Stainton published some actual scan results on tor-dev that showed that this expectation might not be true in reality but close enough.
I fear that overly aggressive relay to relay filter actually help the adversaries more than the network and would advise against filtering practices between relays. The first step should always be on a social level: Try to reach the operator if you feel they attack your relay and NOT iptables DROP without notice.
kind regards, nusenu
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays