Re: [tor-relays] DoS attacks on multiple relays
@ x9p:
# netstat -tupan | grep ESTABLISHED | grep /tor | awk '{print $5}' | awk -F: '{print $1}' | awk -F. '{print $1"."$2"."$3}' | sort | uniq -c | sort | egrep -v ' 1 | 2 | 3 '
with this information in hand, double the max of it (mine was 10 connections from 188.214.30.0/24):
10 188.214.30
iptables -A INPUT -i eth0 -p tcp -m connlimit --connlimit-above 20 --connlimit-mask 24 -j REJECT --reject-with tcp-reset
Thank you! This was extremely helpful. In our case we found a handful of IPs that had *thousands* of concurrent connections on several of our relays. The offending IPs were not in the consensus. After restarting the Tor service, these suspect connections come back rapidly, again across several of our relays. Since our relays are all in the same declared family, it is very difficult to see how this traffic is legitimate. If it's valid Tor clients, they are behaving very strangely, and in either case we need to limit their impact. As such we've implemented connlimits by /24 as suggested (with a much higher limit to err on the side of not rejecting valid traffic). We can already see that this has improved our situation. @ Scott Bennett:
What you are seeing is most likely the same phenomenon brought up on this list repeatedly over at least the last decade or so. That phenomenon is providing HSDir service, or perhaps a rendez-vous point, for a popular hidden service.
If you don't like it, you can set
HidServDirectoryV2 0
Thanks for your reply. The data suggests this was not the case (this time). Some of these relays have been up almost a year with the same configuration, often with the HSDir flag. The recent issues just started occurring, and happened across several relays during the same timeframe. Nonetheless, we're not opposed to disabling HidServDirectoryV2 to rule it out. However it appears that option is deprecated (on 0.3.1.9). Enabling it causes this in the log: [WARN] Skipping obsolete configuration option 'HidServDirectoryV2' It's also no longer listed in the Tor manual (https://www.torproject.org/docs/tor-manual.html.en). It looks like we might be able to achieve the same effect with something like this instead: MinUptimeHidServDirectoryV2 52 weeks Anyone have any info on why HidServDirectoryV2 is consider obsolete? Is using MinUptimeHidServDirectoryV2 instead going to achieve the same effect?
We have file descriptors (/proc/sys/fs/file-max) set to 64000, but it looks like Tor sets MAX_FILEDESCRIPTORS to 16384 per /etc/init.d/tor:
elif [ "$system_max" -gt "40000" ] ; then MAX_FILEDESCRIPTORS=16384
Surely that is high enough for normal service?
If by normal you mean "low traffic", then yes, it's probably enough. However, that's really not very high in a general sense.
Why does the default /etc/init.d/tor (from the Debian/Ubuntu pkg) cap MAX_FILEDESCRIPTORS at 16384 then? We have it set to 64000 otherwise. Obviously we could comment out these lines, but it seems odd the default config tries to cap it at 16384 if that's too low for a high traffic relay. Thanks everyone for your replies!
On 5 Dec 2017, at 22:50, x9p <tor@x9p.org> wrote:
my second and third positions are similar:
...
Please don't publicly share exact connection numbers and netblocks. They can be used in combination with other information to de-anonymise users. (For example, if an adversary knows that a guard has 5 connections from a netblock that they can't otherwise observe, they can go and hunt down those users.)
On 6 Dec 2017, at 07:42, null <null@omuravpn.com> wrote:
... @ Scott Bennett:
What you are seeing is most likely the same phenomenon brought up on this list repeatedly over at least the last decade or so. That phenomenon is providing HSDir service, or perhaps a rendez-vous point, for a popular hidden service.
If you don't like it, you can set
HidServDirectoryV2 0
Thanks for your reply. The data suggests this was not the case (this time). Some of these relays have been up almost a year with the same configuration, often with the HSDir flag. The recent issues just started occurring, and happened across several relays during the same timeframe.
Nonetheless, we're not opposed to disabling HidServDirectoryV2 to rule it out. However it appears that option is deprecated (on 0.3.1.9). Enabling it causes this in the log:
[WARN] Skipping obsolete configuration option 'HidServDirectoryV2'
It's also no longer listed in the Tor manual (https://www.torproject.org/docs/tor-manual.html.en). It looks like we might be able to achieve the same effect with something like this instead:
MinUptimeHidServDirectoryV2 52 weeks
Anyone have any info on why HidServDirectoryV2 is consider obsolete? Is using MinUptimeHidServDirectoryV2 instead going to achieve the same effect?
No, this option only applies to directory authorities, and determines their votes for the HSDir flag. From the tor man page: MinUptimeHidServDirectoryV2 N seconds|minutes|hours|days|weeks Minimum uptime of a v2 hidden service directory to be accepted as such by authoritative directories. (Default: 25 hours)
We have file descriptors (/proc/sys/fs/file-max) set to 64000, but it looks like Tor sets MAX_FILEDESCRIPTORS to 16384 per /etc/init.d/tor:
elif [ "$system_max" -gt "40000" ] ; then MAX_FILEDESCRIPTORS=16384
Surely that is high enough for normal service?
If by normal you mean "low traffic", then yes, it's probably enough. However, that's really not very high in a general sense.
Why does the default /etc/init.d/tor (from the Debian/Ubuntu pkg) cap MAX_FILEDESCRIPTORS at 16384 then? We have it set to 64000 otherwise. Obviously we could comment out these lines, but it seems odd the default config tries to cap it at 16384 if that's too low for a high traffic relay.
Perhaps it's the maximum allowed on some kernels or low-memory systems? Or perhaps it's historical? I suggest that you submit a ticket or patch to the debian bug tracker. T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
On 5 Dec 2017, at 22:50, x9p <tor@x9p.org> wrote:
my second and third positions are similar:
...
Please don't publicly share exact connection numbers and netblocks. They can be used in combination with other information to de-anonymise users.
(For example, if an adversary knows that a guard has 5 connections from a netblock that they can't otherwise observe, they can go and hunt down those users.)
true, didnt thought about it. but didnt told which relay was talking about, either. cheers. x9p
@ x9p:
# netstat -tupan | grep ESTABLISHED | grep /tor | awk '{print $5}' | awk -F: '{print $1}' | awk -F. '{print $1"."$2"."$3}' | sort | uniq -c | sort | egrep -v ' 1 | 2 | 3 '
with this information in hand, double the max of it (mine was 10 connections from 188.214.30.0/24):
10 188.214.30
iptables -A INPUT -i eth0 -p tcp -m connlimit --connlimit-above 20 --connlimit-mask 24 -j REJECT --reject-with tcp-reset
Thank you! This was extremely helpful.
In our case we found a handful of IPs that had *thousands* of concurrent connections on several of our relays. The offending IPs were not in the consensus. After restarting the Tor service, these suspect connections come back rapidly, again across several of our relays. Since our relays are all in the same declared family, it is very difficult to see how this traffic is legitimate. If it's valid Tor clients, they are behaving very strangely, and in either case we need to limit their impact. As such we've implemented connlimits by /24 as suggested (with a much higher limit to err on the side of not rejecting valid traffic). We can already see that this has improved our situation.
nice to hear :) cheers. x9p
null <null@omuravpn.com> wrote:
@ x9p:
# netstat -tupan | grep ESTABLISHED | grep /tor | awk '{print $5}' | awk -F: '{print $1}' | awk -F. '{print $1"."$2"."$3}' | sort | uniq -c | sort | egrep -v ' 1 | 2 | 3 '
with this information in hand, double the max of it (mine was 10 connections from 188.214.30.0/24):
10 188.214.30
iptables -A INPUT -i eth0 -p tcp -m connlimit --connlimit-above 20 --connlimit-mask 24 -j REJECT --reject-with tcp-reset
Thank you! This was extremely helpful.
In our case we found a handful of IPs that had *thousands* of concurrent connections on several of our relays. The offending IPs were not in the consensus. After restarting the Tor service, these suspect connections come back rapidly, again across several of our relays. Since our relays are all in the same declared family, it is very difficult to see how this traffic is legitimate. If it's valid
But it could be legitimate. As has been discussed here previously, they may be connections from a relay that actually is in the consensus, but either a) uses an OutboundBindAddress or b) is on a LAN that has multiple connections to the Internet. Those addresses (of either kind) are not published by tor, and the tor project has deferred to the alleged desires of some unknown portion of the operators of such relays to have those addresses remain secret at the expense of those of us who wish both to protect our systems, especially those systems that support the tor relay network, by the judicious use of packet filters and to give a free pass through those packet filters to currently running relays to access our relays' ORPorts and DirPorts. (I currently have such a "free pass" system for relays' IP addresses that appear in the most recently fetched consensus or the most recently fetched list of exit IP addresses. It also includes a handful of addresses mentioned on this list as being relay addresses that might not appear in the consensus.) No one on this list, BTW, has yet given a rationale for prioritizing the allegedly desired secrecy of those relay operators over our efforts to help to secure our tor relays and our tor relays' hosts. However, a couple of people have resorted to name-calling and absurd accusations of "censorship". (Now, tell me again, *why* is it that I've run a tor relay most of the time for over a decade? To practice censorship? Such stunning illogic is rather appalling.) A script similar to the one used to reveal and make available the otherwise unidentified source IP addresses of exits could be run by the project to gather the hidden addresses of currently running relays, and a list of such addresses could be made available on a compromise basis, e.g., by having a relay at the project that would serve those lists only over tunneled directory connections *from relays*, were it not for obstinacy. Such a list could then be included into our packet filters' "free pass" lists without putting the list up on the project's web site like the exit list is. A torrc option to fetch the list whenever updates were available could default to not fetching the list, so relays whose operators who do not use packet filter defenses would not automatically fetch the hidden address list of non-exit relays.
Tor clients, they are behaving very strangely, and in either case we need to limit their impact. As such we've implemented connlimits by /24 as suggested (with a much higher limit to err on the side of not rejecting valid traffic). We can already see that this has improved our situation.
And it will likely get you roundly denounced by tor project members and certain other individuals on this list. :-( You will also see your Fast and HSDir flags come and go at random, depending upon how many authorities creating testing circuits to reach and test your node(s) go through a node that used a hidden outbound address as the source address that fails your filter to connect to your node. Nevertheless, without knowing whether the sources from which you claim many connections are made to your relay(s) are themselves relays, you cannot know whether they are legitimate. For example, suppose a high-capacity relay is serving as the hop prior to your relay in many circuits, quite possibly from large numbers of users in many different countries. All you see is that a high-capacity relay has a lot of circuits created by clients that have chosen that relay, followed by yours, in their circuits. If you run an exit, you don't know whether that relay was the entry relay or the middle relay for any particular circuit. There may well be some of each.
@ Scott Bennett:
What you are seeing is most likely the same phenomenon brought up on this list repeatedly over at least the last decade or so. That phenomenon is providing HSDir service, or perhaps a rendez-vous point, for a popular hidden service.
If you don't like it, you can set
HidServDirectoryV2 0
Thanks for your reply. The data suggests this was not the case (this time). Some of these relays have been up almost a year with the same configuration, often with the HSDir flag. The recent issues just started occurring, and happened across several relays during the same timeframe.
Nonetheless, we're not opposed to disabling HidServDirectoryV2 to rule it out. However it appears that option is deprecated (on 0.3.1.9). Enabling it causes this in the log:
[WARN] Skipping obsolete configuration option 'HidServDirectoryV2'
It's also no longer listed in the Tor manual (https://www.torproject.org/docs/tor-manual.html.en). It looks like we might be able to achieve the same effect with something like this
Sigh. My apologies. You are indeed correct on this matter. It had slipped my mind that tor no longer is distributed with a man page. Instead, it has a document as HTML, not intended to be read directly by humans, but rather by a web browser, which humans must then use to dig around in their local file systems to find wherever the HTML page got installed in order to read it with the aid of the browser. :-(
instead:
MinUptimeHidServDirectoryV2 52 weeks
Anyone have any info on why HidServDirectoryV2 is consider obsolete?
Not entirely sure, but I *think* I remember reading somewhere that HS v2 is no longer preferred and has been superseded by HS v3. I hope someone will confirm/correct this recollection.
Is using MinUptimeHidServDirectoryV2 instead going to achieve the same effect?
As teor has answered you already, I'll skip this.
We have file descriptors (/proc/sys/fs/file-max) set to 64000, but it looks like Tor sets MAX_FILEDESCRIPTORS to 16384 per /etc/init.d/tor:
elif [ "$system_max" -gt "40000" ] ; then MAX_FILEDESCRIPTORS=16384
Surely that is high enough for normal service?
If by normal you mean "low traffic", then yes, it's probably enough. However, that's really not very high in a general sense.
Why does the default /etc/init.d/tor (from the Debian/Ubuntu pkg) cap MAX_FILEDESCRIPTORS at 16384 then? We have it set to 64000 otherwise. Obviously we could comment out these lines, but it seems odd the default config tries to cap it at 16384 if that's too low for a high traffic relay.
teor has addressed this one with a question. I am not a LINUX user, so I have no idea. Perhaps an operator of one or more high-capacity relays on any OS can provide some idea of what maxima might be reasonable for relays of different capacity scales (e.g., 500 KB/s, 1 MB/s, 5 MB/s, 10 MB/s-- Oleg? Are you still out there? Moritz?). Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************
On 6 Dec 2017, at 19:13, Scott Bennett <bennett@sdf.org> wrote:
null <null@omuravpn.com> wrote:
@ x9p:
# netstat -tupan | grep ESTABLISHED | grep /tor | awk '{print $5}' | awk -F: '{print $1}' | awk -F. '{print $1"."$2"."$3}' | sort | uniq -c | sort | egrep -v ' 1 | 2 | 3 '
with this information in hand, double the max of it (mine was 10 connections from 188.214.30.0/24):
10 188.214.30
iptables -A INPUT -i eth0 -p tcp -m connlimit --connlimit-above 20 --connlimit-mask 24 -j REJECT --reject-with tcp-reset
Thank you! This was extremely helpful.
In our case we found a handful of IPs that had *thousands* of concurrent connections on several of our relays. The offending IPs were not in the consensus. After restarting the Tor service, these suspect connections come back rapidly, again across several of our relays. Since our relays are all in the same declared family, it is very difficult to see how this traffic is legitimate. If it's valid
But it could be legitimate. As has been discussed here previously, they may be connections from a relay that actually is in the consensus, but either a) uses an OutboundBindAddress or b) is on a LAN that has multiple connections to the Internet.
Relays try very hard to have at most one connection to each other relay. (And only two relays are allowed in the consensus per IPv4 address.) Clients try to make one connection to one or two guards. So it's far more likely to be a collection of Tor clients in a network with only a few public IPv4 addresses. (There are entire countries and large networks that only have a few allocated IPv4 addresses.) Or, it might be a bug in Tor, a misconfiguration, or a denial of service attack. We'd like to know more, so we can find out and fix it.
(Snip) A script similar to the one used to reveal and make available the otherwise unidentified source IP addresses of exits could be run by the project to gather the hidden addresses of currently running relays, and a list of such addresses could be made available on a compromise basis, e.g., by having a relay at the project that would serve those lists only over tunneled directory connections *from relays*, were it not for obstinacy. Such a list could then be included into our packet filters' "free pass" lists without putting the list up on the project's web site like the exit list is.
Outbound addresses aren't secret, because they are used for connections. Anyone is free to volunteer to create and maintain a list of outbound relay addresses. It is technically feasible: it requires a few thousand Tor connections per day, one via each relay, to a relay that reports the remote address of each inbound relay connection. It just needs to be done safely, in a way that doesn't collect client addresses, and avoids attaching a timestamp or order to relay connections.
A torrc option to fetch the list whenever updates were available could default to not fetching the list, so relays whose operators who do not use packet filter defenses would not automatically fetch the hidden address list of non-exit relays.
If someone created a list, and showed that it had value to other relay operators, then it might gain support, and be supported by Tor, just like other features have: There is an exit_addresses field for relays in Onionoo and Relay Search that gives the outbound exit addresses of every exit (when they differ from the relay address). It gathers addresses using exitmap. (Rather than relays self-reporting, which is unreliable.) There is also a torrc option that has the side effect of making every address on the machine public, even unused addresses, because it blocks them in the exit policy. (ExitPolicyRejectLocalInterfaces, off by default.) Here's how someone could work on this feature: Create a scanner, publish a list, and show that it has value. Or start with a proposal, ask for advice, then create the scanner. Try for something independent of Tor, like exitmap, because it will be more accurate. (Tor doesn't always know what its outbound address is.) And try to have list downloads rely on existing Tor features, like onion services. They'll be faster to deploy that way. Here's a description of the proposal process: https://gitweb.torproject.org/torspec.git/tree/proposals/001-process.txt
Tor clients, they are behaving very strangely, and in either case we need to limit their impact. As such we've implemented connlimits by /24 as suggested (with a much higher limit to err on the side of not rejecting valid traffic). We can already see that this has improved our situation.
And it will likely get you roundly denounced by tor project members and certain other individuals on this list. :-(
I sketched a proposal like this in another thread just a few days ago. I'm happy to work with others to include inbound or outbound connection limits in the draft proposal. (My initial proposal had outbound circuit and stream limits.)
You will also see your Fast and HSDir flags come and go at random, depending upon how many authorities creating testing circuits to reach and test your node(s) go through a node that used a hidden outbound address as the source address that fails your filter to connect to your node.
If you set the connection limit at or above 512 connections per /24, it will be impossible for well-behaved consensus relays to go above the limit: 2 relays per IPv4 * 256 IPv4 addresses per /24 = 512 connections Or you could check how many relays are in the most popular /24, and use that to work out a limit.
(Snip)
@ Scott Bennett:
(Snip)
Nonetheless, we're not opposed to disabling HidServDirectoryV2 to rule it out. However it appears that option is deprecated (on 0.3.1.9). Enabling it causes this in the log:
[WARN] Skipping obsolete configuration option 'HidServDirectoryV2'
It's also no longer listed in the Tor manual (https://www.torproject.org/docs/tor-manual.html.en). It looks like we might be able to achieve the same effect with something like this
Sigh. My apologies. You are indeed correct on this matter. It had slipped my mind that tor no longer is distributed with a man page.
Tor source is distributed with an asciidoc man page. It might not be on your system because the packager left it out, or left out asciidoc as a build dependency.
(Snip)
Anyone have any info on why HidServDirectoryV2 is consider obsolete?
Not entirely sure, but I *think* I remember reading somewhere that HS v2 is no longer preferred and has been superseded by HS v3. I hope someone will confirm/correct this recollection.
It was removed in 0.2.7 along with the authority option VoteOnHidServDirectoriesV2. There's not much information on the ticket: https://trac.torproject.org/projects/tor/ticket/16543 In general, we try to remove options that very few people use, because it reduces the cost of testing and maintaining Tor.
(Snip)
T
teor <teor2345@gmail.com> wrote:
On 6 Dec 2017, at 19:13, Scott Bennett <bennett@sdf.org> wrote:
null <null@omuravpn.com> wrote:
@ x9p:
[stuff deleted --SB]
relays. Since our relays are all in the same declared family, it is very difficult to see how this traffic is legitimate. If it's valid
But it could be legitimate. As has been discussed here previously, they may be connections from a relay that actually is in the consensus, but either a) uses an OutboundBindAddress or b) is on a LAN that has multiple connections to the Internet.
Relays try very hard to have at most one connection to each other relay.
Good. That tendency must have improved a lot over the years, then.
(And only two relays are allowed in the consensus per IPv4 address.)
Yes.
Clients try to make one connection to one or two guards.
Or however many are specified in torrc. Sometimes one needs, say, three to five guards for better capacity or lower latency or better reliability.
So it's far more likely to be a collection of Tor clients in a network with only a few public IPv4 addresses. (There are entire countries and large networks that only have a few allocated IPv4 addresses.)
An excellent point. Thank you.
Or, it might be a bug in Tor, a misconfiguration, or a denial of service attack. We'd like to know more, so we can find out and fix it.
If it's a LAN with many users and few external IPv4 addresses (or, perhaps, even IPv6 addresses if NATed to the outside via only a few), then there really is nothing to fix.
(Snip) A script similar to the one used to reveal and make available the otherwise unidentified source IP addresses of exits could be run by the project to gather the hidden addresses of currently running relays, and a list of such addresses could be made available on a compromise basis, e.g., by having a relay at the project that would serve those lists only over tunneled directory connections *from relays*, were it not for obstinacy. Such a list could then be included into our packet filters' "free pass" lists without putting the list up on the project's web site like the exit list is.
Outbound addresses aren't secret, because they are used for connections.
Roger has claimed here that some of them are indeed secret in the sense that their owners do *not* want them to be published, one possible reason for which being that they do not want their relays blocked successfully by governments, e.g., China, Iran. (How hiding the source address of a published relay would evade the Great Firewall escapes me somehow, except perhaps for hidden services based inside China that might be reached via those hidden source addresses. Given that most source addresses of relays *are* published, the chances of getting a circuit into China seem rather slim anyway.) I guess tor project members are not all in agreement over such matters.
Anyone is free to volunteer to create and maintain a list of outbound relay addresses. It is technically feasible: it requires a few thousand Tor connections per day, one via each relay, to a relay that reports the
Ideally, per hour, but that is why it should only be done by one site. Note that Exit relays might be skippable because their outbound addresses are already identified by one site, namely, the tor project, and published. IOW, only entry/middle and non-Exit exit nodes need be tested, which would shrink the list by several hundred to a thousand or so.
remote address of each inbound relay connection.
It just needs to be done safely, in a way that doesn't collect client addresses, and avoids attaching a timestamp or order to relay connections.
Why would client addresses ever be involved? What would be gathered are the addresses from which *relays* connect to other relays (N.B. *not* to destinations). The only timestamps that I see might be relevant would be the starting and ending times for each script run, so that an administrator's own script(s) for incorporating those addresses into his "free pass" list might easily discern out-of-date script output files from current script output files.
A torrc option to fetch the list whenever updates were available could default to not fetching the list, so relays whose operators who do not use packet filter defenses would not automatically fetch the hidden address list of non-exit relays.
If someone created a list, and showed that it had value to other relay operators, then it might gain support, and be supported by Tor, just like other features have:
There is an exit_addresses field for relays in Onionoo and Relay Search that gives the outbound exit addresses of every exit (when they differ from the relay address). It gathers addresses using exitmap. (Rather than relays self-reporting, which is unreliable.)
Right.
There is also a torrc option that has the side effect of making every address on the machine public, even unused addresses, because it blocks them in the exit policy. (ExitPolicyRejectLocalInterfaces, off by default.)
Here's how someone could work on this feature:
Create a scanner, publish a list, and show that it has value.
Because such a list would include addresses whose owners might not be pleased about those addresses being published (see above), such a list should *not* be published, but perhaps could be sent to someone in the tor project. Better still, the generating script could be sent to someone in the tor project to enable the project to run the script, rather than encouraging many relay operators all to duplicate the network load of running it.
Or start with a proposal, ask for advice, then create the scanner.
Try for something independent of Tor, like exitmap, because it will be more accurate. (Tor doesn't always know what its outbound address is.)
Like I wrote before, a script would do it, although an actual program could also do it. Execution speed is probably not relevant because the orders-of-magnitude larger delays will come from circuit construction times, which consist primarily of network delays and delays caused by other relays. After putting it off for many years, I'm now going through a small book on python. Aside from a couple of very annoying misfeatures I've run across so far, it looks to be a fairly straightforward language to learn to write. Then I'll need to study the tor project's stem library documentation before writing the script, which I doubt will need to be very complex. The output, after all, should only be a list of otherwise unpublished IP addresses from which connections from relays may arrive at any other relay. No other information need be included, other than possibly the aforementioned file generation timestamps (similar to consensus document timestamps, though perhaps less frequent).
And try to have list downloads rely on existing Tor features, like onion services. They'll be faster to deploy that way.
AFAIK, tor has no such feature. If a relay is to download nothing more than a file of IP addresses, which feature are you suggesting will do that upon demand by a relay (and only an identified relay)? Yes, a relay can ask for a directory download (and so can a client). Yes, a relay can ask for a directory update download (and so can a client). Yes, a relay can ask for ExtraInfo document downloads. How does a relay ask to download a kind of file that doesn't yet exist? Is there already some undocumented, generic feature that a identified relay (but nothing else) can ask a directory mirror or authority to "give me your latest version of file x"? If you mean that the downloading process could be spun off to a worker thread, then yes, of course, it should be, but the actual implementation in tor would be up to the tor developers, not to me.
Here's a description of the proposal process:
https://gitweb.torproject.org/torspec.git/tree/proposals/001-process.txt
Tor clients, they are behaving very strangely, and in either case we need to limit their impact. As such we've implemented connlimits by /24 as suggested (with a much higher limit to err on the side of not rejecting valid traffic). We can already see that this has improved our situation.
And it will likely get you roundly denounced by tor project members and certain other individuals on this list. :-(
I sketched a proposal like this in another thread just a few days ago. I'm happy to work with others to include inbound or outbound connection limits in the draft proposal. (My initial proposal had outbound circuit and stream limits.)
You will also see your Fast and HSDir flags come and go at random, depending upon how many authorities creating testing circuits to reach and test your node(s) go through a node that used a hidden outbound address as the source address that fails your filter to connect to your node.
If you set the connection limit at or above 512 connections per /24, it will be impossible for well-behaved consensus relays to go above the limit:
2 relays per IPv4 * 256 IPv4 addresses per /24 = 512 connections
Apparently, the aforementioned effort to limit each relay pair to a single connection does not apply to hidden service connections, as can be readily seen on a Fast HSDir relay when bursts of connections occur. There is also little to prevent many thousands of clients from connecting simultaneously. A number of high-capacity relay operators have mentioned here in the past that they have had to increase their OS's fd limit to a value far greater than its default in order to prevent relay death under the heavy connection loads typical of such relays. My relay is a relatively low-capacity relay, yet when it has the Fast flag, and especially with an additional HSDir flag, it often has several thousand connections at any given time.
Or you could check how many relays are in the most popular /24, and use that to work out a limit.
(Snip)
@ Scott Bennett:
(Snip)
Nonetheless, we're not opposed to disabling HidServDirectoryV2 to rule it out. However it appears that option is deprecated (on 0.3.1.9). Enabling it causes this in the log:
[WARN] Skipping obsolete configuration option 'HidServDirectoryV2'
It's also no longer listed in the Tor manual (https://www.torproject.org/docs/tor-manual.html.en). It looks like we might be able to achieve the same effect with something like this
Sigh. My apologies. You are indeed correct on this matter. It had slipped my mind that tor no longer is distributed with a man page.
Tor source is distributed with an asciidoc man page.
But not a UNIX/LINUX man page, either as *roff source or plain, ASCII text. If it doesn't have a file that works with the man command, then it does not have a man page, by definition.
It might not be on your system because the packager left it out, or left out asciidoc as a build dependency.
Thanks for the info. I'll try to check into that. asciidoc is in the build dependency list if either the DOCUMENTATION or the MANPAGES configuration option for the port is selected. The MANPAGES option is on by default in the port's Makefile, yet no man page gets installed as nearly as I can see. Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************
The advantages for tor users against censorship, of operators making some fraction of all exit nodes exit from different addresses than those addresses present in consensus, have been explained many times on this list. It's also been noted that in at least one instance, some exit operators looking at constrained / creative hosting issues have chosen that as an enabling option too.
grarpamp <grarpamp@gmail.com> wrote:
The advantages for tor users against censorship, of operators making some fraction of all exit nodes exit from different addresses than those addresses present in consensus, have been explained many times on this list.
There are also advantages for tor users in keeping the hosts on which relays run secured against attacks, which helps to keep the tor network functioning. Having connections arrive from addresses not associable with tor relays by the recipients of those connections hinders connectivity among relays in the consensus, as has been explained many times on this list. I call attention to the fact that the tor project has already decided against Exit node secrecy by regularly publishing the results of a script (exitmap) that reveals such addresses.
It's also been noted that in at least one instance, some exit operators looking at constrained / creative hosting issues have chosen that as an enabling option too.
However, the tor project has publicly promised to identify publicly all Exit source addresses, so that system administrators everywhere who wish to block tor Exits to their networks may do so without blocking all tor relays. The readily downloadable exitmap results allegedly fulfill that promise. Are you now alleging that, in at least one instance, the tor project has surreptitiously broken its promise by omitting that instance's address(es) from the exitmap file? Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************
Furthermore, nodes that think they're being smart and protecting their nodes and the network by filtering out who can access their OR ports... aren't... because - A real attack from clearnet will wipe out significant numbers of the rest of the network leaving their OR's with far fewer left talking to them, and certainly a whole lot of compromised talkers, which means path discovery, correlation, manipulation, and other game over situations. - Attackers will simply stand up sybils and attack from within the core.. again wiping out the network and such "protected" boxes straight through their leeto trusted filter holes. Oopsie, plan demolished. - Are potentially creating segmentation breaks in the p2p network that will get their relays flagged down out of the consesus for obstructing node traffic. A better use of such pedantic time would be inspecting the code and protocols to discover and patch attack vectors that could be used against the network, including those that will sneak right through their filters. Who cares, operators are independant, do what you will, be creative, drop some pps worth of ORPort stray junk into /dev/null, filter out whatever exit traffic content you hate, make sure all subscribers to all blocklists can censor users, etc, etc... a few random boxes don't make a network, let alone a secure one.
Furthermore, nodes that think they're being smart and protecting their nodes and the network by filtering out who can access their OR ports... aren't... because
I strongly agree with grarpamp. Relay operators, do NOT block the IP addresses of the following guards in your firewall: 188.214.30.159 188.214.30.234 188.214.30.220 188.214.30.133 188.214.30.98 188.214.30.124 188.214.30.166 188.214.30.126 188.214.30.143 188.214.30.141 Hidden Service operators, and private guards operators protecting yours Hidden Services, if you believe it is better safe than sorry, I strongly advise on blocking the above IP addresses in your firewall, while they are not pulled out of the network. cheers. x9p
"x9p" <tor@x9p.org> wrote:
Furthermore, nodes that think they're being smart and protecting their nodes and the network by filtering out who can access their OR ports... aren't... because
I strongly agree with grarpamp.
Sorry to read that.
Relay operators, do NOT block the IP addresses of the following guards in your firewall:
[list deleted --SB]
They have been added to a file that gets included into a list of relay addresses (i.e., protected addresses) and will get the "free pass". This, BTW, is basically a temporary hack. If a frequently generated, automatically downloadable-by-relays list of otherwise unpublished relay source addresses becomes available, I'll axe this file because I'd much rather have a file that is kept up to date automatically. Note, however, that if a facility existed in tor for a relay to fetch a list from the tor project, there would have been no reason to publish them on any email list. Also, FWIW, they were not in my block list anyway, but now they won't be blocked from my relay's ORPort or DirPort by the block list even if they do get onto it. Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************
[tor project regularly publishes exit addresses] ...
Which thankfully not all blockers (censors) use, and equally some relay operators then leverage their anti censorship philosophy into those holes.
I call attention to the fact that the tor project has already decided against Exit node secrecy by regularly publishing
However, the tor project has publicly promised to identify publicly all Exit source addresses
In part because fear of getting rejected by the rest of the internet, sued, and chilled into shutdown for not providing easy means of censorship. Legal best for a corporation et al and paychecks, is not always in alignment with profferred, or even all potential, use cases of its products, or users of same.
... so that system administrators everywhere who wish to block tor Exits to their networks may do so without blocking all tor relays.
Those who block tor exit traffic from tor nodes inbound to their networks could care less than a fuck about allowing non-exit traffic from tor nodes inbound to their networks. It's operators trying to surf out from their nodes, and hosting services trying to make money (by taking non-exit money while avoiding taint). that cared about establishing such distinction.
Are you now alleging that
Chilling and censorship is real. That sucks. Fight it.
(Going back a bit in this thread...) On 7 Dec 2017, at 19:56, Scott Bennett <bennett@sdf.org <mailto:bennett@sdf.org>> wrote:
teor <teor2345@gmail.com <mailto:teor2345@gmail.com>> wrote:
On 6 Dec 2017, at 19:13, Scott Bennett <bennett@sdf.org <mailto:bennett@sdf.org>> wrote:
(Snip) A script similar to the one used to reveal and make available the otherwise unidentified source IP addresses of exits could be run by the project to gather the hidden addresses of currently running relays, and a list of such addresses could be made available on a compromise basis, e.g., by having a relay at the project that would serve those lists only over tunneled directory connections *from relays*, were it not for obstinacy. Such a list could then be included into our packet filters' "free pass" lists without putting the list up on the project's web site like the exit list is.
Outbound addresses aren't secret, because they are used for connections.
Roger has claimed here that some of them are indeed secret in the sense that their owners do *not* want them to be published
Then maybe you should respect their wishes? Or provide a compelling argument in a proposal that the Tor network and Tor users will be improved by your proposal.
one possible reason for which being that they do not want their relays blocked successfully by governments, e.g., China, Iran. (How hiding the source address of a published relay would evade the Great Firewall escapes me somehow, except perhaps for hidden services based inside China that might be reached via those hidden source addresses.
That isn't how hidden services work: they connect out to guards. Only exit and relay to relay connections use relay outbound addresses.
Given that most source addresses of relays *are* published, the chances of getting a circuit into China seem rather slim anyway.)
That's not how relays work, either: they need mutual connectivity. We typically ask users to configure their clients with bridges or meek in these countries. So this could only ever have been about exit connections to websites in China. And that doesn't make too much sense to me.
(Snip)
Anyone is free to volunteer to create and maintain a list of outbound relay addresses. It is technically feasible: it requires a few thousand Tor connections per day, one via each relay, to a relay that reports the
Ideally, per hour, but that is why it should only be done by one site.
Scanning every hour increases the cost to the network significantly. Do you really think relays change their outbound addresses that often? I would also encourage you to work out how much relay capacity this costs the network, and develop a plan to provide at least that much. A cost-benefit analysis would be an important part of the proposal. If you don't want to do this, use 24 hours, which is the exitmap interval.
Note that Exit relays might be skippable because their outbound addresses are already identified by one site, namely, the tor project, and published. IOW, only entry/middle and non-Exit exit nodes need be tested, which would shrink the list by several hundred to a thousand or so.
There are separate OutboundBindAddressOR and OutboundBindAddressExit torrc options. This means that connections to relays and connections to websites may come from different addresses. (My relays are set up like this - it allows the provider to null-route exit DoS attacks, with less disruption to users.) Here's an alternative scheme: Run a relay that attempts to maintain an inbound connection from each other relay. Each hour, report the last address seen on the last inbound connection from each other relay, whether that connection is still active or not. Each hour, try to re-establish inbound connections from each other relay. Here's why this works: As long as the original TCP connection stays up, the relay has the same outbound address. Whenever the connection breaks, it is possible that the outbound address has changed. Rate-limiting reconnection attempts decreases network load. (Snip)
It just needs to be done safely, in a way that doesn't collect client addresses, and avoids attaching a timestamp or order to relay connections.
Why would client addresses ever be involved? What would be gathered are the addresses from which *relays* connect to other relays (N.B. *not* to destinations).
You could easily collect and publish client and bridge addresses if you run a relay and dump all inbound connections. You need to understand the difference between inbound client, bridge, and relay connections. Relays authenticate, clients and bridges don't. It's not sufficient to rely on (not having) the Guard flag, because: * bridge clients use the bridge as their Guard * most current Tor versions have a bug that assigns a non-zero probability to excluded relay flags * some clients don't use Guards
The only timestamps that I see might be relevant would be the starting and ending times for each script run, so that an administrator's own script(s) for incorporating those addresses into his "free pass" list might easily discern out-of-date script output files from current script output files.
You will need to sort outputs to destroy order.
(Snip)
Here's how someone could work on this feature:
Create a scanner, publish a list, and show that it has value.
Because such a list would include addresses whose owners might not be pleased about those addresses being published (see above), such a list should *not* be published, but perhaps could be sent to someone in the tor project.
I personally would not handle a list created against the wishes of a group of relay operators.
Better still, the generating script could be sent to someone in the tor project to enable the project to run the script, rather than encouraging many relay operators all to duplicate the network load of running it.
I don't know if anyone would expend resources on this.
Or start with a proposal, ask for advice, then create the scanner.
(Snip)
And try to have list downloads rely on existing Tor features, like onion services. They'll be faster to deploy that way.
AFAIK, tor has no such feature. If a relay is to download nothing more than a file of IP addresses, which feature are you suggesting will do that upon demand by a relay (and only an identified relay)?
Tor relays submit signed descriptors. But there is no feature in Tor that only answers signed requests. That would require another proposal. But relay keys are public, so you could create a server that only accepts signed relay requests. And a client that signs requests using relay private keys. Signing requests with relay private keys introduces potential security holes, so I don't know how many operators would run a client like this. (A better design would be to include another signing key in relay descriptors, cross-certify it, and use that for requests.) Then you could run the server as an onion service, which provides authentication of the server. This looks like a lot of work, and cryptography is notoriously hard to get right.
Yes, a relay can ask for a directory download (and so can a client). Yes, a relay can ask for a directory update download (and so can a client). Yes, a relay can ask for ExtraInfo document downloads. How does a relay ask to download a kind of file that doesn't yet exist? Is there already some undocumented, generic feature that a identified relay (but nothing else) can ask a directory mirror or authority to "give me your latest version of file x"? If you mean that the downloading process could be spun off to a worker thread, then yes, of course, it should be, but the actual implementation in tor would be up to the tor developers, not to me.
We accept patches, that's how people become tor developers. But I'm not sure if we would merge a patch over the objections of a group of existing relay operators.
Here's a description of the proposal process:
https://gitweb.torproject.org/torspec.git/tree/proposals/001-process.txt <https://gitweb.torproject.org/torspec.git/tree/proposals/001-process.txt> I think we've reached the point in this discussion where we need to move to something more structured than email.
If you want to make this happen, and are willing to put some work in, write a proposal.
(Snip)
You will also see your Fast and HSDir flags come and go at random, depending upon how many authorities creating testing circuits to reach and test your node(s) go through a node that used a hidden outbound address as the source address that fails your filter to connect to your node.
If you set the connection limit at or above 512 connections per /24, it will be impossible for well-behaved consensus relays to go above the limit:
2 relays per IPv4 * 256 IPv4 addresses per /24 = 512 connections
Apparently, the aforementioned effort to limit each relay pair to a single connection does not apply to hidden service connections, as can be readily seen on a Fast HSDir relay when bursts of connections occur.
There are multiple resource limits in Tor. Are you sure it's the connection limit that's being hit? We often see bandwidth and circuit limits being hit in these cases.
(Snip) My relay is a relatively low-capacity relay, yet when it has the Fast flag, and especially with an additional HSDir flag, it often has several thousand connections at any given time.
There are several thousand relays, so several thousand connections is normal. And an additional few thousand client or many thousand exit connections is also normal.
(Snip)
T
Outbound addresses aren't secret, because they are used for connections.
Roger has claimed here that some of them are indeed secret in the sense that their owners do *not* want them to be published
Then maybe you should respect their wishes?
Exactly. Just like bridges that want to keep quietly doing their own good work without being autoblocked all to hell. Why the fuck people keep trying to expose different-IP exits and harken for their compliance with some random bs when Tor is supposed to be all about circumvention and privacy and independance of operators to the cause... is beyond me. And that TPO Inc seems bent on publishing them too... ugh. Leave them alone. For that matter, what of operators that run VPN termination to offer users UDP and IP, or I2P or other networks, messaging, storage, webservers, coins, etc on the same IP's... calls to out them for "security", "reasons", or whatever too? Stick to actual bad-exits, bad-relays, statistical anomaly divining of relays. Build better code and relay operator social / WoT proofing groups, explore parameters for subscribable selection sets, etc.
participants (5)
-
grarpamp -
null -
Scott Bennett -
teor -
x9p