On Sat, Dec 30, 2017 at 03:33:23PM -0500, starlight.2017q4@binnacle.cx wrote:
I realize we're in the middle of the Christmas / New Year dead week, but it would be great one of the developers could says something (anything) about the ongoing denial-of-service attacks.
Still alive! Some of us just finished 34c3, and some others of us are trying to be offline for the holidays.
Here are some hopefully useful things to say:
(0) Thanks everybody for your work keeping the network going in the meantime! I see that the total number of relays has dropped off a tiny bit: https://metrics.torproject.org/networksize.html but the overall capacity and load on the network has stayed about the same: https://metrics.torproject.org/bandwidth.html So I wouldn't say the sky is falling at this point.
(1) I don't currently have any reason to think this is an intentional denial-of-service attack. Rather, the overloading is a consequence of the 1million+ new Tor clients that appeared a few weeks ago: https://metrics.torproject.org/userstats-relay-country.html This point shouldn't make you feel completely better, since even an incidental denial-of-service can cause real problems. But the response when modeling it as a malicious attack should be quite different from the response when modeling it as emergent pain from a series of bugs or poor design choices. Whatever these new clients are doing, the Tor design + the current Tor network aren't prepared to handle doing it at that scale.
(2) Now, it is very reasonable to wonder what 1M new Tor clients are doing, especially if they are all running on a handful of IP addresses at hosting facilities like Hetzner and OVH. These are not all new humans running Tor Browser.
(2b) If anybody has great contacts at Hetzner or OVH and can help us get a message to whoever is running these clients, that would be grand. ("Hi, did you know that you're hurting the Tor network? The Tor people would love to talk to you to help you do whatever it is you're trying to do, in a less harmful way.")
(3) I took some steps on Dec 22 to reduce the load that these clients (well, all clients) are putting on the network in terms of circuit creates. It seems like maybe it helped a bit, or maybe it didn't, but I'm the only one who has posted any stats for comparison. You can read more here: https://trac.torproject.org/24716
(4) David identified some bugs and missing features that could be causing extra pain in this situation:
https://trac.torproject.org/24665 https://trac.torproject.org/24666 https://trac.torproject.org/24667 https://trac.torproject.org/24668 https://trac.torproject.org/24671 https://trac.torproject.org/24694
A few of these were fixed in the latest 0.3.2.8-rc release, but some of them involve design work, not just bug fixes, and some of those changes are probably not a wise idea to stick into a late-stage release candidate (or backport to an earlier stable). So, think of this as a great opportunity for us to fix some scalability issues that weren't issues until this new situation appeared. :)
setevents extended orconn
volumes of
650 ORCONN ... LAUNCHED ID=nnnn 650 ORCONN ... FAILED REASON=CONNECTREFUSED NCIRCS=x ID=nnnn
messages appear. First I though these were operators with some new mitigation, but finally realized the relays are crashed and have not yet dropped off the consensus.
Hey, nice one. I've opened a ticket to try to improve that situation too: https://trac.torproject.org/24767
--Roger