[tor-relays] Overload (dropped ntor) due to DDoS??

[Scott Bennett]

lists at for-privacy.net wrote:

> On Friday, August 5, 2022 1:11:27 AM CEST s7r wrote:
> > Richard Menedetter wrote:
>
> > > I have a non exit relay running on a root server (4 AMD Epyc cores, 8 GB
> > > RAM, 2.5 GBit/s Ethernet) I have limited tor to numcpus 2,
> Why? Do you have other services on the server? Otherwise, omit num CPU. Let 
> the tor daemon use all CPU's for crypto stuff.

     The OP has a EPYC chip with four cores and presumably eight CPU threads,
so he should specify at least "NumCPUs 8" as a bare minimum.  "NumCPUs 16",
"NumCPUs 24", or "NumCPUs 32" would be better.  These onion skin decryption
threads use hardly any actual CPU time throughout the course of a busy day, but
having many extras available for work can mean drastically lower times for a
decryption task to remain in the work queue and therefore a much greater chance
of getting through a high connection count period without dropping any due to
the high load.  I've never seen an overload message like the one the OP showed,
and my system is ridiculously slower than the OP's EPYC.
     When I last started tor 41 days ago, it was started with "NumCPUs 12", up
from 10 in the previous iteration.  I've since changed torrc to "NumCPUs 16"
because of the connection and circuit counts during the recent troubles, so the
next time I start it it will be with four onion skin decryptors per core.  This
is on an ancient QX9650 (Core 2 Extreme quad-core, so only one CPU thread per
core) bumped up to 3.3 GHz.  I hope that will be sufficient to get it through
such attacks by making my Internet connection speed, rather than my tor config,
be the weak spot.  During the recent events I've seen tor's main thread take up
15% to 16% of CPU time on one core, while each of the worker threads consumed
far less than 1% apiece.  FWIW, mprime runs nice'd to 20 on all four cores, as
well.  Also, I normally renice tor to -6 about 15 seconds after starting it, to
keep it mostly prioritized higher than anything I might be doing interactively
and also other stuff running all the time (e.g., top(1), privoxy(1)).
>
> > > relaybandwidthburst 15 MB, hardwareaccel 1, maxadvertisedbandwidth 10 MB,
> > > maxmeminqueues 3GB
> > 
> > Thanks for running a relay!
> > 
> > didn't you also use RelayBandwidthRate along with RelayBandwidthBurst ?
> > 
> > 
> > > 
> > > Usually it takes less than 1 CPU core, and like 1 GB of RAM.

     During the couple of hours of the attack the other day my tor instance
never seemed to use more than ~2700 MB.

> > > But recently my relay is foten shown as obverloaded.
> > > I have these LOG entries:
> > > Tor[814]: General overload -> Ntor dropped (290376) fraction 5.3451% is
> > > above threshold of 0.5000%
> > 
> > You are not the only one, it's an ongoing DoS attack on the network, 
> > targeting onion services.
> > 
> > 
> > > 
> > > Is this due to DDoS attacks or a misconfigration on my side?
> > 
> > 
> > Besides the question above about RelayBandwidthRate I don't see anything 
> > wrong.
> > 
> > 
> > > Is there something that I can do to aleviate this issue?
> > 
     Again, I suggest increasing NumCPUs by several times to try to avoid the
overload message.
> > 
> > Nope, there is nothing you can do, unfortunately. Tor has some defenses 
> > against DoS and will blacklist / mark the abusing addresses, etc. as 
> > much as it can. But as you know DoS is a never ending battle, usually 
> > won by having "larger pipe", and it's something hard to tickle in an 
> > environment where anonymity is the grounding law.
> > 
> > What you can do is maintain your relay up and running in good shape with 
> > the latest version of Tor until this "attack" gets through. As I said, I 
> > guess most of relays are getting this at present times. The DoS "attack" 
> > is not targeted at your relay, what you are seeing is just a side effect 
> > of someone creating large amounts of circuits (heavy usage of Tor) which 
> > is reflected network-wide anyways.
> Sometimes 100.000-1.000.000 connections from one IP!
> I block the worst with 2 nftables egress rules.
>
> toralf has developed some smarter ddos scripts:
> https://github.com/toralf/torutils


                                  Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet:   bennett at sdf.org   *xor*   bennett at freeshell.org  *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."                                               *
*    -- Gov. John Hancock, New York Journal, 28 January 1790         *
**********************************************************************