Re: [tor-relays] Detecting Network Attack [re: exit synflooded]
Well, it's still going on, and is pretty much ruining Libero :( . Running CentOS 6, here.
When it's happening it can look like this:
# netstat -n | grep -c SYN 17696
I run a fast exit and can offer some advice: 1) mitigate bug #18580 (also #21394); is a DNS denial-of-service and could be the problem. either upgrade to 0.3.2.4+ or edit resolve.conf per https://trac.torproject.org/projects/tor/wiki/doc/DnsResolver#Tuningeventdns... also check out https://arthuredelstein.net/exits/ 2) if you continue to experience excessive outbound scanning SYNs, I've found that simply enabling connection tracking helps by implicitly limiting the rate a which connections can originate. If an iptables "-m state --state ESTABLISHED,RELATED -j ACCEPT" rule exists it will turn it on or you could modprobe the module if you don't want to configure incoming connection rules. some useful sysctl.conf changes, run sysctl -p after or reboot # Tor Exit tuning. net.ipv4.ip_local_port_range = 1025 65535 net.ipv4.tcp_synack_retries = 2 net.ipv4.tcp_max_tw_buckets = 4194304 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_fin_timeout = 30 net.ipv4.tcp_keepalive_time = 300 net.ipv4.tcp_keepalive_intvl = 10 net.ipv4.tcp_keepalive_probes = 3 net.netfilter.nf_conntrack_tcp_timeout_established = 1000 net.netfilter.nf_conntrack_checksum = 0 you might see many messages: kernel: nf_conntrack: table full, dropping packet which indicates no connection table slots were available for an outbound connection and that rate limiting is effected state of connection tracking appears in /proc/net/nf_conntrack
Do not enable net.ipv4.tcp_tw_recycle: https://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-linux#netipv4tcp_... For ip_local_port_range, make one number even and the other one odd (i.e. 1024 and 65535). Not sure if this is still required, but won't hurt to include port 1024. Consider blacklisting the nf_conntrack module altogether and running a completely stateless firewall: https://javapipe.com/ddos/blog/iptables-ddos-protection/ https://strongarm.io/blog/linux-stateless-firewall/ On Sat, Nov 25, 2017 at 3:14 PM, Dhalgren Tor <dhalgren.tor@gmail.com> wrote:
Well, it's still going on, and is pretty much ruining Libero :( . Running CentOS 6, here.
When it's happening it can look like this:
# netstat -n | grep -c SYN 17696
I run a fast exit and can offer some advice:
1) mitigate bug #18580 (also #21394); is a DNS denial-of-service and could be the problem. either upgrade to 0.3.2.4+ or edit resolve.conf per https://trac.torproject.org/projects/tor/wiki/doc/DnsResolver#Tuningeventdns...
also check out https://arthuredelstein.net/exits/
2) if you continue to experience excessive outbound scanning SYNs, I've found that simply enabling connection tracking helps by implicitly limiting the rate a which connections can originate. If an iptables "-m state --state ESTABLISHED,RELATED -j ACCEPT" rule exists it will turn it on or you could modprobe the module if you don't want to configure incoming connection rules.
some useful sysctl.conf changes, run sysctl -p after or reboot
# Tor Exit tuning. net.ipv4.ip_local_port_range = 1025 65535 net.ipv4.tcp_synack_retries = 2 net.ipv4.tcp_max_tw_buckets = 4194304 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_fin_timeout = 30 net.ipv4.tcp_keepalive_time = 300 net.ipv4.tcp_keepalive_intvl = 10 net.ipv4.tcp_keepalive_probes = 3 net.netfilter.nf_conntrack_tcp_timeout_established = 1000 net.netfilter.nf_conntrack_checksum = 0
you might see many messages:
kernel: nf_conntrack: table full, dropping packet
which indicates no connection table slots were available for an outbound connection and that rate limiting is effected
state of connection tracking appears in /proc/net/nf_conntrack _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
After reading every paper and post on sysctl.conf and iptables tuning I could find, and reading some kernel code, I have come to a conclusion that, while there are a few settings to tune (can share mine, but your mileage *will* vary), most of the defaults are actually not broken in the latest kernels, while a lot of tuning recommendations online are outdated. I enjoyed this guide a lot: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/htm.... It is written for Red Hat, but contains quite a few insights applicable to all Linux distributions. - Igor On Sat, Nov 25, 2017 at 3:14 PM, Dhalgren Tor <dhalgren.tor@gmail.com> wrote:
Well, it's still going on, and is pretty much ruining Libero :( . Running CentOS 6, here.
When it's happening it can look like this:
# netstat -n | grep -c SYN 17696
I run a fast exit and can offer some advice:
1) mitigate bug #18580 (also #21394); is a DNS denial-of-service and could be the problem. either upgrade to 0.3.2.4+ or edit resolve.conf per https://trac.torproject.org/projects/tor/wiki/doc/DnsResolver#Tuningeventdns...
also check out https://arthuredelstein.net/exits/
2) if you continue to experience excessive outbound scanning SYNs, I've found that simply enabling connection tracking helps by implicitly limiting the rate a which connections can originate. If an iptables "-m state --state ESTABLISHED,RELATED -j ACCEPT" rule exists it will turn it on or you could modprobe the module if you don't want to configure incoming connection rules.
some useful sysctl.conf changes, run sysctl -p after or reboot
# Tor Exit tuning. net.ipv4.ip_local_port_range = 1025 65535 net.ipv4.tcp_synack_retries = 2 net.ipv4.tcp_max_tw_buckets = 4194304 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_fin_timeout = 30 net.ipv4.tcp_keepalive_time = 300 net.ipv4.tcp_keepalive_intvl = 10 net.ipv4.tcp_keepalive_probes = 3 net.netfilter.nf_conntrack_tcp_timeout_established = 1000 net.netfilter.nf_conntrack_checksum = 0
you might see many messages:
kernel: nf_conntrack: table full, dropping packet
which indicates no connection table slots were available for an outbound connection and that rate limiting is effected
state of connection tracking appears in /proc/net/nf_conntrack _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Sorry for the spam. One more link to a tuning guide that I have found useful: https://access.redhat.com/sites/default/files/attachments/20150325_network_p... On Sat, Nov 25, 2017 at 10:04 PM, Igor Mitrofanov <igor.n.mitrofanov@gmail.com> wrote:
After reading every paper and post on sysctl.conf and iptables tuning I could find, and reading some kernel code, I have come to a conclusion that, while there are a few settings to tune (can share mine, but your mileage *will* vary), most of the defaults are actually not broken in the latest kernels, while a lot of tuning recommendations online are outdated.
I enjoyed this guide a lot: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/htm.... It is written for Red Hat, but contains quite a few insights applicable to all Linux distributions.
- Igor
On Sat, Nov 25, 2017 at 3:14 PM, Dhalgren Tor <dhalgren.tor@gmail.com> wrote:
Well, it's still going on, and is pretty much ruining Libero :( . Running CentOS 6, here.
When it's happening it can look like this:
# netstat -n | grep -c SYN 17696
I run a fast exit and can offer some advice:
1) mitigate bug #18580 (also #21394); is a DNS denial-of-service and could be the problem. either upgrade to 0.3.2.4+ or edit resolve.conf per https://trac.torproject.org/projects/tor/wiki/doc/DnsResolver#Tuningeventdns...
also check out https://arthuredelstein.net/exits/
2) if you continue to experience excessive outbound scanning SYNs, I've found that simply enabling connection tracking helps by implicitly limiting the rate a which connections can originate. If an iptables "-m state --state ESTABLISHED,RELATED -j ACCEPT" rule exists it will turn it on or you could modprobe the module if you don't want to configure incoming connection rules.
some useful sysctl.conf changes, run sysctl -p after or reboot
# Tor Exit tuning. net.ipv4.ip_local_port_range = 1025 65535 net.ipv4.tcp_synack_retries = 2 net.ipv4.tcp_max_tw_buckets = 4194304 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_fin_timeout = 30 net.ipv4.tcp_keepalive_time = 300 net.ipv4.tcp_keepalive_intvl = 10 net.ipv4.tcp_keepalive_probes = 3 net.netfilter.nf_conntrack_tcp_timeout_established = 1000 net.netfilter.nf_conntrack_checksum = 0
you might see many messages:
kernel: nf_conntrack: table full, dropping packet
which indicates no connection table slots were available for an outbound connection and that rate limiting is effected
state of connection tracking appears in /proc/net/nf_conntrack _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
kernel: nf_conntrack: table full, dropping packet
If rules are dropping exit traffic based on other than traffic content, it's very hard to say other users are not adversly affected with the same, likely quite unsophisticated, hammer. And doing it based on content usually comes with major legal hurdles, besides being arbitrary. And both ways can get you dropped with badexit flag. Further, kernel dropping of packets is not signaled back into tor daemons for exitpolicy management therein, much less back to clients to avoid the censorship. And dropped packets hurts performance. Exitpolicy reject is the preferred method. Don't like the exit traffic, don't advertise to clients that you will carry it outbound in the first place. Please move this talk about tor exit traffic to a new thread or put it back to the first one where it came from.
participants (4)
-
Dhalgren Tor -
grarpamp -
I -
Igor Mitrofanov