On 19 Jan 2018, at 06:06, Quintin tor-admin@portaltodark.world wrote:

No outbound filters, this is my config:

*filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -p tcp -m comment --comment "SSH" -s x.x.x.x -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -p tcp -m comment --comment "Tor" -m state --state NEW -m tcp --dport 80 -j ACCEPT -A INPUT -p tcp -m comment --comment "Tor" -m state --state NEW -m tcp --dport 443 -j ACCEPT -A INPUT -j REJECT --reject-with icmp-host-prohibited -A FORWARD -j REJECT --reject-with icmp-host-prohibited COMMIT

If I stop tor then "dig @127.0.0.1 google.com" works 100%. It's seems like the pattern is that when tor traffic builds up so do DNS failures. And then my dig @127.0.0.1 only succeeds about 0.1% of the time. At this stage large amounts these errors start appearing:

...

Jan 17 19:27:33 torexit unbound: [559:0] notice: remote address is 192.42.93.30 port 53 Jan 17 19:27:33 torexit unbound: [559:0] notice: sendto failed: Operation not permitted

Try setting RelayBandwidthRate to 95% of your link capacity. Then wait a few hours.

If you are still having issues: * check if you have a lot of inbound connections from a small number of IPs, * read recent threads for firewall rules to limit inbound connection load.

T

Do you reach your server's conntrack limit?

The word conntrack never appears in my logs, so I don't think it's that. The ISP also requires this from tor exits: net.netfilter.nf_conntrack_max = 10000

Try setting RelayBandwidthRate to 95% of your link capacity.

Why 95%? Are you thinking to give it more bandwidth?

From the IPs in your logs I assume your unbound is configured to query recursively itself (no upstream forwarding) that is good, can you confirm

that

and provide your unbound config + iptalbes -vnL?

Correct, unbound is recursive. Here's the config: server: verbosity: 1 statistics-interval: 0 statistics-cumulative: no extended-statistics: no num-threads: 2 interface-automatic: no do-ip6: no chroot: "" username: "unbound" directory: "/etc/unbound" log-time-ascii: yes pidfile: "/var/run/unbound/unbound.pid" harden-glue: yes harden-dnssec-stripped: yes harden-below-nxdomain: yes harden-referral-path: yes use-caps-for-id: no unwanted-reply-threshold: 10000000 prefetch: yes prefetch-key: yes rrset-roundrobin: yes minimal-responses: yes module-config: "validator iterator" trusted-keys-file: /etc/unbound/keys.d/*.key auto-trust-anchor-file: "/var/lib/unbound/root.key" val-clean-additional: yes val-permissive-mode: no val-log-level: 1 include: /etc/unbound/local.d/*.conf remote-control: control-enable: no server-key-file: "/etc/unbound/unbound_server.key" server-cert-file: "/etc/unbound/unbound_server.pem" control-key-file: "/etc/unbound/unbound_control.key" control-cert-file: "/etc/unbound/unbound_control.pem" include: /etc/unbound/conf.d/*.conf

Quintin

Ah, thats it. My conntrack entries are full and temporarily increasing it resolves the problem.

What would be a reasonable conntrack limit for a tor exit?

On Thu, Jan 18, 2018 at 10:45 PM nusenu nusenu-lists@riseup.net wrote:

Quintin:

...

...

Do you reach your server's conntrack limit?

The word conntrack never appears in my logs, so I don't think it's that. The ISP also requires this from tor exits:

net.netfilter.nf_conntrack_max =

...

10000

How many conntrack entries do you actually have when you get sendto failed: Operation not permitted log entries?

sysctl net.netfilter.nf_conntrack_count or cat /proc/sys/net/netfilter/nf_conntrack_count

Regardless of whether this is the root-cause or not, nf_conntrack_max = 10k is probably to low for an exit relay.

If nf_conntrack_count is near nf_conntrack_max, does the problem go away when you temporarily increase nf_conntrack_max?

-- https://mastodon.social/@nusenu twitter: @nusenu_

I can kinda answer that.

I run an exit node that happily does 200-250mbit/s according to netdata accounting and my monitoring regularly pegs it at nearly 200k connections. Usually 100-150k.

On Sun, Jan 21, 2018 at 4:06 PM, nusenu nusenu-lists@riseup.net wrote:

Quintin:

...

Ah, thats it. My conntrack entries are full and temporarily increasing it resolves the problem.

I'm glad we found the problem and the solution.

Your exit appears to be offline since 2018-01-20 20:00, expected downtime? https://atlas.torproject.org/#details/92E3764D5485DC4AC01178271FB5A8A2D90DA9...

...

What would be a reasonable conntrack limit for a tor exit?

The amount of states depend on your consensus weight (and probably exit policy), do you require a stateful packet filter?

-- https://mastodon.social/@nusenu twitter: @nusenu_

---

tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

Quintin:

Seems my VPS got suspended when I increased the connlimit above 10000. Do you think my INPUT filters which use conntrack could have caused this issue?

You did confirm that already, no?

If your hoster suspends your server if you exceed 10k concurrent connections I'm afraid it is probably not suitable for an exit relay (regardless of your own iptables ruleset).

A non-exit (single instance) relay would fit into a 10k limit.