[tor-relays] Did 'Sandbox 1' break Tor for anyone else on 0.4.5.6?

[William Kane]

Hi Peter,

>Would be great if you could get details about the failing call.

I already thought of gathering said details by tracing the process,
but did not want to risk my uptime statistics, which would inevitably
happen if I had to restart the server and service over and over (I
disabled tracing globally through the Yama LSM as a security measure,
i.e. kernel.yama.ptrace_scope == 3) - recently I lost the guard flag
multiple times, caused by some sort of attack that I already reported
on this list (tor-relays) - someone kept creating a fuckton of
circuits through my relay (averaging 90k per minute), thus causing tor
to run out of memory / get oom-killed by the kernel before it could
even step in and close the circuits - if it was even trying to, it
would make sense for the DoS mitigation code to be active only for the
first link in the circuit aka the guard, and my node simply being a
middle-only relay, it got completely stomped by said attack.

After somewhat mitigating this attack by tweaking MaxMemInQueues,
creating a bigger swap file and tuning vm.swappiness, I regained the
guard flag, but then the hypervisor my KVM box is running under
experienced some issues and had to be rebooted - once again, I
received no notice of that until the relay was already offline for a
few days, causing me to lose the guard flag again.

Seems like luck is just not on my side these days, or well, it's been weeks now.

> You should simply see a Permission Denied if the capability is the problem.

Here's a copy from stdout, only happening if Sandbox is set to 1.:

Mar 15 20:15:20.000 [notice] Configured to measure statistics. Look
for the *-stats files that will first be written to the data directory
in 24 hours from now.
Mar 15 20:15:21.000 [warn] fstat() on directory /var/lib/tor_debug failed.
Mar 15 20:15:21.000 [err] Can't create/check datadirectory /var/lib/tor_debug
Mar 15 20:15:21.000 [err] Error initializing keys; exiting

Running it as a privileged user does not change thing, so no permissions issue:

Mar 15 20:17:24.000 [notice] Configured to measure statistics. Look
for the *-stats files that will first be written to the data directory
in 24 hours from now.
Mar 15 20:17:24.000 [warn] You are running Tor as root. You don't need
to, and you probably shouldn't.
Mar 15 20:17:25.000 [warn] fstat() on directory /var/lib/tor_debug failed.
Mar 15 20:17:25.000 [err] Can't create/check datadirectory /var/lib/tor_debug
Mar 15 20:17:25.000 [err] Error initializing keys; exiting

I've traced down the origin of the fstat() call to this piece of code:

https://github.com/torproject/tor/blob/master/src/lib/fs/dir.c#L158

However, looking at the code that establishes and populates seccomp
rules, it seems like fstat and it's 64 bit counterpart are not subject
to (parameter) filtering, i.e. seccomp_rule_add_0 is invoked with the
parameter SCMP_ACT_ALLOW, reading the manpage for seccomp_rule_add(3)
reveals: "The seccomp filter will have no effect on the thread calling
the syscall if it matches the filter rule."

References:

https://github.com/torproject/tor/blob/master/src/lib/sandbox/sandbox.c#L148
https://github.com/torproject/tor/blob/master/src/lib/sandbox/sandbox.c#L1595
https://man7.org/linux/man-pages/man3/seccomp_rule_add.3.html

So, even though technically, seccomp should allow these syscalls to be
invoked, no matter which parameters are passed, somehow enabling the
whole sandbox subsystem still breaks fstat.

We are getting into tor-dev@ territory here, so maybe someone who is
more experienced with the code base could figure out why this is
happening, I'm pretty good at C/C++ on Linux / Windows and all the
shenanigans that come with it, but lack time to debug this further.

I've added tor-dev at lists.torproject.org as a CC just in case.

- William

2021-03-15 17:10 GMT, Peter Gerber <tor at arbitrary.ch>:
> Hi William
>
> William Kane:
>> Hi everyone,
>>
>> Ever since I upgraded to tor version 0.4.5.6, enabling tor's built-in
>> seccomp sandbox completely breaks tor, i.e. it gets killed by the
>> kernel on start for a seccomp violation (fstat(..)) - sandboxing
>> worked fine on 0.4.4.6, my system configuration did not change between
>> the updates.
>
> Tor itself usually fails with a Permission Denied error when a syscall
> fails due to seccomp. So, this is rather odd.
>
>> I figured this was happening because I do not grant the
>> CAP_DAC_READ_SEARCH capability, but I'm not so sure anymore if that's
>> the reason.
>
> You should simply see a Permission Denied if the capability is the problem.
>
> Would be great if you could get details about the failing call. If
> seccomp is involved, you should be able to get details like this:
>
> • install package auditd
> • make sure auditd is running
> • crash Tor
> • find the syscall with `ausearch -ts recent -i`
>
> Peter
>