Hello all,
I've discovered that the Linux kernel appears to have a leak in how it applies transproxy rules to the TCP CLOSE_WAIT shutdown condition under certain circumstances. This applies to both the kernels in use by common Android devices (Cyanogenmod 10.x and 11-M4), as well as the Linux kernel in Ubuntu 13.04 (3.8.0-35-generic).
The bug can be triggered either by a remote server closing a connection, or by restarting the local tor client.
Basically, the bug happens when a transproxy connection shuts down completely before a client application properly closes the socket. This seems to cause the kernel to lose track of the fact that the client application connection was being transproxied, and when the client application finally does close its socket (or exits), the Linux kernel generates a FIN ACK that completely bypasses any transproxy rules you have installed. It sends this packet first as the UID of the app in question, and if that fails, it resends it as a blank UID (the kernel itself).
Here's how to reproduce it and see for yourself:
First run the attached iptables script, and launch a tor daemon with the attached torrc (edit the iptables script's TOR_UID=`id -u debian-tor` and NETWORK_USER_ID=1000 vars if your setup is different).
Then, fire up tcpdump, like so: sudo tcpdump -n -i wlan0 host 74.125.28.104 and tcp port 80
Replace '-i wlan0' with your network interface. If you use '-i any', you will also see transproxied packets (which are not normally leaked).
Then, as your transproxied user, paste this python snippet into a python interpreter: import socket s = socket.create_connection(("74.125.28.104", 80))
(That IP handles www.google.com).
After the connection is made, you should see something like the following in 'netstat -natp': tcp 0 0 127.0.0.1:9040 192.168.1.23:42235 ESTABLISHED 1121/tor tcp 0 0 192.168.1.23:42235 74.125.28.104:80 ESTABLISHED 977/python
At this point, either wait a couple minutes for Google to close that connection on you, or shut down your Tor daemon. In either case, you should see the first connection transition to TIME_WAIT, or FIN_WAIT2, or similar, lose track of its PID+UID, and then finally disappear entirely, but the python program will remain in CLOSE_WAIT indefinitely.
Once the first connection is fully gone (this takes 60s from TIME_WAIT state with default TCP settings), issue this in your python shell: s.close()
At this point, you will see a FIN ACK or RST ACK packet appear in your tcpdump window. That packet has leaked past the iptables firewall rules, and past the transproxy rules. It went straight to Google.
I have noticed several Android apps (including Firefox, F-Droid, and many Google apps and Android services) that allow their sockets to sit in CLOSE_WAIT upon remote close while transproxied, and they all leak packets in this case, which happens frequently in normal usage. I am not sure if this is just a common programming error, an issue with how the Android networking APIs are designed, something specifically exacerbated by the transproxy, or some combination of these.
For a workaround, I was able to prevent this issue with the addition of the following rules: iptables -I OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,FIN ACK,FIN -j DROP iptables -I OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,RST ACK,RST -j DROP
None of the transproxy documentation I could find mentions this issue, nor suggests any additional safety rules. This means every transproxied Tor user is unwittingly leaking packets, at least some of the time. Sorry to be the bearer of bad news.
Please send workaround discussion to tor-talk, and kernel/TCP state machine discussion to tor-dev. I Cc'd both like a jerk, because I figure each group might have different sets of commentary, and both groups should be aware of this issue. Don't be a jerk like me, please. Use your best judgment to Cc one list or the other.