Another Virtual Network Environment needs your I/O
Hi folks, During this hot summer I did some cool programming which resulted in... Another Virtual Network Environment (avne) AVNE is a small Linux program that runs other programs inside a virtual network environment. All network traffic from this environment is intercepted and forwarded to the Tor network. Interception is transparent and is guaranteed to be 100% effective. To forward network traffic you can simply type avne followed by a program name. Examples: avne bash avne ip link avne iceweasel -P avne -no-remote avne chromium --disable-cache avne wget https://blog.torproject.org/ Because the network interception takes place at the IP-level, no special Tor configuration is needed. In theory all TCP-clients should now be able to use the Tor network. AVNE is currently in the alpha phase of development. It works quite well, but for the for the first beta version I need input from this community. First of all I want to know if there are programs that do not work with avne. I tested with firefox, chromium, wget etc. and encountered no problems. The second question I have is: are there programs / protocols which should be blocked by avne? For example: I think its a bad idea to support unencrypted POP3. Of course I am also interested in bug-reports and your ideas for improvement. On my blog I wrote an article about avne that explains the technologies it uses (User Space Networking and Lightweight virtualization), and also gives an overview of its implementation. You can find the article here: https://hoevenstein.nl/another-virtual-network-environment The source code can be downloaded at: https://hoevenstein.nl/downloads An on-line viewable version of the code is at: https://hoevenstein.nl/source-code-avne-c Hope you like the ideas and code. And of course, your feedback is much appreciated! Rob. https://hoevenstein.nl
On Fri, 16 Oct 2015 15:03:49 +0200 Rob van der Hoeven <robvanderhoeven@ziggo.nl> wrote: [snip]
Because the network interception takes place at the IP-level, no special Tor configuration is needed. In theory all TCP-clients should now be able to use the Tor network.
Cute. The networking part works in a near-identical manner Orbot's Android VPN mode, under the hood, except they opted to use a 3rd party implementation (that bundles lwIP IIRC). Why did you write your own IP/ICMP/TCP (partial) implementations instead of using something like lwip? As your blog post notes "There are still some loose ends", most of these should be known issues (From quickly skimming the code, there is a possibility I've missed things or I'm wrong): * No IPv6. * No congestion control (TCP Tahoe is fairly trivial to implement, and would be more than sufficient). * Things will break/perform poorly if there ever is loss on the TUN interface (Unlikely, I know). * Out-of-order but in window segments are discarded, instead of stashed for use later. * No slow retransmission timer. * The implementation does not ACK enough. Send a dup. ACK when you receive something that's Out-of-order but in window. Probably doesn't matter much since the stack is never exposed to a public network and is only ever within the confines of a single host.
The second question I have is: are there programs / protocols which should be blocked by avne? For example: I think its a bad idea to support unencrypted POP3.
I think users should be allowed to shoot themselves in the foot if they want to.
Of course I am also interested in bug-reports and your ideas for improvement.
Code looks fairly clean. I'd feel better if it was in a memory safe language, but I like C/C++11 more than all the new popular stuff so I probably would have made a similar decision. Should be easily portable to the U*IXes, Windows will give you pain, but I'm not sure you care. I assume you use version control for the code and are just not exposing it to the world. Regards, -- Yawning Angel
On Fri, 2015-10-16 at 15:31 +0000, Yawning Angel wrote:
Cute. The networking part works in a near-identical manner Orbot's Android VPN mode, under the hood, except they opted to use a 3rd party implementation (that bundles lwIP IIRC).
Interesting...
Why did you write your own IP/ICMP/TCP (partial) implementations instead of using something like lwip?
When I was researching my idea I came across lwIP and was planning to use it. Unfortunately I could not find documentation. It's not on the project homepage and the wikia pages were not helpful too. What I needed was the info provided by the lwIP paper by Adam Dunkel. Found this paper yesterday trough the wikipedia article about lwIP. I always try to understand a technology that I use, even if I use an implementation by someone else. So I studied the TCP protocol. From this I got the idea that a full TCP protocol implementation was not needed to correctly handle *my* tun traffic. Writing a simplified TCP protocol would give me some major advantages: perfect integration with my design and no extra external dependencies.
As your blog post notes "There are still some loose ends", most of these should be known issues (From quickly skimming the code, there is
Fair enough, the loose ends are: - The proxy uses two connections, an upstream connection and a TCP connection. If one of the two connections closes, the other connection is closed immediately. There is however a (very?) small change that there is still data in a buffer that must be written to the open connection. The open connection should close after this data is written. - Needs more testing. I have tested the program in a loose way. For hours and hours I have watched YouTube videos, visited nasty websites (lots of advertisements, scripts and simultaneous connections), and downloaded very large files using WGET. Everything seems to work, but *seems* is not enough it most be *proven* to work correctly. - No unit tests.
a possibility I've missed things or I'm wrong):
* No IPv6. * No congestion control (TCP Tahoe is fairly trivial to implement, and would be more than sufficient).
Congestion control is used to prevent dropped segments. This can not happen on the User Space <-> kernel connection of a tun interface. The TCP-window flow control prevents this.
The second question I have is: are there programs / protocols which should be blocked by avne? For example: I think its a bad idea to support unencrypted POP3.
I think users should be allowed to shoot themselves in the foot if they want to.
"Want to" implies that the user is aware of the danger. Most users have no idea about the dangers of using a particular protocol over Tor. I want to protect these users.
Should be easily portable to the U*IXes, Windows will give you pain, but I'm not sure you care.
I care, and I think it can be ported to Windows. Since there are no kernel namespaces, another way to force all traffic through the tun interface must be found. It can be done by using the Windows firewall like this: https://community.hide.me/tutorials/bind-your-connection-to-vpn-with-windows... Regards, Rob. https://hoevenstein.nl
On Sun, 18 Oct 2015 13:08:01 +0200 Rob van der Hoeven <robvanderhoeven@ziggo.nl> wrote: [snip]
When I was researching my idea I came across lwIP and was planning to use it. Unfortunately I could not find documentation. It's not on the project homepage and the wikia pages were not helpful too. What I needed was the info provided by the lwIP paper by Adam Dunkel. Found this paper yesterday trough the wikipedia article about lwIP.
I always try to understand a technology that I use, even if I use an implementation by someone else. So I studied the TCP protocol. From this I got the idea that a full TCP protocol implementation was not needed to correctly handle *my* tun traffic. Writing a simplified TCP protocol would give me some major advantages: perfect integration with my design and no extra external dependencies.
Sounds reasonable, and yeah the lwIP documentation isn't great.
As your blog post notes "There are still some loose ends", most of these should be known issues (From quickly skimming the code, there is
Fair enough, the loose ends are:
- The proxy uses two connections, an upstream connection and a TCP connection. If one of the two connections closes, the other connection is closed immediately. There is however a (very?) small change that there is still data in a buffer that must be written to the open connection. The open connection should close after this data is written.
Fairly easy.
- Needs more testing. I have tested the program in a loose way. For hours and hours I have watched YouTube videos, visited nasty websites (lots of advertisements, scripts and simultaneous connections), and downloaded very large files using WGET. Everything seems to work, but *seems* is not enough it most be *proven* to work correctly.
- No unit tests.
a possibility I've missed things or I'm wrong):
* No IPv6. * No congestion control (TCP Tahoe is fairly trivial to implement, and would be more than sufficient).
Congestion control is used to prevent dropped segments. This can not happen on the User Space <-> kernel connection of a tun interface. The TCP-window flow control prevents this.
Hm. Your code never shrinks the advertised window, and ACKs once when data has been added to the receive buffer. As a perfectionist for such things, this makes me really sad, but since it the stack never sees the light of day outside a TUN interface, the behavior is probably ok. As a side note, the TUN driver will happily drop packets once the backlog exceeds 500 or if a skb allocation fails. Both unlikely, both can happen. [snip]
"Want to" implies that the user is aware of the danger. Most users have no idea about the dangers of using a particular protocol over Tor. I want to protect these users.
Fair enough. I'd suggest this be configurable behavior. See `WarnPlaintextPorts` in the tor manpage for what should be a sensible list. If you want to protect users from danger, you'd probably also want to disallow port 80 and 443 by default, because they're giving up a lot by using what is probably the Wrong Browser[1]. They may even think that viewing Flash/Java content is safe, when neither are because the plugins are all sorts of terrible.
Should be easily portable to the U*IXes, Windows will give you pain, but I'm not sure you care.
I care, and I think it can be ported to Windows. Since there are no kernel namespaces, another way to force all traffic through the tun interface must be found. It can be done by using the Windows firewall like this:
https://community.hide.me/tutorials/bind-your-connection-to-vpn-with-windows...
Guess you'll use the OpenVPN TUN/TAP driver? Some more random things: * PSH is set incorrectly. You set PSH when you're ACKing received data (without piggybacked payload). You're supposed to PSH as part of sending data to the peer (See RFC 1112 4.2.2.2). * DNS resolution does not work if /etc/resolv.conf points at 127.0.0.1. This probably is a documentation thing rather than a code thing. * I couldn't figure out how to get X apps to work at all. (Eg: chromium fails with "Gtk: cannot open display: :0") * There should be documentation that this requires a minimum of CAP_SYS_ADMIN (for the various unshare() calls) and CAP_NET_ADMIN (to bring the tun interface up). * The config file load/parse routine has a trivially exploitable buffer overflow. -- Yawning Angel [1]: Tor Browser would be the Less Wrong Browser.
On Sun, 2015-10-18 at 14:43 +0000, Yawning Angel wrote:
Congestion control is used to prevent dropped segments. This can not happen on the User Space <-> kernel connection of a tun interface. The TCP-window flow control prevents this.
Hm. Your code never shrinks the advertised window, and ACKs once when data has been added to the receive buffer. As a perfectionist for such things, this makes me really sad, but since it the stack never sees the light of day outside a TUN interface, the behavior is probably ok.
I never adjust the size of the TCP window, that's correct. The code only sends an ACK for data that is *removed* from the buffer. If data is added to the buffer, the amount of data the TCP-client is allowed to send decreases. Eventually becoming zero if no data is removed at all.
"Want to" implies that the user is aware of the danger. Most users have no idea about the dangers of using a particular protocol over Tor. I want to protect these users.
Fair enough. I'd suggest this be configurable behavior. See `WarnPlaintextPorts` in the tor manpage for what should be a sensible list.
Good idea.
Should be easily portable to the U*IXes, Windows will give you pain, but I'm not sure you care.
I care, and I think it can be ported to Windows. Since there are no kernel namespaces, another way to force all traffic through the tun interface must be found. It can be done by using the Windows firewall like this:
https://community.hide.me/tutorials/bind-your-connection-to-vpn-with-windows...
Guess you'll use the OpenVPN TUN/TAP driver?
Yep. One thing: At the moment I do not have the resources to work on a Windows version of the program. Creating a consumer-grade Linux version is my current goal.
Some more random things: * PSH is set incorrectly. You set PSH when you're ACKing received data (without piggybacked payload). You're supposed to PSH as part of sending data to the peer (See RFC 1112 4.2.2.2).
Ok.
* DNS resolution does not work if /etc/resolv.conf points at 127.0.0.1. This probably is a documentation thing rather than a code thing.
Maybe, I will look into this.
* I couldn't figure out how to get X apps to work at all. (Eg: chromium fails with "Gtk: cannot open display: :0")
Ah, your system probably uses an Abstract Unix Socket for communication with the X server. Abstract Unix Sockets are created inside a network namespace, and your X server socket lives inside the global network namespace. To solve this I have to write proxy code to create an X server socket inside the network namespace of the TCP client. Maybe you can temporarily solve the problem by binding your X server socket to a normal Unix socket? (filesystem Unix Sockets are not network namespace aware). What OS (name+version) are you using? I'm using Debian Wheezy which does not have this problem.
* There should be documentation that this requires a minimum of CAP_SYS_ADMIN (for the various unshare() calls) and CAP_NET_ADMIN (to bring the tun interface up).
AVNE is a suid program at the moment. To do the privileged calls the program runs (for a while) with root privileges. I'm going to move to Debian Jessie. This version has a newer kernel that supports user namespaces. As I understand it (have not played with user namespaces), these can be used to create programs that can have full privileges inside the user namespace without having full privileges themselves. No more suid needed. The downside is that user namespaces are only available for kernels with versions >= 3.8
* The config file load/parse routine has a trivially exploitable buffer overflow.
The final config file will be owned by root and stored in /etc. Good input Yawning! Rob. https://hoevenstein.nl
On Sun, 18 Oct 2015 19:20:47 +0200 Rob van der Hoeven <robvanderhoeven@ziggo.nl> wrote:
I never adjust the size of the TCP window, that's correct. The code only sends an ACK for data that is *removed* from the buffer. If data is added to the buffer, the amount of data the TCP-client is allowed to send decreases. Eventually becoming zero if no data is removed at all.
Yeah. Since your receive window isn't ever sizable, and the interface is rather reliable, this behavior is probably the simplest. [snip]
* I couldn't figure out how to get X apps to work at all. (Eg: chromium fails with "Gtk: cannot open display: :0")
Ah, your system probably uses an Abstract Unix Socket for communication with the X server. Abstract Unix Sockets are created inside a network namespace, and your X server socket lives inside the global network namespace. To solve this I have to write proxy code to create an X server socket inside the network namespace of the TCP client. Maybe you can temporarily solve the problem by binding your X server socket to a normal Unix socket? (filesystem Unix Sockets are not network namespace aware).
What OS (name+version) are you using? I'm using Debian Wheezy which does not have this problem.
Arch. It's a rolling release system, and it's up to date-ish. It's not a big deal at the moment...
* There should be documentation that this requires a minimum of CAP_SYS_ADMIN (for the various unshare() calls) and CAP_NET_ADMIN (to bring the tun interface up).
AVNE is a suid program at the moment. To do the privileged calls the program runs (for a while) with root privileges.
Oh, well. Using setcap and granting the relevant capabilities also works, though dropping the capabilities requires a bit more work. The elevated capabilities aren't persisted across exec() at least so the child side handling is easy.
I'm going to move to Debian Jessie. This version has a newer kernel that supports user namespaces. As I understand it (have not played with user namespaces), these can be used to create programs that can have full privileges inside the user namespace without having full privileges themselves. No more suid needed. The downside is that user namespaces are only available for kernels with versions >= 3.8
Well. Debian requires enabling it by writing to proc (they have a patch for this). Arch flat out doesn't include support for that in the vanilla kernel (https://bugs.archlinux.org/task/36969?project=1), ditto anyone using grsec. It's probably not that much code to support user namespaces, running with just the required capabilities, and running as suid root in the same code base.
* The config file load/parse routine has a trivially exploitable buffer overflow.
The final config file will be owned by root and stored in /etc.
Ah I see (I assume/hope you'll fix this anyway). (Will there also ever be an option for configuring a different tun address?) Neat project. I'll be looking forward to subsequent releases, since this is slick, and I think a better approach than torsocks once it matures a bit. Regards, -- Yawning Angel
participants (2)
-
Rob van der Hoeven -
Yawning Angel