On Wed, Feb 05, 2020 at 02:46:58PM -0500, Cecylia Bocovich wrote:
I've made some updates to snowbox to easily run all of the pieces needed for Turbo Tunnel :
Thanks for doing that.
It took me a while to wrap my head around the fact that the QueuePacketConn is a single abstract connection that handles *all* incoming and outgoing traffic for *all* clients. The result is a relatively clean interface with turbotunnel in the actual server code while there's a lot going on behind the scenes.
That's right. The analogy is with a UDP server, which is also a PacketConn. You do ListenUDP once, and it does duty for all clients. In fact, that's the typical way of using KCP, you do it over UDP. But the kcp-go interface allows replacing the UDPConn with any other type implementing the PacketConn interface.
The behaviour I am still unsure about is which websocket connection the data from the server (data going from the server to the client) is written to. From what I can tell, each new websocket connection from a proxy will pull from the OutgoingQueue that corresponds to the clientID of the connection until the connection times out. This means that, since the server is not in charge of redial, there are potentially multiple connections pulling from this queue. If a connection is dropped and a new one redialed at the client, the server may write data out to the dropped connection instead of the newer redialed connection. Presumably KCP will take care of retransmitting the dropped packet, but I'm curious about the latency cost here. It's also a bit different from an earlier proposal to do connection migration similar to Mosh: https://github.com/net4people/bbs/issues/14
This is a good observation. Indeed, not only could there be an about-to-die WebSocket connection pulling packets from OutgoingQueue, losing them, and forcing another WebSocket connection to retransmit them; there could be multiple simultaneous live WebSocket connections with the same ClientID, all pulling from OutgoingQueue at once (as would happen in a setup like https://bugs.torproject.org/25723). I haven't tried it yet, but my guess is that it will all just work out. When a WebSocket dies, it will lose some number of packets, but those will be retransmitted after a timeout that I believe KCP dynamically calculates. And if there are multiple, each will deliver packets as fast as it can support, and the client will take care of reordering them. It may cause some RTT estimation algorithm to freak out, but it's not really different than TCP segments taking different routes within a connection.
I did the connection migration more straightforwardly in the obfs4proxy implementation. The clientMap type in Snowflake is a descendant of connMap in obfs4proxy, and in fact they store almost the same data: In obfs4proxy: type connMapRecord struct { Addr net.Addr LastSeen time.Time Conn net.Conn } In Snowflake: type clientRecord struct { Addr net.Addr LastSeen time.Time SendQueue chan []byte } The difference is that Snowflake stores a (shared) SendQueue instead of single Conn. The advantage is that many Conns can pull from the shared queue at once. It's like a more general version of connection migration. The obfs4proxy implementation also would have supported a limited version of multiplexing across multiple connections simultaneously, but it would be dependent on the client consciously striping its sends over the connections, in order to coax the server into updating its Conn mapping constantly.
But the situation in the turbotunnel branch is better than the status quo, even without multiplexing, for two reasons. First, the connection actually *can* recover after 30 seconds. Second, the smux layer sends keepalives, which means that you won't discard a proxy merely because you're temporarily idle, but only when it really stops working.
Yes! This is really great work. We should talk at the next anti-censorship meeting perhaps on the steps we should take to get this merged and deployed.
I was going to propose deploying the backward-compatible turbotunnel server to the public bridge, then making some Tor Browser builds that our helpers on the issue tracker can try out.
Minor note:
- There is some old buffering code at the client side that could be
rolled into the new RedialPacketConn:
https://gitweb.torproject.org/user/dcf/snowflake.git/tree/client/lib/webrtc.... https://gitweb.torproject.org/user/dcf/snowflake.git/tree/client/lib/webrtc....
Thanks, I would not have noticed this myself. I guess the original idea was that WebRTCPeer would be not a single WebRTC connection, but a wrapper for a sequence of WebRTC connections? That's why you can Connect it, Reset it, and Connect it again. I suppose we can eliminate all those features and make WebRTCPeer less stateful by having it represent just one WebRTC connection. RedialPacketConn takes care of sequencing multiple connections.