On Tue, Feb 04, 2020 at 01:17:33AM -0500, Roger Dingledine wrote:
Awesome news about the turbotunnel integration. I've been pondering the backward compatibility thing ever since your first mail. Here are two hopefully useful thoughts. Let me know if they are misunderstanding things in some way.
FYI, we've discussed these and other approaches at https://bugs.torproject.org/30704 and https://bugs.torproject.org/29206.
(A) An extra 8 bytes for every new or resumed flow, forever, is kind of sad. But it doesn't have to be forever: in the future, once the old Snowflakes that don't send that static token are long gone, we can stop requiring the token. (That is, if clients send it, that's fine, we can recognize and strip it. And if they don't send it, that's fine too.)
Personally I don't feel 8 bytes is anything to cry over. Let it be 4 bytes, or 1 byte, or 16 bytes. Compare it to the overhead of a TLS connection or a WebSocket negotiation. Compare it to the overhead of Tor's cell padding or the 24-byte KCP header. Especially considering that it's amortized over the life of a proxy. 8 bytes every 100 KB is hardly worth considering, in my opinion.
(B) Another approach to backward compatibility might be to leave the old Snowflake bridge up, not knowing what a turbotunnel is, and to put up a new one nearby or somewhere else that expects turbotunnel connections. Then after a few releases of Tor Browser, we could shut down the old one and move on.
This is a reasonable idea, but certain characteristics of the current Snowflake deployment make it more difficult. The key element is that it requires changes in proxy code and proxies to upgrade. The reason for this is that you'll need some proxies to serve the old bridge and some to serve the new. So you could partition your pool of proxies—each one choosing which bridge it will use randomly at startup, for example. You would then need to run two brokers too, so that Turbo Tunnel clients get matched with Turbo Tunnel proxies, and likewise with the non–Turbo Tunnel clients and proxies. Or instead of running two brokers, you could have the client, indicate in its rendezvous message, which protocol it intends to use, and then the broker, when matching the client up with a proxy, would do something like https://bugs.torproject.org/25598 and simply instruct the proxy as to what bridge it should use. But beside the fact that none of that is implemented, the protocol change is no longer an end-to-end matter between clients and the bridge; it's something that all pieces of the pipeline have to be aware of: client, broker, proxy, and bridge. And the client now will have to spend some number of bytes in its rendezvous message to indicate what protocol it's using: the overhead moves out of the client→proxy flow and into the client→broker flow. And all other things being equal, we prefer to send bytes on the client→proxy flow rather than the client→broker flow, because the domain-fronted CDN link is more expensive (in $$) than the WebRTC/WebSocket link. And on top of everything, you still end up with lingering code complexity; for example what happens if proxy joins that doesn't know how to obey the broker's instructions about what bridge to connect to? You need some logic in the client to detect the situation and fall back to the legacy protocol, or in the broker to detect and deny service to such proxies (with its own indeterminate sunset period?), or something.
Running a second copy of the bridge wouldn't require a second IP address or even a second port. Because it's an HTTP-based protocol, it could be a second URL path on the same server; say /tt for the Turbo Tunnel clients and / for the old clients, with a reverse proxy dispatching to one of two local servers. It's a feasible design. For me, when I weigh the cost of 25 lines of compatibility code living in the server itself, versus the added risk of now having to restart two things instead of one whenever doing a bridge upgrade, I tend to come down on the side of a small amount of compatibility code.