Hi everyone,
Cloudflare has added support to TLS 1.3 for encrypted server name indication (SNI). This mailing list post is a high level overview of how meek could take advantage of this in relation to Cloudflare who until just now wasn’t an option for domain fronting.
What this means: Effectively domain fronting works by sending a different SNI and host header. CDN providers like Cloudflare started double checking to make governments happy, scratch that line, I mean to protect their customers from fraud and abuse. They seem to of backtracked now. Encrypted SNI means that a firewall or coffee shop owner won’t be able to use SNI to see the real origin of TLS traffic.
Why this matters: With the right adjustments for TLS 1.3 and Encrypted SNI support, Cloudflare may be a viable option for Meek.
Risks: * Firewall products could always use DPI and block TLS 1.3 altogether. * Firewall products could block all requests with encrypted SNI.
Thoughts anyone?
References: * https://blog.cloudflare.com/encrypted-sni/ * https://blog.cloudflare.com/esni
On Mon, Sep 24, 2018 at 01:46:10PM -0400, Nathaniel Suchy wrote:
Why this matters: With the right adjustments for TLS 1.3 and Encrypted SNI support, Cloudflare may be a viable option for Meek.
Risks:
- Firewall products could always use DPI and block TLS 1.3 altogether.
- Firewall products could block all requests with encrypted SNI.
I wrote an essay on some of the implications of encrypted SNI here: https://groups.google.com/d/msg/traffic-obf/UyaLc9jPNmY/ovNImK5HEQAJ tl;dr: yes, encrypted SNI is a very good thing for censorship circumvention and privacy more generally; a risk though is that it further centralizes access to online resources.
----
2018-08-18
Efforts are underway to add SNI encryption as an extension in TLS 1.3. * https://tools.ietf.org/html/draft-rescorla-tls-esni-00 * https://www.ietf.org/mail-archive/web/tls/current/msg26842.html I find this a really hopeful development. I appreciate the work of everyone who is helping to make it a reality (some of them are on this list). Encrypted SNI will of course be a boon for online privacy generally, but in our world of censorship circumvention it could be the biggest thing since the ascendance of TLS. Along with its benefits, I foresee that encrypted SNI will change the basic game in ways that we need to be ready for. I expect that we'll need to reevaluate our customary models and begin to consider new challenges.
At first glance, encrypted SNI—in whatever form it may eventually take—is a silver bullet. It's domain fronting without the downsides. It solves all our problems, up to traffic analysis: payload encryption prevents blocking by content, and SNI encryption protects the destination address. The censor cannot distinguish various connections to a TLS server and faces the old dilemma: block all, or block none. And experience shows that we can find servers with a sufficient degree of co-hosting that a censor will hesitate to "block all."
So what's the catch? I don't really think there is one, at least not a major one. SNI encryption is poised to put censorship circumvention on substantially securer footing. As https://tools.ietf.org/html/draft-ietf-tls-sni-encryption-03 (which I encourage you to read) says, "Historically, adversaries have been able to monitor the use of web services through three channels... These channels are getting progressively closed." But what I do think SNI encryption will do is force us to reconsider our threat models. Solving our current batch of problems is going to uncover new problems—lesser problems, to be sure—but ones that we have until now mostly ignored because they were not the most pressing. Censors, too, will be forced evolve, when they are finally deprived of their last easy traffic distinguisher. I predict a displacement of the battleground, from the land of firewalls to new arenas.
It's a credit to everyone's work on domain fronting, and the basic soundness of the idea, that when it began to falter, it was not because of the Great Firewalls of the world, but because of the Googles and Amazons. This phenomenon is an example of what I mean when I say that old challenges will give way to new ones. We beat the censors at their game, and resisted direct blocking long enough for another weakness to reveal itself; i.e., that network intermediaries don't reliably perform the functions that we depended on. I mean that as an observation of fact, not as implied judgement—personally I don't really blame Google and Amazon for their policy change regarding domain fronting. While the wisdom of the decision is debatable, and I suspect there is more to their rationale than they have stated publicly, certainly they are under no obligation to continue supporting an unintended feature, no matter how useful we find it. But whatever the cause, the fact is that domain fronting, while demonstrably robust against border-firewall censors, is susceptible to the changing dispositions of intermediary services. We reached this frontier of experiential knowledge only because we had beaten the censor's usual tricks—we transcended the cat-and-mouse game. I like to draw an analogy with human health: our caveman forebears didn't worry about dying from heart disease, because it was almost certain that they would be killed by something else first, a woolly mammoth, say. It was only after the immediate threat of death by mammoth had subsided, that humans had the comparative luxury of being concerned about heart disease. So it is with us now: we built a cat-proof mouse, and now we see what other worries a mouse has.
I can see something similar playing out with encrypted SNI, only on a larger and more pervasive scale. Network intermediaries—CDNs, app stores, hosting providers—are going to face more and more pressure: as other links in the chain of communication are strengthened, those intermediaries will become attractive targets of action by censors. We already see examples of censors having to step out of their accustomed comfort zones because of generally increasing Internet security, for example when the government of China pressured Apple to yank VPN apps from its app store. I contend that if the government had the ability to block apps all by itself, without petitioning Apple, then that's what it would have done. That the censor resorted to pressuring a third party shows a certain weakness, but the fact that it succeeded shows it is still strong enough for its purposes. It also highlights a shift in moral responsibility. If the government were able to block apps without asking, then Apple could just throw up its hands and say: "not my fault." But because the censor has no choice but to ask, Apple must make the deliberate choice of whether to become an agent—however unwilling—of censorship.
We circumvention designers have customarily assumed that network intermediaries are benevolent, or at least non-malicious—that they do not collaborate with a censor. We assumed so, because the risk of direct blocking by a censor overshadowed any other risk. In a world of encrypted SNI, where the direct risk from the censor is greatly diminished, we will need to reexamine this assumption. Intermediaries will become de-facto gatekeepers of information, to an even greater degree than they are now, and they'll be in the unenviable position of being the logical place at which to implement censorship. As things stand now, when a court in India orders a site blocked, it's Airtel's problem to block it. But when encrypted SNI renders Airtel unable, it'll be Cloudflare's problem. Now, if I had to choose between the good will of Cloudflare et al. and that of the GFW, there's no comparison: obviously you choose Cloudflare every time. And yet, we can't overlook that Cloudflare once booted a site on a CEO's whim; nor that Google secretly started building a censored search engine for China. The operators of network intermediaries, and their commitment to human rights, will be tested more than ever, and the population of Internet users will increasingly rely on them to do the right thing.
As circumvention designers, one thing we can do to help those services help themselves is not to proxy directly from the services themselves, but to use at least one additional encrypted hop to an separate proxy server. That way, it becomes technically harder for the services to do surgical blocking.
I have to admit that I don't fully understand the apparent enthusiasm for encrypted SNI from groups that formerly were not excited about domain fronting. It's possible I misunderstand some subtlety, but to my mind, they both should amount to about the same thing from the their perspective. The primary difference is one of scale. The stated concerns with domain fronting also apply to encrypted SNI; in particular that if one customer gets blocked, it will have a collateral effect on other customers. Maybe the difference in scale is really it: the cloud companies are happier to bet against blocking when *all* their customer's domains are potentially affected, rather than just one. It's a rational enough viewpoint (greater potential collateral damage → lower probability of blocking), but to my mind encrypted SNI doesn't fundamentally alter the nature of the game, it just raises the stakes. Don't get me wrong: I welcome the adoption of encrypted SNI for whatever reason. It's better than domain fronting, it'll be nice to have it in a standard, and once we have it we won't want to go back. But I hope that operators understand what they're getting into. Will they get cold feet when push comes to shove—when a future version of Telegram uses encrypted SNI and Russia again blocks millions of IPs? Or malware adopts it for C&C and infosec blue teamers get annoyed?
I said earlier that I didn't see any major catch with encrypted SNI. The minor catch I see is the potential for increased centralization. TLS with encrypted SNI is likely to be the most effective form of circumvention, which means that unless you're a major player, or hosted on one, you'll be at increased risk of being blocked. I've read some criticism online of circumvention systems, like domain fronting, that route traffic through the major cloud companies. One the one hand, I find that kind of criticism annoying, because it's not that the use of centralized services is a desired, designed-in feature; it's that we don't yet know how to do it better. Circumvention is already hard enough, and by demanding that is be simultaneously decentralized, these critics are asking us not only to juggle, but to do so backwards on roller skates. But on the other hand, I can sympathize with their point of view. Despite the difficulty, we *should* aspire to better designs. I dislike giving connection metadata to Amazon and Microsoft as much as anyone. Unfortunately, encrypted SNI is likely to move us even farther from the decentralized end of the scale. It will be so effective, and so easy to use, that I predict there will be a convergence of systems using it. We see something like that effect today, where there is a perception that if you want to resist DoS, you have no choice but to be on one of the big CDNs. The outcome I fear for the web is something like we have today with SMTP, where the costs of setting up an independent server are so great as to make the choice effectively impossible. But I don't want to overblow my concern here. We should be thinking about ways to decentralize, but encrypted SNI is worth pursuing even if we can't think of any.
What are the risks to reaching a future of easy and effective circumvention using encrypted SNI? The worst case is if the proposal fails or is permanently stalled: we'll be stuck in a world that is pretty much like the world of today, except more hostile to domain fronting, waiting for something else to come along. As I understand it, draft-rescorla-tls-esni-00 is subject to change before standardization, and I suppose there's a chance it could morph into something so unwieldy or undeployable that it fails despite standardization. Most of the discussion that I've seen so far has been positive, but not all stakeholders at the IETF love the idea; in particular I get the impression that some people rely on plaintext SNI for monitoring or regulatory compliance, and encrypted SNI will make their lives more difficult. So we have to watch out for it being neutered in a way that enables censorship. Crucially, the value of encrypted SNI for circumvention depends on its adoption. If at first it's only circumventors and malware authors using encrypted SNI, then censors and security firewalls will start to block it, and then it's permanently skunked, no use to anybody. What we need is for at least one of the major browsers to implement encrypted SNI and (importantly) enable it by default. It's browsers that have to lead the way here, just as they effectively snuck TLS deployment past the censors' notice, until it was too late to do anything about it.
And what about traffic analysis; that is, classification based on the timing and size of packets in an SNI-encrypted TLS stream? My gut feeling is that it still won't be the first tool that censors reach for. I see pressure on third parties as a more likely threat. But it becomes more likely with each passing day, and anyway, my instinct could be wrong. So I think that research on traffic analysis obfuscation will become more and more relevant.
On Mon, 24 Sep 2018 11:57:48 +0000, David Fifield wrote:
I have to admit that I don't fully understand the apparent enthusiasm for encrypted SNI from groups that formerly were not excited about domain fronting.
It's simply wrong to use different names in SNI and the host header. :-)
customer's domains are potentially affected, rather than just one. It's a rational enough viewpoint (greater potential collateral damage ??? lower probability of blocking), but to my mind encrypted SNI doesn't fundamentally alter the nature of the game, it just raises the stakes.
But in a game-changingly massive way. Remember the github blocking?
When you block one domain that is on cloudflare, almost noboby will care. When you block all of cloudflare you will get an outcry of a lot of people, and probably worse for the censors, businesses.
- Andreas
On Mon, Sep 24, 2018, 12:46 PM Nathaniel Suchy me@lunorian.is wrote:
Hi everyone,
Cloudflare has added support to TLS 1.3 for encrypted server name indication (SNI). This mailing list post is a high level overview of how meek could take advantage of this in relation to Cloudflare who until just now wasn’t an option for domain fronting.
What this means: Effectively domain fronting works by sending a different SNI and host header. CDN providers like Cloudflare started double checking to make governments happy, scratch that line, I mean to protect their customers from fraud and abuse. They seem to of backtracked now. Encrypted SNI means that a firewall or coffee shop owner won’t be able to use SNI to see the real origin of TLS traffic.
Why this matters: With the right adjustments for TLS 1.3 and Encrypted SNI support, Cloudflare may be a viable option for Meek.
Risks:
- Firewall products could always use DPI and block TLS 1.3 altogether.
- Firewall products could block all requests with encrypted SNI.
Thoughts anyone?
The latter concern seems real enough for me that we should consider not front-running major adoption in browsers.
-tom
On Mon, Sep 24, 2018 at 01:46:10PM -0400, Nathaniel Suchy wrote:
What this means: Effectively domain fronting works by sending a different SNI and host header. CDN providers like Cloudflare started double checking to make governments happy, scratch that line, I mean to protect their customers from fraud and abuse. They seem to of backtracked now. Encrypted SNI means that a firewall or coffee shop owner won’t be able to use SNI to see the real origin of TLS traffic.
What we would need in order for meek to used encrypted SNI would be either: 1) support for encrypted SNI in Go's crypto/tls package; or 2) support for encrypted SNI in the Firefox that ships with Tor Browser, which meek-client could use through its TLS camouflage helper support.
IMO (2) is less desirable because I'd like to get rid of the TLS camouflage helper support and replace it with a Go-level TLS camouflage library: https://github.com/refraction-networking/utls. The TLS helper works, but its complexity is a pain to deal with and leads to problems like https://bugs.torproject.org/12774 https://bugs.torproject.org/25405.
Note, however, that a transport based on encrypted SNI doesn't have to be meek per se. I use "meek" to refer to a specific combination of domain fronting and a certain (fairly stupid and inefficient) HTTP/1.1 serialization of a bidirectional byte stream. It's conceptually straightforward to simply replace the "domain fronting" part with an "encrypted SNI" part. But it's possible to do better: if you're willing to abandon HTTP/1.1 compatibility and require HTTP/2, you can use the "server push" feature to implement a serialization that's much more efficient than the current one in meek. It may even be better to start over with a new codebase, it's not like meek's code is that large.
But tjr's point stands: I think it's best not to push anything like this out until after encrypted SNI has seen some level of adoption by browsers.
On Mon, 24 Sep 2018 20:23:58 +0000, David Fifield wrote: ...
"encrypted SNI" part. But it's possible to do better: if you're willing to abandon HTTP/1.1 compatibility and require HTTP/2, you can use the "server push" feature to implement a serialization that's much more efficient than the current one in meek.
How about websockets instead of trying to cram this into HTTP/2?
- Andreas
On Mon, Oct 01, 2018 at 07:55:31PM +0200, Andreas Krey wrote:
On Mon, 24 Sep 2018 20:23:58 +0000, David Fifield wrote: ...
"encrypted SNI" part. But it's possible to do better: if you're willing to abandon HTTP/1.1 compatibility and require HTTP/2, you can use the "server push" feature to implement a serialization that's much more efficient than the current one in meek.
How about websockets instead of trying to cram this into HTTP/2?
And for that matter, why not a plain old HTTP CONNECT proxy? That would be even more efficient. But we're limited to what the CDN supports. Most CDNs only support basic methods like GET and POST, not CONNECT or the special headers needed by WebSocket.
Cloudflare does support WebSocket, though: https://www.cloudflare.com/website-optimization/web-sockets/ https://blog.cloudflare.com/cloudflare-now-supports-websockets/ So this, combined with encrypted SNI, could be a viable technique when tunneling through Cloudflare--it just wouldn't be portable to other services. We even already have an existing WebSocket-based pluggable transport implementation--it would need changes to the client to support encrypted SNI. https://gitweb.torproject.org/pluggable-transports/websocket.git/
On Wed, Oct 03, 2018 at 07:01:21PM -0600, David Fifield wrote:
And for that matter, why not a plain old HTTP CONNECT proxy? That would be even more efficient.
I should add that--leaving out domain fronting/encrypted SNI--there's an implementation of exactly this, a pluggable transport built on an HTTP proxy, by Sergey Frolov et al. He has been trying to get some attention or buy-in to get it integrated into Tor Browser, but hasn't had much luck so far. In my opinion, it will make a great alternative to obfs4 and be effective in many situations.
There's a bit more to it than I've described above. It can work with any HTTP proxy (with HTTPS encryption to hide the destination from the censor, of course)--but they've also implemented a proxy plugin for the Caddy web server, which supports authentication. The authentication is to resist active probing like the GFW does: a genuine client who got the password through BridgeDB will be able to use the proxy, while a censor probing IP address will just get the web server's normal pages. Check the links for more info.
https://bugs.torproject.org/26923 https://github.com/sergeyfrolov/httpsproxy
On Wed, 03 Oct 2018 19:01:21 +0000, David Fifield wrote: ...
And for that matter, why not a plain old HTTP CONNECT proxy?
Because the typical load balancer/forwarder would have to decide whether to forward that CONNECT or do it itself, and some other. CONNECT with a Host: header - I'm not sure there is such a thing.
That would be even more efficient. But we're limited to what the CDN supports. Most CDNs only support basic methods like GET and POST, not CONNECT or the special headers needed by WebSocket.
Yes. No. A quick search indicates that aws and azure are already supporting it, although I'm unable to interpret whether that is actually the respective product you are/were using.
But websockets are a relevant thing unlike CONNECT, so I do expect all major players to implement that (and the components I know of (haproxy, nginx, apache, golang) are there already).
- Andreas
On Thu, Oct 04, 2018 at 09:37:18AM +0200, Andreas Krey wrote:
A quick search indicates that aws and azure are already supporting it, although I'm unable to interpret whether that is actually the respective product you are/were using.
That's exactly it. Of course you can spin up a random EC2 machine or Azure VM and connect to it with WebSocket. But a random AWS or Azure IP address doesn't have the properties we want--it is easily blockable. Why? Because there are no other valuable services running on the same IP address, once it is discovered, the censor can block it with virtually zero cost. (Not that it's an unworkable model--that's basically how obfs4 works--it means you have to have to be careful about distributing bridges, and keep replenishing the pool as they are inevitably blocked.) Encrypted SNI does not help you here, not when there is only one service running on the IP address.
We specifically use the CDN service, because the high degree of co-hosting behind a CDN edge server makes it more expensive to block. In this case, encrypted SNI (or formerly domain fronting) really does help, because there are many and valuable sites hosted there, and connections to different sites look the same from the censor's point of view. We're not connecting via a cloud service just for fun, but because we depend critically on that co-hosting to make blocking difficult. It's possible that services other than the CDN services also have that property--it would require some investigation to uncover--but then you would have to ensure that the other service, too, supports WebSocket.
I'm not saying a WebSocket-in-TLS transport is a bad idea. It's just that there's more to it than the protocol you use. You also have to go through a proxy server that's expensive to block. As a proxy server, most CDNs will support an HTTP/2 tunnel, but not WebSocket. And if you don't have a CDN in the middle, you may as well use a plain CONNECT proxy rather than WebSocket.
Related reading: https://tools.ietf.org/html/draft-ietf-tls-sni-encryption-03#section-2.2 Many deployments still allocate different IP addresses to different services, so that different services can be identified by their IP addresses. However, content distribution networks (CDN) commonly serve a large number of services through a small number of addresses. https://tools.ietf.org/html/draft-ietf-tls-esni-01#section-3.1 In principle, the provider might not be the origin for any domains, but as a practical matter, it is probably the origin for a large set of innocuous domains, but is also providing protection for some private domains.
On Mon, Sep 24, 2018 at 08:23:58PM -0600, David Fifield wrote:
What we would need in order for meek to used encrypted SNI would be either:
- support for encrypted SNI in Go's crypto/tls package; or
- support for encrypted SNI in the Firefox that ships with Tor Browser, which meek-client could use through its TLS camouflage helper support.
IMO (2) is less desirable because I'd like to get rid of the TLS camouflage helper support and replace it with a Go-level TLS camouflage library: https://github.com/refraction-networking/utls. The TLS helper works, but its complexity is a pain to deal with and leads to problems like https://bugs.torproject.org/12774 https://bugs.torproject.org/25405.
I wrote an untested overview of how to adapt meek to use ESNI, using an external copy of Firefox Nightly rather than Tor Browser's built-in copy of Firefox. Testing this out to see if it works would be a good task for someone who wants to get involved with pluggable transports.
Use ESNI via Firefox HTTPS helper https://bugs.torproject.org/28168
1. Download Tor Browser and Firefox Nightly. 2. Go to about:config in Firefox nightly and set network.trr.mode=3 network.trr.uri=https://1.1.1.1/dns-query network.security.esni.enabled=true 3. Copy the meek-http-helper@bamsoftware.com.xpi from Tor Browser to Firefox Nightly. 4. Hack meek-client-torbrowser/{mac,linux,windows}.go to point firefoxPath at the copy of Firefox Nightly and disable the custom profile. (Additional hacks to remove hardcoded Tor Browser assumptions may be required.) 5. Set up a Cloudflare instance pointing to https://meek.bamsoftware.com/, call it https://meek.example.com/. 6. Set up a custom bridge in Tor Browser, using url= without front= (because we're no longer domain fronting). bridge meek 0.0.2.0:3 url=https://meek.example.com/
The only slightly weird part I foresee is hacking meek-client-torbrowser; it has some internal hardcoded paths and profiles that are specific to the Tor Browser directory layout, and you'll have to point those to an external Firefox Nightly. Of course, once ESNI support makes its way into Tor Browser itself, there won't be a need for another external copy of Firefox.