Building better pluggable transports - GSoC 2013 project
Hello everyone! I am a Tor GSoC student who will be working on the pluggable transports this summer. My mentor is Steven and my co-mentor is George Kadianakis. It is great to be part of the Tor community! Steven already kicked off the discussion about how to build better transports. The original project proposal[1] discussed the possibility of sending data over UDP with extra efforts to guarantee reliable in-order delivery. However, as George mentioned recently[3], ScrambleSuite[2] may already solve the issue of scanning resistance. Given that ScrambleSuite is being deployed, improving protocol obfuscation will be my main focus. HTTP impersonation is really useful, since there are numerous HTTP proxy outside the censored region, while the number of bridges is quite limited. What I'm gonna be doing during the summer is implementing a good enough HTTP impersonation based on pluggable transports specification. There are still many open questions indeed. Discussions are more than welcome! I'll be keeping my GSoC log here: http://changlan.github.io/obfsproxy I'm also available #tor-dev@oftc with the handle "clan", feel free to ping me there! [1]: http://changlan.info/gsoc.html [2]: http://www.cs.kau.se/philwint/scramblesuit/ [3]: https://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/changlan... Cheers, Chang
Chang Lan <changlan9@gmail.com> writes:
Hello everyone!
Hi there,
I am a Tor GSoC student who will be working on the pluggable transports this summer. My mentor is Steven and my co-mentor is George Kadianakis. It is great to be part of the Tor community!
Steven already kicked off the discussion about how to build better transports. The original project proposal[1] discussed the possibility of sending data over UDP with extra efforts to guarantee reliable in-order delivery. However, as George mentioned recently[3], ScrambleSuite[2] may already solve the issue of scanning resistance.
Given that ScrambleSuite is being deployed, improving protocol obfuscation will be my main focus. HTTP impersonation is really useful, since there are numerous HTTP proxy outside the censored region, while the number of bridges is quite limited. What I'm gonna be doing during the summer is implementing a good enough HTTP impersonation based on pluggable transports specification. There are still many open questions indeed. Discussions are more than welcome!
Which approach are you thinking of taking for writing your HTTP transport? Which HTTP covert channel are you planning on using? Have you seen https//trac.torproject.org/projects/tor/ticket/8676 for a different approach of writing an HTTP transport? In the google-melange page I see that your timetable doesn't include any specification. Can we hope to have a specification document before you start implementing the pluggable transport? Thanks!
On Wed, May 29, 2013 at 01:22:31PM +0800, Chang Lan wrote:
I am a Tor GSoC student who will be working on the pluggable transports this summer. My mentor is Steven and my co-mentor is George Kadianakis. It is great to be part of the Tor community!
Steven already kicked off the discussion about how to build better transports. The original project proposal[1] discussed the possibility of sending data over UDP with extra efforts to guarantee reliable in-order delivery. However, as George mentioned recently[3], ScrambleSuite[2] may already solve the issue of scanning resistance.
Given that ScrambleSuite is being deployed, improving protocol obfuscation will be my main focus. HTTP impersonation is really useful, since there are numerous HTTP proxy outside the censored region, while the number of bridges is quite limited. What I'm gonna be doing during the summer is implementing a good enough HTTP impersonation based on pluggable transports specification. There are still many open questions indeed. Discussions are more than welcome!
Hi Chang! I want to make sure you know about a transport based on WebSocket that we hope to have deployed in the near future. A WebSocket bridge is already running; it's what browser-based flash proxies talk to. (Flash proxy is a circumvention system for IP obfuscation, not protocol obfuscation.) https://crypto.stanford.edu/flashproxy/ However, you can connect directly to a WebSocket bridge as a client, without going through a flash proxy. https://gitweb.torproject.org/flashproxy.git/blob/e4f3ced2:/doc/websocket-tr... https://gitweb.torproject.org/flashproxy.git/tree/e4f3ced2:/websocket-transp... Our conjecture is that WebSocket looks enough like HTTP to evade protocol filters, at least for a while. (WebSocket starts with an HTTP header.) David Fifield
Hi Chang, On 29 May 2013, at 06:22, Chang Lan <changlan9@gmail.com> wrote:
Given that ScrambleSuite is being deployed, improving protocol obfuscation will be my main focus. HTTP impersonation is really useful, since there are numerous HTTP proxy outside the censored region, while the number of bridges is quite limited. What I'm gonna be doing during the summer is implementing a good enough HTTP impersonation based on pluggable transports specification. There are still many open questions indeed. Discussions are more than welcome!
There certainly are quite a few open questions, so it would be good to start planning early. Implementing HTTP is a deceptively difficult project. I'd suggest starting by reading the HTTP specification in detail, particularly the parts that deal with caching: http://tools.ietf.org/html/rfc2616 For comparison HTTP/1.0 is also worth looking at: http://tools.ietf.org/html/rfc1945 Some issues that you will need to deal with are: - Individual HTTP requests may be re-ordered if they are over different TCP connections - Responses may be truncated without an error being reported to higher layers (which is why HTTP includes length fields as an option). - HTTP doesn't give the same congestion avoidance as TCP - Proxies can both cache and modify data they transmit. - Proxies deviate from what is permitted by the specification - (and others) When dealing with these, you will need to ensure you don't introduce any new ways for a censor to efficiently and reliably distinguish your protocol from HTTP. I think it would also be a good idea to implement scanning resistance. Since it will be over TCP, you can't hide that something is listening, but you can ensure that if the initial request does not demonstrate knowledge of a valid secret, the response does not disclose that it is a Tor bridge. As you start implementing, you should have some way of testing. Initially this can be a direct connection from your pluggable transport client to pluggable transport server. You can set up an OP and bridge on the same machine (set your bridge not to advertise itself), and get your OP to talk to your bridge via your pluggable transport. However, you shouldn't keep to this setup for very long, as it won't test how your pluggable transport works with a proxy. So you should put a caching proxy (e.g. Squid) between your pluggable transport server and client, and make sure they keep working. You can try configuring Squid in ways to stress your pluggable transport, and also replace Squid with a proxy server you create (e.g. based on one of the many Python HTTP proxies http://proxies.xhaus.com/python/). This proxy server could behave pathologically, and test the corner cases of your pluggable transport. When working on your experiments, automate the set up, running of the test, and processing of the results. This is not just to make your life easier but it means that your experiments can be repeatable. The scripts and configuration files should be checked into version control. Your goal should be that someone can check out your code, install a few standard packages via apt-get or yum, run a single command, and get the same results. There are tools to help do this (e.g. http://software-carpentry.org/4_0/data/mgmt.html and http://software-carpentry.org/4_0/data/bein.html) but just using make and shell scripts might be fine. There's a lot to think about here, so we don't need answers to everything now, but if you have any questions or comments do let me know. Best wishes, Steven
On 11 Jun 2013, at 12:49, Steven Murdoch <steven.murdoch@cl.cam.ac.uk> wrote:
There certainly are quite a few open questions, so it would be good to start planning early. Implementing HTTP is a deceptively difficult project.
I've started a design document https://github.com/sjmurdoch/http-transport/blob/master/design.md which is very much a work-in-progress but I'm interested in comments. Best wishes, Steven
On Tue, Jun 11, 2013 at 05:46:49PM +0100, Steven Murdoch wrote:
On 11 Jun 2013, at 12:49, Steven Murdoch <[1]steven.murdoch@cl.cam.ac.uk> wrote:
There certainly are quite a few open questions, so it would be good to start planning early. Implementing HTTP is a deceptively difficult project.
I've started a design document https://github.com/sjmurdoch/http-transport/ blob/master/design.md which is very much a work-in-progress but I'm interested in comments.
Here are some ideas on a few things that I've been thinking about recently, mostly taken from https://www.bamsoftware.com/papers/oss.pdf. That's an HTTP-based transport, though one with different goals: It's meant to evade IP-based blocking and not DPI. (The paper does have a section at the end about mitigations against DPI.)
Bi-directional data Tor requires that communication exchanges be initiated either by the bridge client or bridge server. In contrast HTTP clients initiate all communications. There are a few ways to avoid this problem: * The client periodically polls the server to check if any data is available * The client keeps a long-running Comet TCP connection, on which the server can send responses * The client and server both act as HTTP clients and HTTP servers, so can each send data when they wish
Making the client an HTTP server has the same NAT problems that flash proxy has. The OSS model has the worst of both worlds: the client has to be an HTTP server and also has to poll. But we implemented polling and it was usable.
Proxy busting Proxies will, under certain conditions, not send a request they receive to the destination server, but instead serve whatever the proxy thinks is the correct response. The HTTP specification dictates a proxy's behaviour but some proxy servers may deviate from the requirements. The pluggable transport will therefore need to either prevent the proxy from caching responses or detect cached data and trigger a re-transmission. It may be unusual behaviour for a HTTP client to always send unique requests, so it perhaps should occasionally send dummy requests which are the same as before and so would be cached.
To inhibit caching we added a random number to every request. However that's a good point about not having all requests be unique.
Client to server (requests) Cookies Short and usually do not change, so possibly not a good choice HTTP POST file uploads Quite unusual, but permit large uploads
Another avenue is URLs--they are sometime kilobytes long (and clients and servers support much longer than that), and often contain opaque binary data. David
I've been thinking about writing a lessons-learned document about StegoTorus; I'll bump that up a little on the todo queue. For right now I want to mention that any greenfields design should take a hard look at MinimaLT <http://cr.yp.to/tcpip/minimalt-20130522.pdf> as its cryptographic layer. It looks like it addresses most if not all of the problems I was trying to tackle with ST's crypto layer, only (unlike ST's crypto layer) it's actually *finished*. On Tue, Jun 11, 2013 at 12:46 PM, Steven Murdoch <Steven.Murdoch@cl.cam.ac.uk> wrote:
On 11 Jun 2013, at 12:49, Steven Murdoch <steven.murdoch@cl.cam.ac.uk> wrote:
There certainly are quite a few open questions, so it would be good to start planning early. Implementing HTTP is a deceptively difficult project.
I've started a design document https://github.com/sjmurdoch/http-transport/blob/master/design.md which is very much a work-in-progress but I'm interested in comments.
Best wishes, Steven
_______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
participants (5)
-
Chang Lan -
David Fifield -
George Kadianakis -
Steven Murdoch -
Zack Weinberg