- tor-dev - lists.torproject.org

Def Con Kaminsky talk (censorship detection)
by David Fifield 30 Jul '12

30 Jul '12

I saw an interesting talk by Dan Kaminsky at Def Con that touched on some ideas for censorship detection. He mentioned OONI-probe and talked about his project CensorSweeper. It tests blockedness of web sites by making cross-domain requests for favicon.ico and displaying them in a minesweeper-like grid. http://www.censorsweeper.com/ https://www.hackerleague.org/hackathons/wsj-data-transparency-code-a-thon/h… He also mentioned something, which unfortunately I didn't follow very closely, about using Flash sockets to spoof HTTP and HTTPS headers. I think the gag here was sending these spoofed connections to a server you control (so you can answer the crossdomain policy requests without which Flash Player will refuse to connect), but you give it a Host header of a censored site or something like that. http://miriku.com/wp/2012/07/decon-day-3/comment-page-1/#comment-1416 Unfortunately I don't have the conference DVD which presumably contains the slides he used, but videos usually show up online after some number of months. David Fifield

1 0

Brainstorming about steganographic transports
by David Fifield 29 Jul '12

29 Jul '12

This is a summary of some discussion among developers of pluggable transports about steganographic channels and deriving them from protocol grammars. Two things prompted the discussion: * A program called appid, http://code.google.com/p/appid/ , which compiles simple protocol specifications into big regular expressions and uses them to classify network streams. Supported protocols are https://code.google.com/p/appid/source/browse/trunk/apps ; a sample is https://code.google.com/p/appid/source/browse/trunk/apps/afs . Roger suggested to make a program that takes one of these specifications and embeds data within a stream that appid will classify as the chosen protocol. * This paper: "Provably Secure Steganography" http://www-users.cs.umn.edu/~hopper/tc-stego.pdf The authors define a "channel" as a probability distribution on sequences. You can think of a channel as something that answers "given this prior history of packets, what are the possible next packets and their probabilities?" or even as an oracle that merely allows sampling this distribution. They show a method of steganographically encoding a bit within a channel such that that a passive adversary cannot distinguish it from "normal" traffic (Sec. 4.1 and Fig. 1). For the topics we were talking about, the most important sections to read are 2 and 4.1. You can embed data in a string that will match a given regular expression by constructing a DFA (http://swtch.com/~rsc/regexp/regexp1.html) and walking it state by state, choosing state transition edges so as to encode something. For example, if there are two outgoing edges from a state, which one you choose can encode a bit. Many of the appid signatures contain any* blocks that can straightforwardly encode bytes. An example of this idea is the regular expression /^(ab*[cd])+/. The b* lets us encode one bit (say "" = 0 and "b" = 1), and the choice of c or d allows another bit ("c" = 0 and "d = 1). We could send the message 00110001 as "acabdacad". This idea can really be thought of an application of the paper. A regular expression defines a probabilistic channel if we assign probabilties to edge transitions, which we may do uniformly. So, for example, with the regular expression given above, the first byte is 'a' with probability 1. The second byte is 'b' with probability 0.5, 'c' with probability 0.25, and 'c' with probability 0.25. But we see how a channel is sensitive to history: if the string so far is "ab", then the next byte may be 'c' or 'd'; but if the string so far is "ac", then the next byte must be 'a' (or EOF). We can use appid-like signatures to make steganographic channels, if we assume that the signatures are a realistic reflection of actual use of the protocols. But: this relies critically on the accuracy of the model. (Specifically, does it match the censor's model? If he uses simple regular expressions for blocking, then we win; if not, then we probably lose.) Robert Ransom gave the example of a model of HTTP using just the syntax; i.e. using RFC 2616 as the definition of a channel. Such a thing would not only not look like real-world HTTP, but would be broken easily by proxies that rearrange headers, for example. rransom also suggested a way to use arithmetic encoding to obfuscate a stream using a probabilistic model of a protocol; I'll quote him in part: * send it through an arithmetic-code encoder, using a model which contains a (low-probability) 'pad' symbol; * encrypt it with a stream cipher; * send it through an arithmetic-code decoder, using a model which matches the 'cover protocol' (the protocol you are trying to mimic). David (yours truly) wants to write or help write a simple pluggable transport derived from regular-expression signatures, even with the limitations shown above. Client and relay would need matching signature models. For the same of simplicity, it will only seek to match the given signature, and not be indistinguishable in the strong sense mentioned above. It won't do symmetric encryption of the underlying TLS (or if it does, will use a fixed key). It won't use the constructions from the Provably Secure Steganography paper, rather it will just encode its stream directly in DFA edge transitions. I think it will be interesting to see 1) how far a simple system can get us, and 2) what additional changes we would have to make to be provably secure against censors using more sophisticated computational models than regex. David Fifield

4 3

Re: [tor-dev] [tor-assistants] Protocol grammars as probabilistic channels
by Robert Ransom 28 Jul '12

28 Jul '12

On 7/28/12, Zack Weinberg <zack.weinberg(a)sv.cmu.edu> wrote: > [I'm moving this from a giant cc: list to tor-dev, I hope that's okay. > I *think* everyone involved is already on that list.] > > [I apologize for not picking up this ball for so long, I have been ill > and mostly without the brain.] > > On Sun, Jul 22, 2012 at 2:26 AM, Robert Ransom <rransom.8774(a)gmail.com> > wrote: >> >> Specifically, to obfuscate a ‘payload protocol’ data stream: >> >> * send it through an arithmetic-code encoder, using a model which >> contains a (low-probability) ‘pad’ symbol; >> * encrypt it with a stream cipher; >> * send it through an arithmetic-code decoder, using a model which >> matches the ‘cover protocol’ (the protocol you are trying to mimic). > > I like this way of modeling the problem, but I see some practical > problems with it. The payload protocol in this case is Tor's link > protocol, which is already encrypted, so your first step would appear > not to do anything (well, except to TLS's cleartext record headers, > but I don't think that will gain you much). The first step (the arithmetic-code encoder) allows the obfuscator to introduce padding symbols into a bytestream without either understanding or breaking the protocol it is obfuscating. This is the only reason for the obfuscator to use an arithmetic coder on the payload protocol. If the payload protocol is datagram-oriented and can tolerate junk appended to the end of a datagram, or if the software implementing a stream-oriented payload protocol can generate padding at the end of a message when the obfuscator requests it, the payload-side arithmetic coder should be omitted. (Note that Tor itself doesn't satisfy either of these requirements; but Tor-over-TCP-over-IP is a datagram-oriented protocol with a length field in each datagram.) > The stream cipher > provides no integrity protection for the message, which maybe we can > get away with since the higher level has it, and more importantly, no > sequencing. The stream cipher in the middle prevents the payload-side arithmetic-code encoder (or the underlying protocol) from making the stream fed to the cover-side decoder (computationally) distinguishable from a random bitstream. A datagram-oriented payload protocol will need a different scrambling layer as well. The easiest solution I can think of is to encrypt the message header with a large-block block cipher, and assume that clients usually (but not always) keep the same IP address during a protocol session in order to reduce the number of keys that the server needs to try for each message. > Several cover protocols of interest (most importantly > HTTP) break a session up into many short-lived TCP connections. That > means you have to allow for the possibility that cover-protocol > messages will arrive at the decoder out of order. You also need the > ability to send control messages at a level below the payload > protocol, for key agreement, reassociation of new connections with > ongoing sessions, and possibly other things we haven't thought of yet. TCP transports a stream protocol over an unreliable unordered datagram protocol. Surely it can't be too hard to add a security (i.e. encryption and authentication) layer to an unreliable unordered datagram protocol. > The "chopper" component of StegoTorus tackles these problems. Have > you seen the draft paper? Probably not. > It has its own problems -- the one I'm > wrestling with now is, the _amount of payload data_ sent in any given > message gets frozen the first time it is transmitted; if you need to > _re_transmit, you'd better be able to send at least that much data > right now! (Padding can of course be varied.) TCP can increase or decrease the amount of data sent in a retransmitted datagram. (I suspect that real TCP implementations actually do retransmit datagrams with less data, during path MTU discovery.) > I would be curious to know if you think your arithmetic-decoding > approach to cover generation could reasonably be made to work within > StegoTorus' framework. The arithmetic-code decoder on the cover-protocol side may (should) have been proposed before. The other two pieces were my idea, but they were for stream-oriented obfuscation, and you pointed out above that they suck for a datagram-oriented obfuscator and need to be replaced. >>> George turned me on to this paper, >>> >>> "Provably Secure Steganography" >>> http://www-users.cs.umn.edu/~hopper/tc-stego.pdf > > I have to say that I am very skeptical about the practical value of > this paper, because it requires one to characterize the probability > distribution of all possible messages in the cover protocol, which is > not practically possible for any nontrivial protocol. I could see an > approximation being good enough for our purposes, though. (Note that Zack is quoting someone other than me there.) >> Don't hide information directly in the syntax -- it's easy enough for >> Them to rearrange protocol messages in ways that will not interfere >> with standard-conformant clients and servers, but will disrupt crappy >> protocol obfuscators that don't implement the cover protocol >> correctly. Use a semantic model of the cover protocol instead. > > vmon and I have been wrestling with this one a bit in the context of > HTTP and we've pretty much come to the conclusion that we need an > actual implementation of HTTP, into which we plug data. Fortunately > there is no shortage of HTTP implementations. > >> ‘Blocking resistance’ is *very* different from ‘detection resistance’ >> (i.e. ‘indistinguishability from normal traffic’ for some value of >> ‘normal traffic’). >> >> * ‘Blocking resistance’ does not require that a cover protocol be >> ‘indistinguishable from X’ by the adversary, only that the adversary >> be unable to block it. For example, an adversary which can only >> ‘block’ communications by disrupting a connection using extra (forged) >> packets will not be able to block a UDP-based protocol. > > The adversary we're concerned with has control of all the border > routers between the client and the server, so they can do things like > blackhole all packets with a particular source or destination. I > would argue that detection resistance is in fact required for blocking > resistance against this adversary. But only ‘detection resistance’ against any box that They can use to control a blocking system, and that can detect the protocol quickly enough to usefully reduce the amount of data that each user can exchange with each bridge. >> * A protocol which is ‘indistinguishable from’ some protocol which the >> adversary wants to not block may not provide ‘blocking resistance’. >> For example, a crappy obfuscator which uses a syntactic model of HTTP >> instead of a semantic model can be blocked by just about any >> off-the-shelf HTTP proxy. > > That effect also provides a distinguisher from real HTTP, doesn't it? Only if They can tell that They blocked an obfuscator's traffic. They may not be able to detect that. Robert Ransom

1 0

Re: [tor-dev] [tor-assistants] Protocol grammars as probabilistic channels
by Zack Weinberg 28 Jul '12

28 Jul '12

[I'm moving this from a giant cc: list to tor-dev, I hope that's okay. I *think* everyone involved is already on that list.] [I apologize for not picking up this ball for so long, I have been ill and mostly without the brain.] On Sun, Jul 22, 2012 at 2:26 AM, Robert Ransom <rransom.8774(a)gmail.com> wrote: > > Specifically, to obfuscate a ‘payload protocol’ data stream: > > * send it through an arithmetic-code encoder, using a model which > contains a (low-probability) ‘pad’ symbol; > * encrypt it with a stream cipher; > * send it through an arithmetic-code decoder, using a model which > matches the ‘cover protocol’ (the protocol you are trying to mimic). I like this way of modeling the problem, but I see some practical problems with it. The payload protocol in this case is Tor's link protocol, which is already encrypted, so your first step would appear not to do anything (well, except to TLS's cleartext record headers, but I don't think that will gain you much). The stream cipher provides no integrity protection for the message, which maybe we can get away with since the higher level has it, and more importantly, no sequencing. Several cover protocols of interest (most importantly HTTP) break a session up into many short-lived TCP connections. That means you have to allow for the possibility that cover-protocol messages will arrive at the decoder out of order. You also need the ability to send control messages at a level below the payload protocol, for key agreement, reassociation of new connections with ongoing sessions, and possibly other things we haven't thought of yet. The "chopper" component of StegoTorus tackles these problems. Have you seen the draft paper? It has its own problems -- the one I'm wrestling with now is, the _amount of payload data_ sent in any given message gets frozen the first time it is transmitted; if you need to _re_transmit, you'd better be able to send at least that much data right now! (Padding can of course be varied.) I would be curious to know if you think your arithmetic-decoding approach to cover generation could reasonably be made to work within StegoTorus' framework. >> George turned me on to this paper, >> >> "Provably Secure Steganography" >> http://www-users.cs.umn.edu/~hopper/tc-stego.pdf I have to say that I am very skeptical about the practical value of this paper, because it requires one to characterize the probability distribution of all possible messages in the cover protocol, which is not practically possible for any nontrivial protocol. I could see an approximation being good enough for our purposes, though. > Don't hide information directly in the syntax -- it's easy enough for > Them to rearrange protocol messages in ways that will not interfere > with standard-conformant clients and servers, but will disrupt crappy > protocol obfuscators that don't implement the cover protocol > correctly. Use a semantic model of the cover protocol instead. vmon and I have been wrestling with this one a bit in the context of HTTP and we've pretty much come to the conclusion that we need an actual implementation of HTTP, into which we plug data. Fortunately there is no shortage of HTTP implementations. > ‘Blocking resistance’ is *very* different from ‘detection resistance’ > (i.e. ‘indistinguishability from normal traffic’ for some value of > ‘normal traffic’). > > * ‘Blocking resistance’ does not require that a cover protocol be > ‘indistinguishable from X’ by the adversary, only that the adversary > be unable to block it. For example, an adversary which can only > ‘block’ communications by disrupting a connection using extra (forged) > packets will not be able to block a UDP-based protocol. The adversary we're concerned with has control of all the border routers between the client and the server, so they can do things like blackhole all packets with a particular source or destination. I would argue that detection resistance is in fact required for blocking resistance against this adversary. > * A protocol which is ‘indistinguishable from’ some protocol which the > adversary wants to not block may not provide ‘blocking resistance’. > For example, a crappy obfuscator which uses a syntactic model of HTTP > instead of a semantic model can be blocked by just about any > off-the-shelf HTTP proxy. That effect also provides a distinguisher from real HTTP, doesn't it? zw

1 0

Re: [tor-dev] Pyonionoo Descriptors
by Karsten Loesing 28 Jul '12

28 Jul '12

Hi Megan, hi Erik, cc'ing Sathya and tor-dev, hoping you don't mind. On 7/27/12 5:21 PM, Megan Chang wrote: > We are currently almost done with the first draft of pyonionoo. Awesome! :) > We were > wondering what some of the entries might look like so we would be able to > see what information was being parsed to fill in some of the fields such as > the bandwidth handler's "write_history" and "read_history". We noticed that > the router descriptor format includes "write_history" and "read_history" > according to torspec but again, we aren't too sure on what these entries > look like. The bandwidth documents are tricky, because one cannot just look up these values from Tor descriptors. They're based on histories in extra-info descriptors, but they aggregate these values on different levels of detail, and they're specifically designed for graphing. I don't know about your schedule, but if you're short on time, I'd say let's discuss summary and details documents first and ignore bandwidth documents for the moment. (Of course, if you have plenty of time left, happy to discuss bandwidth documents more!) Is your code online somewhere? I'd like to take a look and give you some feedback about Tor descriptor specific things (not so much about Python/Twisted (?) things). > Also, we don't know how to approach the fields in the details > handler that require GeoIP information. Any tips on this? Onionoo uses Maxmind's Java API, their GeoIP database, and their AS database to resolve IP addresses to country/AS information. There's a Python API, too, which I think Sathya used for his network entropy code: https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-6232/pygeoip If you're unsure how to use it, best ask Sathya for more details. Best, Karsten

2 2

script for get bridges
by ??Phoenixing 27 Jul '12

27 Jul '12

hi Firstly, install wget, privoxy, then schedule a task three minutes one time just like: someone>$ wget https://bridges.torproject.org. Then you can get a lot of .htmlxxx files which contain the bridges from the website. Lastly, write a bash command or VBscript under windows, you can grap the xxx.xxx.xxx.xxx:xxxx from the .htmlxxx files, and insert into the torrc file. It 's done! I hope that I can do this again! best regards

1 0

Why change the bridges.torproject.org website to avoid to get bridges by script?
by ??Phoenixing 26 Jul '12

26 Jul '12

Hi I live in a country which it block the tor net strictly! Only I can do is that I get the bridges from the website. I just want to get the bridges automaticly by writing a script. But now I must get the bridges hand by hand and one by one. It's more uncomfortable! Could you change it back? best regards!

3 2

Faravahar back online
by SiNA Rabbani 25 Jul '12

25 Jul '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear Tor, Faravahar is back online. Please see https://trac.torproject.org/projects/tor/ticket/5749 and let me know what my next steps should be. All the best, SiNA - -- ?If I had eight hours to chop down a tree, I?d spend the first six of them sharpening my axe.? -Abraham Lincoln -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJQD5h1AAoJEJPBwXYLR9VtmsEP/2TfkLYURwyuxPWl/uDdZb7A C3iVLQBgPU1lpKraAPC8/pzNsfm8JncDoPKMZbvzRmX9ltpZz6brUaxOP/bTLy9f diUYtSIW3l8FBxtsU52QO0mvZIlDKlK7+rPguVmMDxSmu1IhQFSNE8wlMEVELUjZ 0yJ5WX9kKZ2Wx0n3g1tIo8+Zst3Iu/8vtw00hCNKX1Y7KC297c8Ueo2kuaqxMN8a S4meVbthImdgGonyCww1BfDJG1cSK5w3nia7mDqI8Ne9O+6EBYN9o4Vm4S94S13C h4Cd3eQ9K06+u7Cs2mfr4n54X0J+fbh0dkCBnKVtS4HnGrGyTwtuq0GxRZ9ZLb2e 50MWrxTrB2c6xAJDZZ7J1CVl8jQH8+qZrr5VfFxwo486Jy1CiUh4cIlBNWyc7k7e xRh9HOW2kHhiPEPTSOheDCN6JNMnaIGnoKXmuVAV5z21NMIRiQWIad1LVroFFS2i 2aI81KosqspvY5vqcgVZFI8WTWG/1H0xNMSFHmqLaW7R1yz6tzz7yo6oICkox48V UQMkQsDDE+vAqB93EFoD1Fg+0PybT64wa7Q6iMaPKc4fhnSyYRj2b+5ztiA6FqZ/ Spg9OnksrVLcq5Oa7KHDWEOXUQldXbX3+DATugQxWSKPhpb1LYcjJ/avaQmyFKha kvQByRGllywwMXF5fg22 =MJ4O -----END PGP SIGNATURE-----

1 0

[GSoC] Stem - Status update
by Ravi Chandra Padmala 25 Jul '12

25 Jul '12

Hello The network status descriptor parsing has taken a lot longer than expected. I should be putting it up for review soon. Followed by the microdescriptor parsers. Given the current status of the project, I have prepared a revised schedule for the remaining GSoC period. The final evaluation is about a month away. I have spoken to Damian and we decided to reschedule the Arm port deliverable to after the GSoC timeline (most likely the week immediately following the end of the GSoC coding period). July 25 - July 28 Descriptor parsing July 29 - August 4 | Event handling methods for the controller class. August 5 - August 11 | I will be working on them as alloted to week 4, 5 and August 11 - August 20 | 6 from my original proposal[1]. 1. https://www.torproject.org/about/gsocProposal/gsoc12-proposal-stemImproveme… -- neena

1 0

ApachePayloadServer pushed
by vmonmoonshine＠gmail.com 24 Jul '12

24 Jul '12

Disc.:In the spirit of our discussion in Italy about unifying mailing list and that we should submit our technical discussions on tor-dev instead of tor-assistants, I'm moving my conversations with Zack here. If there's a strong opposition I'll go back to the old model (but you should have opposed it in Italy). Hey Zack, I pushed the ApachePayloadServer on your git on f--payload-servers branch. It compiles but I have to test it yet. However, it would be much easier to be tested when I change the current payload-based-on-trace to be inherited from the same parent. so I'm postponing testing for few days after I write that part (so you don't need to look at it till then). On another note, your pgen_pcap.cc and pgen_fake.cc don't compile on g++ 4.7.0 on linux, because it doesn't like the fact that you don't care about the return value of fwrite. That's it for now. Cheers, vmon I

2 1