- tor-dev - lists.torproject.org

Re: [tor-dev] Def Con Kaminsky talk (censorship detection)
by David Fifield 31 Jul '12

31 Jul '12

Dan replies: On Mon, Jul 30, 2012 at 11:33:09AM -0700, Dan Kaminsky wrote: > Basically, if you spoof HTTP or HTTPS headers from a Flash socket to your own > IP, with someone else's Host/SNI, a transparent proxy is going to send its > interposing content to the Flash SWF and not to the browser. It's a really > deployable way to see nasty stuff. > > One warning is that if hijacking is DNS based, and not transparent proxy based, > you don't see anything with this stunt (though favicon.ico detection still > works). > > On Mon, Jul 30, 2012 at 10:57 AM, David Fifield <david(a)bamsoftware.com> wrote: > > I saw an interesting talk by Dan Kaminsky at Def Con that touched on > some ideas for censorship detection. He mentioned OONI-probe and talked > about his project CensorSweeper. It tests blockedness of web sites by > making cross-domain requests for favicon.ico and displaying them in a > minesweeper-like grid. > > http://www.censorsweeper.com/ > https://www.hackerleague.org/hackathons/wsj-data-transparency-code-a-thon/ > hacks/censorsweeper > > He also mentioned something, which unfortunately I didn't follow very > closely, about using Flash sockets to spoof HTTP and HTTPS headers. I > think the gag here was sending these spoofed connections to a server you > control (so you can answer the crossdomain policy requests without which > Flash Player will refuse to connect), but you give it a Host header of a > censored site or something like that. > > http://miriku.com/wp/2012/07/decon-day-3/comment-page-1/#comment-1416 > > Unfortunately I don't have the conference DVD which presumably contains > the slides he used, but videos usually show up online after some number > of months. > > David Fifield > >

1 0

bridge's lifetime
by ??Phoenixing 31 Jul '12

31 Jul '12

hi </:includetail> I have get more than 1000 bridges and I totally inserted into the torrc file. So I can not notice which bridge is no longer work. The bridges can not be blocked totally. Some bridges seem that no longer work, but after sometime (three days or more) they work again.:-)</:includetail> </:includetail> best regards</:includetail> </:includetail> </:includetail>

2 1

introduction and progress toward an interactive version of a Tor metrics website
by Walter Kim 31 Jul '12

31 Jul '12

Hi Everyone, I'm a mathematician/coder/visualization person living in San Francisco. I've been in contact with Karsten Loesing about making more interactive graphs for the tor metrics website. I've made some progress which can be viewed here: http://tigerpa.ws/tor_metrics/ You can select a region in the bottom graph to set the time domain of the top graph. You can click on the buttons on the right to add lines to the graph. It's still a work in progress, so there are still bugs and I've open to suggestions on making it better. Currently it only shows graphs for network/relays data. I'm planning on adding the ability to view graphs for all the other data on the current tor metrics site. The code lives here: https://github.com/tigerpaws/tor_metrics Thanks, Walter -- Walter Kim, Ph.D. Mathematician and Engineering Contractor, San Francisco +1-415-509-1429 (mobile) wltr.km(a)gmail.com

2 3

Def Con Kaminsky talk (censorship detection)
by David Fifield 30 Jul '12

30 Jul '12

I saw an interesting talk by Dan Kaminsky at Def Con that touched on some ideas for censorship detection. He mentioned OONI-probe and talked about his project CensorSweeper. It tests blockedness of web sites by making cross-domain requests for favicon.ico and displaying them in a minesweeper-like grid. http://www.censorsweeper.com/ https://www.hackerleague.org/hackathons/wsj-data-transparency-code-a-thon/h… He also mentioned something, which unfortunately I didn't follow very closely, about using Flash sockets to spoof HTTP and HTTPS headers. I think the gag here was sending these spoofed connections to a server you control (so you can answer the crossdomain policy requests without which Flash Player will refuse to connect), but you give it a Host header of a censored site or something like that. http://miriku.com/wp/2012/07/decon-day-3/comment-page-1/#comment-1416 Unfortunately I don't have the conference DVD which presumably contains the slides he used, but videos usually show up online after some number of months. David Fifield

1 0

Brainstorming about steganographic transports
by David Fifield 29 Jul '12

29 Jul '12

This is a summary of some discussion among developers of pluggable transports about steganographic channels and deriving them from protocol grammars. Two things prompted the discussion: * A program called appid, http://code.google.com/p/appid/ , which compiles simple protocol specifications into big regular expressions and uses them to classify network streams. Supported protocols are https://code.google.com/p/appid/source/browse/trunk/apps ; a sample is https://code.google.com/p/appid/source/browse/trunk/apps/afs . Roger suggested to make a program that takes one of these specifications and embeds data within a stream that appid will classify as the chosen protocol. * This paper: "Provably Secure Steganography" http://www-users.cs.umn.edu/~hopper/tc-stego.pdf The authors define a "channel" as a probability distribution on sequences. You can think of a channel as something that answers "given this prior history of packets, what are the possible next packets and their probabilities?" or even as an oracle that merely allows sampling this distribution. They show a method of steganographically encoding a bit within a channel such that that a passive adversary cannot distinguish it from "normal" traffic (Sec. 4.1 and Fig. 1). For the topics we were talking about, the most important sections to read are 2 and 4.1. You can embed data in a string that will match a given regular expression by constructing a DFA (http://swtch.com/~rsc/regexp/regexp1.html) and walking it state by state, choosing state transition edges so as to encode something. For example, if there are two outgoing edges from a state, which one you choose can encode a bit. Many of the appid signatures contain any* blocks that can straightforwardly encode bytes. An example of this idea is the regular expression /^(ab*[cd])+/. The b* lets us encode one bit (say "" = 0 and "b" = 1), and the choice of c or d allows another bit ("c" = 0 and "d = 1). We could send the message 00110001 as "acabdacad". This idea can really be thought of an application of the paper. A regular expression defines a probabilistic channel if we assign probabilties to edge transitions, which we may do uniformly. So, for example, with the regular expression given above, the first byte is 'a' with probability 1. The second byte is 'b' with probability 0.5, 'c' with probability 0.25, and 'c' with probability 0.25. But we see how a channel is sensitive to history: if the string so far is "ab", then the next byte may be 'c' or 'd'; but if the string so far is "ac", then the next byte must be 'a' (or EOF). We can use appid-like signatures to make steganographic channels, if we assume that the signatures are a realistic reflection of actual use of the protocols. But: this relies critically on the accuracy of the model. (Specifically, does it match the censor's model? If he uses simple regular expressions for blocking, then we win; if not, then we probably lose.) Robert Ransom gave the example of a model of HTTP using just the syntax; i.e. using RFC 2616 as the definition of a channel. Such a thing would not only not look like real-world HTTP, but would be broken easily by proxies that rearrange headers, for example. rransom also suggested a way to use arithmetic encoding to obfuscate a stream using a probabilistic model of a protocol; I'll quote him in part: * send it through an arithmetic-code encoder, using a model which contains a (low-probability) 'pad' symbol; * encrypt it with a stream cipher; * send it through an arithmetic-code decoder, using a model which matches the 'cover protocol' (the protocol you are trying to mimic). David (yours truly) wants to write or help write a simple pluggable transport derived from regular-expression signatures, even with the limitations shown above. Client and relay would need matching signature models. For the same of simplicity, it will only seek to match the given signature, and not be indistinguishable in the strong sense mentioned above. It won't do symmetric encryption of the underlying TLS (or if it does, will use a fixed key). It won't use the constructions from the Provably Secure Steganography paper, rather it will just encode its stream directly in DFA edge transitions. I think it will be interesting to see 1) how far a simple system can get us, and 2) what additional changes we would have to make to be provably secure against censors using more sophisticated computational models than regex. David Fifield

4 3

Re: [tor-dev] [tor-assistants] Protocol grammars as probabilistic channels
by Robert Ransom 28 Jul '12

28 Jul '12

On 7/28/12, Zack Weinberg <zack.weinberg(a)sv.cmu.edu> wrote: > [I'm moving this from a giant cc: list to tor-dev, I hope that's okay. > I *think* everyone involved is already on that list.] > > [I apologize for not picking up this ball for so long, I have been ill > and mostly without the brain.] > > On Sun, Jul 22, 2012 at 2:26 AM, Robert Ransom <rransom.8774(a)gmail.com> > wrote: >> >> Specifically, to obfuscate a ‘payload protocol’ data stream: >> >> * send it through an arithmetic-code encoder, using a model which >> contains a (low-probability) ‘pad’ symbol; >> * encrypt it with a stream cipher; >> * send it through an arithmetic-code decoder, using a model which >> matches the ‘cover protocol’ (the protocol you are trying to mimic). > > I like this way of modeling the problem, but I see some practical > problems with it. The payload protocol in this case is Tor's link > protocol, which is already encrypted, so your first step would appear > not to do anything (well, except to TLS's cleartext record headers, > but I don't think that will gain you much). The first step (the arithmetic-code encoder) allows the obfuscator to introduce padding symbols into a bytestream without either understanding or breaking the protocol it is obfuscating. This is the only reason for the obfuscator to use an arithmetic coder on the payload protocol. If the payload protocol is datagram-oriented and can tolerate junk appended to the end of a datagram, or if the software implementing a stream-oriented payload protocol can generate padding at the end of a message when the obfuscator requests it, the payload-side arithmetic coder should be omitted. (Note that Tor itself doesn't satisfy either of these requirements; but Tor-over-TCP-over-IP is a datagram-oriented protocol with a length field in each datagram.) > The stream cipher > provides no integrity protection for the message, which maybe we can > get away with since the higher level has it, and more importantly, no > sequencing. The stream cipher in the middle prevents the payload-side arithmetic-code encoder (or the underlying protocol) from making the stream fed to the cover-side decoder (computationally) distinguishable from a random bitstream. A datagram-oriented payload protocol will need a different scrambling layer as well. The easiest solution I can think of is to encrypt the message header with a large-block block cipher, and assume that clients usually (but not always) keep the same IP address during a protocol session in order to reduce the number of keys that the server needs to try for each message. > Several cover protocols of interest (most importantly > HTTP) break a session up into many short-lived TCP connections. That > means you have to allow for the possibility that cover-protocol > messages will arrive at the decoder out of order. You also need the > ability to send control messages at a level below the payload > protocol, for key agreement, reassociation of new connections with > ongoing sessions, and possibly other things we haven't thought of yet. TCP transports a stream protocol over an unreliable unordered datagram protocol. Surely it can't be too hard to add a security (i.e. encryption and authentication) layer to an unreliable unordered datagram protocol. > The "chopper" component of StegoTorus tackles these problems. Have > you seen the draft paper? Probably not. > It has its own problems -- the one I'm > wrestling with now is, the _amount of payload data_ sent in any given > message gets frozen the first time it is transmitted; if you need to > _re_transmit, you'd better be able to send at least that much data > right now! (Padding can of course be varied.) TCP can increase or decrease the amount of data sent in a retransmitted datagram. (I suspect that real TCP implementations actually do retransmit datagrams with less data, during path MTU discovery.) > I would be curious to know if you think your arithmetic-decoding > approach to cover generation could reasonably be made to work within > StegoTorus' framework. The arithmetic-code decoder on the cover-protocol side may (should) have been proposed before. The other two pieces were my idea, but they were for stream-oriented obfuscation, and you pointed out above that they suck for a datagram-oriented obfuscator and need to be replaced. >>> George turned me on to this paper, >>> >>> "Provably Secure Steganography" >>> http://www-users.cs.umn.edu/~hopper/tc-stego.pdf > > I have to say that I am very skeptical about the practical value of > this paper, because it requires one to characterize the probability > distribution of all possible messages in the cover protocol, which is > not practically possible for any nontrivial protocol. I could see an > approximation being good enough for our purposes, though. (Note that Zack is quoting someone other than me there.) >> Don't hide information directly in the syntax -- it's easy enough for >> Them to rearrange protocol messages in ways that will not interfere >> with standard-conformant clients and servers, but will disrupt crappy >> protocol obfuscators that don't implement the cover protocol >> correctly. Use a semantic model of the cover protocol instead. > > vmon and I have been wrestling with this one a bit in the context of > HTTP and we've pretty much come to the conclusion that we need an > actual implementation of HTTP, into which we plug data. Fortunately > there is no shortage of HTTP implementations. > >> ‘Blocking resistance’ is *very* different from ‘detection resistance’ >> (i.e. ‘indistinguishability from normal traffic’ for some value of >> ‘normal traffic’). >> >> * ‘Blocking resistance’ does not require that a cover protocol be >> ‘indistinguishable from X’ by the adversary, only that the adversary >> be unable to block it. For example, an adversary which can only >> ‘block’ communications by disrupting a connection using extra (forged) >> packets will not be able to block a UDP-based protocol. > > The adversary we're concerned with has control of all the border > routers between the client and the server, so they can do things like > blackhole all packets with a particular source or destination. I > would argue that detection resistance is in fact required for blocking > resistance against this adversary. But only ‘detection resistance’ against any box that They can use to control a blocking system, and that can detect the protocol quickly enough to usefully reduce the amount of data that each user can exchange with each bridge. >> * A protocol which is ‘indistinguishable from’ some protocol which the >> adversary wants to not block may not provide ‘blocking resistance’. >> For example, a crappy obfuscator which uses a syntactic model of HTTP >> instead of a semantic model can be blocked by just about any >> off-the-shelf HTTP proxy. > > That effect also provides a distinguisher from real HTTP, doesn't it? Only if They can tell that They blocked an obfuscator's traffic. They may not be able to detect that. Robert Ransom

1 0

Re: [tor-dev] [tor-assistants] Protocol grammars as probabilistic channels
by Zack Weinberg 28 Jul '12

28 Jul '12

[I'm moving this from a giant cc: list to tor-dev, I hope that's okay. I *think* everyone involved is already on that list.] [I apologize for not picking up this ball for so long, I have been ill and mostly without the brain.] On Sun, Jul 22, 2012 at 2:26 AM, Robert Ransom <rransom.8774(a)gmail.com> wrote: > > Specifically, to obfuscate a ‘payload protocol’ data stream: > > * send it through an arithmetic-code encoder, using a model which > contains a (low-probability) ‘pad’ symbol; > * encrypt it with a stream cipher; > * send it through an arithmetic-code decoder, using a model which > matches the ‘cover protocol’ (the protocol you are trying to mimic). I like this way of modeling the problem, but I see some practical problems with it. The payload protocol in this case is Tor's link protocol, which is already encrypted, so your first step would appear not to do anything (well, except to TLS's cleartext record headers, but I don't think that will gain you much). The stream cipher provides no integrity protection for the message, which maybe we can get away with since the higher level has it, and more importantly, no sequencing. Several cover protocols of interest (most importantly HTTP) break a session up into many short-lived TCP connections. That means you have to allow for the possibility that cover-protocol messages will arrive at the decoder out of order. You also need the ability to send control messages at a level below the payload protocol, for key agreement, reassociation of new connections with ongoing sessions, and possibly other things we haven't thought of yet. The "chopper" component of StegoTorus tackles these problems. Have you seen the draft paper? It has its own problems -- the one I'm wrestling with now is, the _amount of payload data_ sent in any given message gets frozen the first time it is transmitted; if you need to _re_transmit, you'd better be able to send at least that much data right now! (Padding can of course be varied.) I would be curious to know if you think your arithmetic-decoding approach to cover generation could reasonably be made to work within StegoTorus' framework. >> George turned me on to this paper, >> >> "Provably Secure Steganography" >> http://www-users.cs.umn.edu/~hopper/tc-stego.pdf I have to say that I am very skeptical about the practical value of this paper, because it requires one to characterize the probability distribution of all possible messages in the cover protocol, which is not practically possible for any nontrivial protocol. I could see an approximation being good enough for our purposes, though. > Don't hide information directly in the syntax -- it's easy enough for > Them to rearrange protocol messages in ways that will not interfere > with standard-conformant clients and servers, but will disrupt crappy > protocol obfuscators that don't implement the cover protocol > correctly. Use a semantic model of the cover protocol instead. vmon and I have been wrestling with this one a bit in the context of HTTP and we've pretty much come to the conclusion that we need an actual implementation of HTTP, into which we plug data. Fortunately there is no shortage of HTTP implementations. > ‘Blocking resistance’ is *very* different from ‘detection resistance’ > (i.e. ‘indistinguishability from normal traffic’ for some value of > ‘normal traffic’). > > * ‘Blocking resistance’ does not require that a cover protocol be > ‘indistinguishable from X’ by the adversary, only that the adversary > be unable to block it. For example, an adversary which can only > ‘block’ communications by disrupting a connection using extra (forged) > packets will not be able to block a UDP-based protocol. The adversary we're concerned with has control of all the border routers between the client and the server, so they can do things like blackhole all packets with a particular source or destination. I would argue that detection resistance is in fact required for blocking resistance against this adversary. > * A protocol which is ‘indistinguishable from’ some protocol which the > adversary wants to not block may not provide ‘blocking resistance’. > For example, a crappy obfuscator which uses a syntactic model of HTTP > instead of a semantic model can be blocked by just about any > off-the-shelf HTTP proxy. That effect also provides a distinguisher from real HTTP, doesn't it? zw

1 0

Re: [tor-dev] Pyonionoo Descriptors
by Karsten Loesing 28 Jul '12

28 Jul '12

Hi Megan, hi Erik, cc'ing Sathya and tor-dev, hoping you don't mind. On 7/27/12 5:21 PM, Megan Chang wrote: > We are currently almost done with the first draft of pyonionoo. Awesome! :) > We were > wondering what some of the entries might look like so we would be able to > see what information was being parsed to fill in some of the fields such as > the bandwidth handler's "write_history" and "read_history". We noticed that > the router descriptor format includes "write_history" and "read_history" > according to torspec but again, we aren't too sure on what these entries > look like. The bandwidth documents are tricky, because one cannot just look up these values from Tor descriptors. They're based on histories in extra-info descriptors, but they aggregate these values on different levels of detail, and they're specifically designed for graphing. I don't know about your schedule, but if you're short on time, I'd say let's discuss summary and details documents first and ignore bandwidth documents for the moment. (Of course, if you have plenty of time left, happy to discuss bandwidth documents more!) Is your code online somewhere? I'd like to take a look and give you some feedback about Tor descriptor specific things (not so much about Python/Twisted (?) things). > Also, we don't know how to approach the fields in the details > handler that require GeoIP information. Any tips on this? Onionoo uses Maxmind's Java API, their GeoIP database, and their AS database to resolve IP addresses to country/AS information. There's a Python API, too, which I think Sathya used for his network entropy code: https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-6232/pygeoip If you're unsure how to use it, best ask Sathya for more details. Best, Karsten

2 2

script for get bridges
by ??Phoenixing 27 Jul '12

27 Jul '12

hi Firstly, install wget, privoxy, then schedule a task three minutes one time just like: someone>$ wget https://bridges.torproject.org. Then you can get a lot of .htmlxxx files which contain the bridges from the website. Lastly, write a bash command or VBscript under windows, you can grap the xxx.xxx.xxx.xxx:xxxx from the .htmlxxx files, and insert into the torrc file. It 's done! I hope that I can do this again! best regards

1 0

Why change the bridges.torproject.org website to avoid to get bridges by script?
by ??Phoenixing 26 Jul '12

26 Jul '12

Hi I live in a country which it block the tor net strictly! Only I can do is that I get the bridges from the website. I just want to get the bridges automaticly by writing a script. But now I must get the bridges hand by hand and one by one. It's more uncomfortable! Could you change it back? best regards!

3 2