tor-dev August 2011

tor-dev@lists.torproject.org

17 participants
16 discussions

Tor2web 2.0 is live!
by Arturo Filastò 20 Aug '11

20 Aug '11

Hi all, We are glad to announce the release of the new tor2web software. For those of you who are not aware of what tor2web is let us give you a brief description. The goal of tor2web is that of promoting the use of Tor Hidden Services (https://www.torproject.org/docs/hidden-services.html.en). Hidden Services allow people to run TCP based services without disclosing the identity/location of their server. In the specific they allow people to anonymously publish content to the web. Also, since you are being reached trough the Tor network, you are not required to have a static ip address or purchase a domain. This lowers the entry barrier to content publishing and protect the content publisher from retaliation and Denial of Service attacks. The problem though is that Hidden Services are usually only accessible by installing a Tor client (https://www.torproject.org/projects/torbrowser.html.en). Tor2web creates a transport, by acting as a web proxy, between the internet and the Tor network. This means that anonymous publishers are able to reach a much wider audience. The user visiting a website though tor2web is always advised to install a Tor client as by doing so he will protect his identity and leverage Hidden Services end-to-end encryption. This version of tor2web (called tor2web 2.0) is based on glype PHP web proxy (http://www.glype.com) and it is by no means the definitive solution. We are currently working on a new design that will be able to withstand other "attacks" that are currently possible. What we have implemented is: * A clear disclaimer warning the user that the content is not being served directly from the server, but it comes from the Tor network * Contact forms for abuse complaints and to report broken websites * Transparent rewriting of URLs into the tor2web form (i.e. so4rmjdiwmqjosxz.onion become so4rmjdiwmqjosxz.tor2web.org) * Blocklists to allow a tor2web node maintainer to block particular websites, the blocklists are stored in md5 format so the node maintainer does not need to store potentially illegal site lists. At this current stage we would like the community to stand-up and help us by: * Finding security and functional bugs in the existing implementation * Volounteering to run new tor2web servers: In this first stage we are looking for reliable systems, run or endorsed by trustworthy organizations involved in anonymity and privacy research and development. For the new release the goals that we wish to further pursue are: * Distribute responsibility across multiple actors * Minimize the probability of takedown of a tor2web node If you want further information on the tor2web project visit: Wiki for new developments: http://wiki.tor2web.org/ Tor2web original website: http://www.tor2web.org Github: https://github.com/globaleaks/tor2web-2.0 Mailing List: tor2web-talk(a)lists.tor2web.org on http://bit.ly/pxFwNS . IRC: irc.oftc.net #tor2web Have a nice day, Some Random GlobaLeaks Contributors Please spread across the anonimity communities and mailing lists

1 0

Blocking-Test: GSoC Penultimate Status Report
by Brandon Wiley 12 Aug '11

12 Aug '11

Scoring and reporting for detectors and encoders is pretty much done. The graphs aren't quite ready yet, for aesthetic reasons, but I have some nice tables to report: Length detector: 16% accuracy Timing detector: 89% accuracy Entropy detector: 94% accuracy SSL was correctly identified 25% of the time, 70% of the time other protocols were misidentified as being SSL, and 5% of the time SSL was misidentified as another protocol. obfsproxy was correctly identified 55% of the time, 10% of the time other protocols were misidentified as being obfsproxy, and 35% of the time obfsproxy was misidentified as another protocol. Dust was correctly identified 48% of the time, 4% of the time other protocols were misidentified as being Dust, and 48% of the time Dust was misidentified as another protocol. obfsproxy is distinguishable from SSL 96% of the time. obfsproxy is distinguishable from Dust 98% of the time. Dust is distinguishable from SSL 56% of the time. I'm sure these numbers are confusing as several different sorts of things are being measured, all with similar numbers. It is probably quite unclear how all of these percentages add up. I think this is where the graphs will help. In particular I think it's hard to visualize accuracy when there are both false positives and false negatives to take into account. I think the most interesting result here is the entropy detector. It's the simplest and also the most effective. Not only that, but the entropy detector only looks at the entropy of the first packet, so it's inexpensive. The original reason that I implemented this detector was that there was an argument against Dust that it could be trivially filtered by setting an entropy threshold above which all traffic is filtered. Surprisingly, Dust has lower entropy than SSL, so this attack will not work to specifically target Dust. This is unexpected because Dust uses totally random bytes, whereas the SSL handshake does not. This is a result of a general issue with how Shannon entropy is defined. Shannon entropy is normally defined across a large statistical sampling, in which case all three protocols tested will converge to maximum since they are all encrypted and therefore apparently random. However, the attacker does not in this case get a large statistical sample. Since they don't know a priori which connections are Dust connections, they only get however many packets are in each trace to use as samples. The current version of Dust that I'm testing has most of the advanced obfuscation stripped out as I haven't had time to add it back in after completely reimplementing the protocol to use TCP instead of UDP and all of the modified assumptions that come with that change. So this version of Dust is basically just the ntor protocol with no packet length or timing hiding in use. The first packet is therefore 32 random bytes consisting of the curve25519 ephemeral ECDH key, as compared to the 1k first packet of SSL. This is the reason for the low entropy calculation, because given only 32 bytes to sample almost all possible bytes are going to have a probability of 0 or 1. There is an upper bound on entropy when sample sizes are small. Achieving maximum first-order entropy here would require at least 512 bytes so that each value 0 to 255 could occur exactly twice. There are of course many ways to measure entropy and you could say that I'm just doing it wrong. Some alternative measurements have already been suggested that I might implement as well. I am not suggesting that Dust is in any way immune to entropy-based detection. In fact, Dust is currently detected very well by the current entropy detector (it passed 100% of the packet length and timing tests, so all detection of Dust was done by the entropy detector) precisely because it has strangely low entropy because of it's small first packets. I bring this up only as an interesting example of how intuition can fail when trying to use Shannon entropy to measure individual messages instead of channels. As far as the state of the project goes, I think the project can be considered done. Looking at the project documentation<http://stepthreeprivacy.org/documentation>, everything didn't work out exactly as originally planned, I think it all worked out for the best and the project is where I wanted it to be by the completion of Summer of Code. I have generation of HTTP and HTTPS traffic, both over Tor and plain. I have two encoders: obfs2 and Dust. I have three detectors: length, timing, and entropy. The string detector didn't work out, but the entropy detector worked out better than planned, so on balance I think it was a win. I didn't implement all of the detector modes because upon further reflection it doesn't really make sense to just try all of these different modes. You want the mode that works best. So for packet lengths and timing that is full conversations and for entropy that's the first packet (could be first N packets, but I tried N=1 and it worked great). Random sampling of packets just seems like a way to limit your success at this point when non-random sampling of just the first packet seems to be so effective. All of the utilities have been written, and more. I have the specified utilities for separating the pcap files into streams, and extracting the strings, lengths, and timings. Additionally, I have some utilities for extracting the entropy of streams, tagging the streams with what protocol they are using (by port), and extracting dictionary and corpus models for latent semantic analysis. I also have some utilities for graphing distributions of properties of different protocols which I hope to finish up next week during the wrap-up period. Looking to the future, I have changed my goals somewhat from what I had in mind at the beginning of the project. After having attended the TorDev meetup in Waterloo and the PETS conference, I am much more interested in integrating my work with the Tor project instead of just testing the various protocols that have been proposed in order to see how my own protocol stacks up against the competition. As part of this transition, I have rewritten Dust from scratch, both in terms of implementation and in terms of the protocol, in order to be better suited for use as a pluggable transport for Tor. My goal for Dust is to have it shipped with both Tor clients and bridges so that it can be used in the field to contact bridges despite DPI filtering of Tor connections. My goal now for the blocking-test project is to focus on blocking resistance for Tor. So instead of adding all of the protocols under the sun, I'm going to concentrate on protocols that have been implemented or are under consideration to become pluggable transports. Additionally, the next protocol I'm going to add after GSoC is over is just plain Tor. There is at least one bug filed about a suspicion that Tor can currently be fingerprinted based on some packet characteristics and I'd like to be able to close that bug because we now have the capability to easily generate the necessary information. Finally, I would like to offer some insight into what I think is the future of blocking-resistant transports now that I have done some work trying to break them. Fundamentally, the only thing an attacker can do is limit the bitrate of the connection. Given any sort of channel, we can encode arbitrary information over that channel. However, the smaller the channel the slower the bitrate. The goal of encodings should be to maximize the bitrate given the constraints of the censor. It's easy to develop very slow encodings such as natural language encodings over HTTP. However, in order to maximize the throughput you need to have a good definition of the attack model. All encoding for blocking-resistant purposes lowers the efficiency above just transmitting the normal content (assumed it has already been compressed). Therefore all encoded conversations are going to be longer than unencoded conversations. I think this is the future of writing detectors because it's a fundamental constraint on encoding. I think this should even be able to defeat something like Telex (although let's not get into the details of attacks on Telex specifically, I'm talking about a general idea here). Given a (static, for the sake of argument simplicity) website, I can download every page on that website. Then I can compare the length of intercepted downloads and detect when they don't match up. For dynamic websites you can do the same thing with a statistical model. I think, therefore, that the future of blocking-resisting protocols is to encode single logical connections over an ensemble of multiple actual connections. Ideally these connections would mimic normal traffic in terms of number of connections, duration, directional flow rates, etc.. It's something to think about anyway. It could be an interesting problem.

1 0

Instructions for building the Vidalia-bundle?
by Steve Snyder 09 Aug '11

09 Aug '11

Is there documentation anywhere on how to build the Vidalia-bundle for Windows? If so, where can I find it? Thanks.

3 3

GSoC Update
by Brandon Wiley 05 Aug '11

05 Aug '11

With the TorDev meetup and everything it's been a while since I sent out an update on the state of the project, although I think we more or less caught up on the status at the meetup. The bulk of the implementation work had already been completed, so what I've been working on since then is just cleaning up the code and rewriting some parts. In particular this week I rewrote the detectors to be mostly python. The core Bayesian model is still in BUGS/JAGS and there is still R code to interface with that. However, the output of the models after training is now converted to a CSV file and the actually detection code is a python program which reads the CSV file with the trained model's prediction data and the compares it to the data of a stream to be classified. This means that you only need R for training the models. If we can put together a canonical model for each protocol then you can use it to classify new traffic just using python. I think this is a win because R was the bulkiest dependency. It also segments the process more nicely in that there can be some people gathering traces, other people building models, and then other people running detectors against new traces. In terms of code cleanup, I've been adding comments and refactoring large files (in particular pavement.py) to be more readable. I also have the scoring tasks implemented so that, given the output of the detectors, the various detectors and encoders are scored based on their number of correct guesses, false positives, and false negatives. The only functionality left to implement is some sort of reporting, probably in the form of graphs of the scores so that detectors and encoders can be compared. The rest of the work is just going to be refining everything and getting both the workflow and the code structure to be smooth. Also, I've stopped working on the string detector because I realized that it's irrelevant for the two encoders I actually have support for right now: obfs2 and Dust. They both already use totally randomized packet contents and so will not be vulnerable to this kind of detector. Other encodings will be vulnerable, but the string detector can wait until we support these encodings. So instead I've implemented an entropy detector, which should work very well against both obfs2 and the current implementation of Dust. The entropy detector is actually the easiest model so far as the entropy of a trace is just a single number, so that's convenient.

1 0

Requesting feedback on TorDNSd v1.1
by LETO 02 Aug '11

02 Aug '11

Could some of you be so kind to try out my latest version of TorDNSd? It works a lot like ttdnsd with some additional features: - Filters to tell TorDNSd what requests to forward directly, forward through the SOCKS proxy or reject. - Remaps to define your own query replies (Currently only supports A, MX and NS replies) - Interactive shell (tordnsd-shell.exe): config-add settings and see them in action right away, will add extra features in the future. - Accepts both UDP and TPC DNS requests, forwards both through TCP requests - Basic caching of replies (optional) The rejects are handy to for example prevent leaking .onion requests (or any request if you want) You can also find some additional information about this release on my blog ( http://leto-r.blogspot.com/2011/07/tordnsd-v11-is-out.html ) It is written in C# and has been tested by myself under mono 2.6.7 / ubuntu natty. Check the default configuration @ https://raw.github.com/LETO-R/TorDNSd/b7aa04e980ad62308d4f2fa3143700c85b537… the full list of available settings. You can either download the mono compatible bins ( https://github.com/downloads/LETO-R/TorDNSd/tordnsd-v1.1-all-bin.zip ) or the windows installer ( https://github.com/downloads/LETO-R/TorDNSd/tordnsd-v1.1-win-setup.exe ) You can find the source code of these bins @ https://github.com/LETO-R/TorDNSd/tree/b7aa04e980ad62308d4f2fa3143700c85b53… sure to get the submodules too) The sourcecode should be compilable using either MonoDevelop 2.4.2 (using the mono runtime) or Visual Studio 2010. If you do not run it as root on a non-windows system, a fatal error is printed since TorDNSd will most likely not be able to bind on port 53. Specify --no-root to skip this check. Kind of feedback I'd like of you: - What OS did you try it on? When ran using mono, what version? - Did it work? Did it require the elevated rights (root / administrator)? Did the root-checking work on non-windows systems? - Did the shell work for you? If not, what issues did you notice? Any improvements that you suggest? - Are the settings explained well enough (check tordnsd.conf)? - Is the supplied default configuration sufficient? Any filter / remaps I forgot? - Any other feedback you can think of, don't hold back! This is my first attempt at a cross-platform .NET project that contains some non-windows specific code / workarounds (mostly for the shell) so I really could use the feedback. If you do not wish to run it as your main dns resolver 'just yet', you can always test it by using dig (examples: 'dig @127.0.0.1 bla.onion' to test rejection, or 'dig @127.0.0.1 vescum.tor' to test the remap feature) All I ask is to stay constructive. If you wonder, I constantly have TorDNSd (shell) running now and have configured it as my main dns resolver as I consider it secure. Hope you find this to be true as well. - LETO

3 5

[Patch] or/networkstatus.c
by Gisle Vanem 01 Aug '11

01 Aug '11

Hi list. In or/networkstatus.c there is a "#if 0" block inside the macro "SMARTLIST_FOREACH_JOIN". Not all compilers handle such contructs. In the prosess of making solution/projects file for "MS Visual C++ 2010 Express", I hit this problem (cl Version 16.00.30319.01). Can you please accept the following patch: diff --git a/src/or/networkstatus.c b/src/or/networkstatus.c index 2586ce6..d8d6680 100644 --- a/src/or/networkstatus.c +++ b/src/or/networkstatus.c @@ -2001,19 +2001,25 @@ routers_update_status_from_consensus_networkstatus(smartlist_t *routers, routers_sort_by_identity(routers); + /* Since not all compilers handles '#' inside macros, we use a helper-macro. */ +#if 0 + #define CHECK_ROUTER_PURPOSE() do { \ + /* We have no routerstatus for this router. Clear flags and skip it. */ \ + if (!authdir) { \ + if (router->purpose == ROUTER_PURPOSE_GENERAL) \ + router_clear_status_flags(router); \ + } \ + } while (0) +#else + #define CHECK_ROUTER_PURPOSE() +#endif + SMARTLIST_FOREACH_JOIN(ns->routerstatus_list, routerstatus_t *, rs, routers, routerinfo_t *, router, tor_memcmp(rs->identity_digest, router->cache_info.identity_digest, DIGEST_LEN), + { CHECK_ROUTER_PURPOSE(); }) { -#if 0 - /* We have no routerstatus for this router. Clear flags and skip it. */ - if (!authdir) { - if (router->purpose == ROUTER_PURPOSE_GENERAL) - router_clear_status_flags(router); - } -#endif - }) { /* We have a routerstatus for this router. */ const char *digest = router->cache_info.identity_digest; ------------------------ --gv

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev August 2011