- tor-dev - lists.torproject.org

Building better pluggable transports (Google Summer of Code)
by Steven Murdoch 29 May '13

29 May '13

Hi Chang, We've been discussing how to build better pluggable transports for Tor as part of your application to Google Summer of Code. Now that you've been accepted, I thought it would be good to bring this discussion to tor-dev so that others can contribute. The basic idea behind the project is to build a pluggable transport which is as unlikely has possible to be blocked. In particular, we are most interested in improving the situation in countries where obfs2/obfs3 is blocked or likely to be blocked in the near future. There are two basic options, which would ideally be combined: - Better camouflaging Tor traffic - Scanning resistance On better camouflaging Tor traffic, the benchmark we have is obfs3, which converts Tor traffic to data which is indistinguishable from random bytes (but timing and packet-size patterns are not disguised). As far as I'm aware, this is not being blocked anywhere but it may be possible to block based on the fact that there's not much truly random data on the Internet. Also, obfs3 will not get through a HTTP proxy, as it is clearly not HTTP. So one option for the project is to impersonate HTTP. This is deceptively difficult because although HTTP is transmitted over TCP, the properties it offers to higher layers are not as strong as TCP (and not as required by Tor). For instance, individual HTTP requests may be re-ordered if they are over different TCP connections. Also, responses may be truncated without an error being reported to higher layers (which is why HTTP includes length fields as an option). HTTP doesn't give the same congestion avoidance as TCP and proxies can both cache and modify data they transmit. The HTTP specification is vague on some topics, and even when it specifies a particular behaviour, proxies frequently violate the specification. On the up-side of a HTTP proxy, HTTP is probably one of the last protocols a country will block before they turn off the Internet completely, so it has a good chance of getting through. Also, in some scenarios the only way for traffic to get out is via a HTTP proxy. So I think there is significant usefulness in this option. Also, it is incredibly difficult to hide one protocol inside a different one, because just recording the approximate number of bytes sent vs bytes received can give a good recognition of the protocol [4]. Hiding HTTP-over-Tor as BitTorrent traffic will likely be detectable based on such a statistic, but HTTP-over-Tor as HTTP at least has a chance. On the side of scanning resistance, we discussed the challenge of implementing scanning resistance with TCP. Here, the problem is that someone sending a SYN packet to a port which is open will receive a SYN-ACK, regardless of what user code does. To resist scanning (with something like BridgeSPA [1]) the pluggable transport would need to be quite tightly integrated with the OS rather than just using the standard socket API. Therefore it will create deployment difficulties, especially on Windows which has locked-down the raw sockets API. Therefore it might be interesting to send data over UDP rather than TCP, as then it is the responsibility of the user code to send the SYN-ACK-equivalent. Tor needs properties similar to TCP from it's pluggable transport, and so any UDP-pluggable transport would need something which replaces TCP: reliable in-order delivery with congestion management. One option here is libutp, as used by BitTorrent [2]. There is a vast amount of libutp traffic on the Internet, but it's timing and upstream/downstream characteristics will be different from how Tor would use it. Alternatively, it might not be worth worrying about this type of scanning resistance, and just focus on what it is possible to do with TCP, as done with ScrambleSuite [3]. Chang, George, do you have anything to add to this summary? Does anyone else on tor-dev have thoughts on these topics? Best wishes, Steven [1] http://www.cypherpunks.ca/~iang/pubs/bridgespa-wpes.pdf [2] https://github.com/bittorrent/libutp [3] http://www.cs.kau.se/philwint/pdf/scramblesuit2013.pdf [4] http://dedis.cs.yale.edu/2010/anon/papers/parrot.pdf

7 10

Searchable Tor descriptor archive - GSoC 2013 project
by Kostas Jakeliunas 29 May '13

29 May '13

Greetings! I'm a student who will be working on the Searchable Tor descriptor archive as part of Google Summer of Code. Yay! I've been following Tor development for a while and hope that this opportunity will be my way of sneaking into the development kitchen of Tor. In any case, I hope to stay around for a longer time to come. The original GSoC project proposal is based on one of the Tor project ideas available [1] and is part of the Tor Metrics project [2]. The GSoC proposal itself is also available to read [3] (TXT; if there's any interest, I can work on reformatting.) My primary mentor is Karsten and my secondary mentor is Damian. I will quote the abstract from the proposal to sum up the high-level goals of this project: I'd like to create a more integrated and powerful descriptor archival > search and browse system. (The current tools are very restrictive and the > experience disjointed.) To do this, I'll write an archival browsing > application wherein the results are interactive: they may act as further > search filters. Together with a search string input tool which will have > more filtering options, the application will provide a more cohesive > archival browse & search experience and will be a more efficient tool. > So as of now, we have an array of tools for inspecting, searching for and getting aggregate data about running relays. (For an overview, see the Tools page in the Metrics portal. [4]) These tools include relay search, consensus info, exit-by-IP search, and quite a few more; furthermore, two Onionoo [5] based applications/tools: Atlas and Compass. This project would proposes to: - implement a more powerful backend that would allow one to search for all available relays since mid-2007 (I should have clarified in the previous discussions, and Karsten already includes this bit; i.e., since v2 statuses became available [6]; I guess this can also be discussed). "More powerful" here means, first and foremost, "all (>= v2) archival data" (relay descriptors and consensuses at the very least), and furthermore (at least per the original proposal), involving more complex queries: we'd be looking into, I think, minimally, combined AND/OR filters referring to a wider range of data fields available in the archival data and the ability to specify multiple date ranges. Referring to consensus-related data while searching for relays and vice versa would also be possible. (The capabilities would therefore also include those of exoneraTor.) - implement backend results which would, as of current standing, aim for Onionoo compatibility (again see protocol design in [5]), or perhaps supersede it while providing backwards compatibility (e.g. returning paginated lists of consensus-status-entries where a specified relay was present.) - (as per original proposal,) implement a more powerful archival descriptor search & browse tool (frontend) which would provide a more uniform "looking up relays" / "searching by using many criteria" / "further refining search in the results page" experience - "refining search results", i.e. adjusting filters would be semantically the same as entering search criteria in the beginning; hence a more interactive experience, a more powerful search/browse tool. The goals and design of the project have to be clarified, however. There is ongoing discussion (see another tor-dev thread [7] e.g.) whether perhaps the focus could be to create a backend which would speak the full Onionoo protocol and therefore be a potential replacement not only for relay search and exoneraTor, but also for other components: all presently-speaking Onionoo applications could be made to talk to the new backend, for example. The overall count of components will hopefully be reduced in any case, but ideally, we would end up with a much more integrated Tor Metrics (and maybe beyond) ecosystem. Many open questions, however - see again [7]. Obviously discussions are very welcome indeed! I'm wfn on OFTC (#tor-dev, #nottor), also reachable via XMPP < phistopheles(a)jabber.org>, and am very much up for any kind of chat. :) I'll be busy with exams in the first three weeks of June, though - but will find time for sure! Regards Kostas. [1] https://www.torproject.org/getinvolved/volunteer#metricsSearch [2] https://metrics.torproject.org/ [3] http://kostas.mkj.lt/gsoc2013.txt [4] https://metrics.torproject.org/tools.html [5] https://onionoo.torproject.org/ [6] https://metrics.torproject.org/data.html#relaydesc [7] https://lists.torproject.org/pipermail/tor-dev/2013-May/004940.html

1 0

New control function
by grarpamp 28 May '13

28 May '13

I'm implementing a new control command to print some info requested from the network and/or internal state, similar to getinfo. Found the function that it will call and am making those changes. Is there a suggested entry to the control command parser for new commands? Or a dummy template that one should follow?

2 2

Damian's Status Report - May 2013
by Damian Johnson 27 May '13

27 May '13

Hi all. This is a little early but considering the time sink GSoC has been I doubt I'll get much more done, so here's my status report for May. The few stem tasks I've snuck in included... * Added Ubuntu and Fedora to our download page [1], Fedora package is thanks to Juan Orti * Several testing fixes so our Jenkins tests runs now pass [2] * The Controller mangled non-unicode descriptor content when using python 3.x (caught thanks to aj00200, #8755) * Expanded our client usage tutorial [3] to use SocksiPy and include an example for polling Twitter (thanks to Ashish) * Integ tests now assert ownership on the tor process (patch by Jeremy Kushner, #8634) * The DescriptorReader mishandled relative paths (patch from Kostas, #8815) * Fix so the Controller cache is thread safe (patch by Akshit, #8607) * Running our integ tests littered the /tmp directory (patch by Ashish, #8622) * Improvements so we use the staticmethod decorator and new style exception catching (patches from Sean, #8824 and #8823) Cheers! -Damian [1] https://stem.torproject.org/download.html [2] https://jenkins.torproject.org/job/stem-tor-ci/ [3] https://stem.torproject.org/tutorials/to_russia_with_love.html

1 0

Re: [tor-dev] "Trawling for Tor Hidden Services: Detection, [...]"
by Jon Smithe 25 May '13

25 May '13

Hi, > As for the deanonymization attack, I think it is pretty novel in that it > uses a custom traffic signature to make the attack from > http://freehaven.net/anonbib/cache/hs-attack06.pdf more reliable, but > otherwise that is why we introduced guard nodes. I'm still pretty sure this only makes people more secure in theory; the basis appears to be that instead of connecting to a random node every time a circuit is built, you instead connect to only one entry/guard node for 30-60 days. I think on paper that reduces the probability, but it seems to presume an attacker that doesn't modify their attack and doesn't appear to account for a keyspace of sorts that has reduced by over 50% and the introduction of a "touch me here" flag that says you're not only a relay, but you're guaranteed to be an entry point relay. Looking at the numbers from a snapshot a few days ago: 839 exits, 865 guards, 1676 generic relays. Previously for entry you'd be looking at a 'keyspace' of n/(839+865+1676), with the obvious problem that connecting to them at random every time you create a circuit is going to increase the probability that you hop through a compromised node. However, you need a fairly large value of N to make that realistic, I think the number cited was approaching 50%, but I could be remembering that incorrectly. Now, you're at 865 _guaranteed_ entry points, which automatically reduces the required value for N and increases the value of each rogue node that gains the guard flag, which appears to be strictly based on bandwidth and stability. I'm hardly an expert on math or probability, but it seems like the 'weight' so to speak of a compromised node was never increased to account for the reduced number of nodes or guarantee that being a guard will always yield entry point traffic, which is to say that N is actually going to be some value multiplied by N instead of just N/C^2, (N*nWeight/C or possibly even N^nWeight/C). It at least seems like the probabilities were only worked for the state the network was in, and not for the state/behavior of the network after the modifications. So now you have a network where one can setup two boxes that are fast and stable and be guaranteed to receive exit and entry traffic and only needing about half as many boxes to reach the same 50% mark as before. The math behind this concept is not overly compelling or I'm just dumb, both are probable and neither are mutually exclusive, but if I were looking for a state-based backdoor, I'd imagine it to look a bit like this (which is not to imply that is the case here by any means). Jon On Thu, May 23, 2013 at 11:35 PM, <tor-dev-request(a)lists.torproject.org> wrote: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Hidden services performance (Petter Solberg) > 2. Re: Hidden services performance (Jeroen Massar) > 3. Chrome browser and sand boxing. (Andrew F) > 4. Re: Chrome browser and sand boxing. (Andrew Lewman) > 5. Re: Chrome browser and sand boxing. (Andrew F) > 6. "Trawling for Tor Hidden Services: Detection, Measurement, > Deanonymization" (Tom Ritter) > 7. Re: "Trawling for Tor Hidden Services: Detection, > Measurement, Deanonymization" (Mike Perry) > 8. Re: "Trawling for Tor Hidden Services: Detection, > Measurement, Deanonymization" (Nick Mathewson) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 23 May 2013 14:46:12 +0200 > From: Petter Solberg <pettersolberg88(a)gmail.com> > To: tor-dev(a)lists.torproject.org > Subject: [tor-dev] Hidden services performance > Message-ID: > <CAPnamRP==01+HkQQWbg_mQv0uF=f3hK02jvffXDJBW8UfyMueQ(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hi. > > We are two master thesis students at the Norwegian University of Science > and Technology (NTNU) which is looking into the performance of hidden > services. We have already set up 19 physical Linux servers connected by > gigabit Ethernet to be able to benchmark hidden services in a private tor > network. We have also some public servers that we plan to utilize ( > https://atlas.torproject.org/#search/disasterTor ). This mail is to inform > you of our work and a request for ideas in where some bottlenecks are and > if someone have done some previous research before. > > Without any modifications to Tor we have been able to have a throughput of > 180Mb/s through a single hidden service. > > > If you have some Ideas or a comment. Please reply. > > > > Regards > > Petter Solberg (pettso AT stud.ntnu.no ) & Bram Bezem ( bezem AT > stud.ntnu.mo ) >

2 1

Chrome browser and sand boxing.
by Andrew F 23 May '13

23 May '13

I have Googled and searched the Tor website, but I have not found any information on chrome's sandbox feature and Tor. Specifically with flash. Is there any information out their on this? Thanks

2 2

Hidden services performance
by Petter Solberg 23 May '13

23 May '13

Hi. We are two master thesis students at the Norwegian University of Science and Technology (NTNU) which is looking into the performance of hidden services. We have already set up 19 physical Linux servers connected by gigabit Ethernet to be able to benchmark hidden services in a private tor network. We have also some public servers that we plan to utilize ( https://atlas.torproject.org/#search/disasterTor ). This mail is to inform you of our work and a request for ideas in where some bottlenecks are and if someone have done some previous research before. Without any modifications to Tor we have been able to have a throughput of 180Mb/s through a single hidden service. If you have some Ideas or a comment. Please reply. Regards Petter Solberg (pettso AT stud.ntnu.no ) & Bram Bezem ( bezem AT stud.ntnu.mo )

2 1

[Fwd: Welcome to the "tor-dev" mailing list]
by bones44x6013＠tormail.org 22 May '13

22 May '13

Hello again, Now that after several hours of work, we have come full circle back to the same email address I used for my original contacting you? I was not allowed to contact you through this address because I was not a registered user of this system? Now that I am a registered user of this system, I still have the same questions for you...Is this your most secure form of communication and is it safe for me to send you this information via a MS word document here through this email? This information should only be viewed by top Tor Developers and Management it is not meant for general discussion or a general discussion board viewing. This is the reason for my asking again about this again. I am providing you with a new technique for the secure communications of bridge address network locations. This method is proven and will work for you. Bones44 ---------------------------- Original Message ---------------------------- Subject: Welcome to the "tor-dev" mailing list From: tor-dev-request(a)lists.torproject.org Date: Wed, May 22, 2013 4:25 pm To: bones44x6013(a)tormail.org -------------------------------------------------------------------------- Welcome to the tor-dev(a)lists.torproject.org mailing list! To post to this list, send your email to: tor-dev(a)lists.torproject.org General information about the mailing list is at: https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: https://lists.torproject.org/cgi-bin/mailman/options/tor-dev/bones44x6013%4… You can also make such adjustments via email by sending a message to: tor-dev-request(a)lists.torproject.org with the word `help' in the subject or body (don't include the quotes), and you will get back a message with instructions. You must know your password to change your options (including changing the password, itself) or to unsubscribe. It is: zenuokna Normally, Mailman will remind you of your lists.torproject.org mailing list passwords once every month, although you can disable this if you prefer. This reminder will also include instructions on how to unsubscribe or change your account options. There is also a button on your options page that will email your current password to you.

2 1

Move RecommendedTBBVersions from check.torproject.org to www.torproject.org
by Micah Lee 22 May '13

22 May '13

I just opened a ticket for this: https://trac.torproject.org/projects/tor/ticket/8940 Right now the way to check for the current TBB version is by loading https://check.torproject.org/RecommendedTBBVersions. It would be really helpful if there were a way to check the current version of TBB from a URL at https://www.torproject.org/. The main website has a .onion address, and all of the Tor mirrors listed at https://www.torproject.org/getinvolved/mirrors.html.en are mirrors of the main site, not check. So if you want to only use the hidden service or only use a mirror (because *.torproject.org is blocked on your network), there's no way to easily know the current recommended version. -- Micah Lee @micahflee

1 0

building from source in a 64-bit windows environment..
by not me 20 May '13

20 May '13

Hi, It seems fairly self-evident that tor hasn't been build for 64-bit windows and questionable whether it's ever been built in an environment that doesn't utilize mingw. There are literally hundreds of thousands of warnings about integer truncation, at least some of which seem somewhat problematic (64-bit pointer being stored in a 32-bit integer (why?! why?!)), a small amount of signed/unsigned comparisons or sign extensions, and a bunch of calls to functions without including the proper header files or using the correct function name ('open()' vs '_open()' without including io.h) and so on. This of course is ignoring all the calls to str{,n}cpy/{v,}s{,n}printf/etc and posix function names long since deprecated (fileno() vs _fileno()) and so on. Errors relating to MSVC finally including a stdint header (cstdint) and typedef's for intptr_t not producing the right size primitive, etc. I realize that many/most/all of the developers here are going to see no problem with using mingw, but if you spend some time looking underneath the hood and reviewing the code generated by mingw v. msvc both in terms of security features and general performance, I can't feel that its a prudent or responsible choice to produce anything under windows using GNUs development tools. So I'd prefer to just eliminate all such 'fixes'. Moving to 32-bit isn't an option either, I'd have to switch out far too many dependencies that frankly are a bit more important, and well I need 64-bit for some aspects and it's only been a decade since AMD released their first 64-bit chip. So, has anyone out there successfully built tor with a 64-bit msvc, and if so, you don't happen to have a list of voodoo tricks you performed or a diff or similar? Thanks.

6 22