- tor-dev - lists.torproject.org

The ScrambleSuit pluggable transport / Philipp's May 2013 and earlier
by Philipp Winter 01 Jun '13

01 Jun '13

Over the last months, I have been working on a pluggable transport protocol called "ScrambleSuit" [1, 2]. It is one---but not the only---answer to active probing as done by the Great Firewall of China. Active probing allows the GFW to detect and block vanilla as well as obfs2 Tor bridges; and perhaps also obfs3 in the future. ScrambleSuit uses a secret which is shared out-of-band. In practice, this will be a long password distributed by BridgeDB. A ScrambleSuit server only responds if the client is able to prove knowledge of the shared secret. Besides, the protocol is able to randomise its packet length distribution and inter arrival times. The protocol header understands packet padding, so it would be possible to improve traffic analysis defence in the future. While ScrambleSuit will obviously work with Tor, it is also supposed to support protocols other than Tor---as long as they understand SOCKS. This should make it possible to tunnel, e.g., VPN traffic over ScrambleSuit. Implementation-wise, the protocol is a set of Python modules for obfsproxy. The source code is now available but still under development [3]; so please don't use it on real systems just yet. There are also two open tickets which need to be resolved before ScrambleSuit can publish its shared secret to BridgeDB [4, 5]. [1] http://www.cs.kau.se/philwint/scramblesuit/ [2] https://en.wikipedia.org/wiki/A_Scanner_Darkly [3] https://gitweb.torproject.org/user/phw/scramblesuit.git [4] https://trac.torproject.org/projects/tor/ticket/8929 [5] https://trac.torproject.org/projects/tor/ticket/8979 Cheers, Philipp

1 0

"Trawling for Tor Hidden Services: Detection, Measurement, Deanonymization"
by Tom Ritter 31 May '13

31 May '13

RPW's, et al's paper was made public today, and demonstrates several practical attacks on Hidden Services. http://www.ieee-security.org/TC/SP2013/papers/4977a080.pdf I was wondering if there were any private trac tickets, discussions, or development plans about this that might be also be made public. -tom

6 9

obfs3 and Android
by Gordon Morehouse 30 May '13

30 May '13

Hi there, I know a user in China with an Android device who is having trouble with the GFW. I run private bridges for a couple friends in odd places and recently upgraded to obfs3, but after inquiring on IRC this morning I found that Android doesn't have obfs3 support yet: 17:17 < gamambel> gmorehou: unfortunately, not yet. someone has to either implement obfs3 in C, or make the new python obfsproxy work on Android. i don't think anybody is working on that yet. Just a quick question (apologies if it's a repeat) from a totally Android-naive developer: is using Jython a possibility to speed the availability of obfs3 on Android? Last I knew, which was in the dark ages, some Python code could be tortured into running on Android devices by compiling it with Jython. Just curious if this has been considered or not. I don't know what Android-specific problems there may be, but I know that in general Jython lags behind the latest stable Python, so e.g. code written against Python 2.7 might need to be tweaked to run against 2.5, etc. -- Sent from my device

4 6

Building better pluggable transports (Google Summer of Code)
by Steven Murdoch 29 May '13

29 May '13

Hi Chang, We've been discussing how to build better pluggable transports for Tor as part of your application to Google Summer of Code. Now that you've been accepted, I thought it would be good to bring this discussion to tor-dev so that others can contribute. The basic idea behind the project is to build a pluggable transport which is as unlikely has possible to be blocked. In particular, we are most interested in improving the situation in countries where obfs2/obfs3 is blocked or likely to be blocked in the near future. There are two basic options, which would ideally be combined: - Better camouflaging Tor traffic - Scanning resistance On better camouflaging Tor traffic, the benchmark we have is obfs3, which converts Tor traffic to data which is indistinguishable from random bytes (but timing and packet-size patterns are not disguised). As far as I'm aware, this is not being blocked anywhere but it may be possible to block based on the fact that there's not much truly random data on the Internet. Also, obfs3 will not get through a HTTP proxy, as it is clearly not HTTP. So one option for the project is to impersonate HTTP. This is deceptively difficult because although HTTP is transmitted over TCP, the properties it offers to higher layers are not as strong as TCP (and not as required by Tor). For instance, individual HTTP requests may be re-ordered if they are over different TCP connections. Also, responses may be truncated without an error being reported to higher layers (which is why HTTP includes length fields as an option). HTTP doesn't give the same congestion avoidance as TCP and proxies can both cache and modify data they transmit. The HTTP specification is vague on some topics, and even when it specifies a particular behaviour, proxies frequently violate the specification. On the up-side of a HTTP proxy, HTTP is probably one of the last protocols a country will block before they turn off the Internet completely, so it has a good chance of getting through. Also, in some scenarios the only way for traffic to get out is via a HTTP proxy. So I think there is significant usefulness in this option. Also, it is incredibly difficult to hide one protocol inside a different one, because just recording the approximate number of bytes sent vs bytes received can give a good recognition of the protocol [4]. Hiding HTTP-over-Tor as BitTorrent traffic will likely be detectable based on such a statistic, but HTTP-over-Tor as HTTP at least has a chance. On the side of scanning resistance, we discussed the challenge of implementing scanning resistance with TCP. Here, the problem is that someone sending a SYN packet to a port which is open will receive a SYN-ACK, regardless of what user code does. To resist scanning (with something like BridgeSPA [1]) the pluggable transport would need to be quite tightly integrated with the OS rather than just using the standard socket API. Therefore it will create deployment difficulties, especially on Windows which has locked-down the raw sockets API. Therefore it might be interesting to send data over UDP rather than TCP, as then it is the responsibility of the user code to send the SYN-ACK-equivalent. Tor needs properties similar to TCP from it's pluggable transport, and so any UDP-pluggable transport would need something which replaces TCP: reliable in-order delivery with congestion management. One option here is libutp, as used by BitTorrent [2]. There is a vast amount of libutp traffic on the Internet, but it's timing and upstream/downstream characteristics will be different from how Tor would use it. Alternatively, it might not be worth worrying about this type of scanning resistance, and just focus on what it is possible to do with TCP, as done with ScrambleSuite [3]. Chang, George, do you have anything to add to this summary? Does anyone else on tor-dev have thoughts on these topics? Best wishes, Steven [1] http://www.cypherpunks.ca/~iang/pubs/bridgespa-wpes.pdf [2] https://github.com/bittorrent/libutp [3] http://www.cs.kau.se/philwint/pdf/scramblesuit2013.pdf [4] http://dedis.cs.yale.edu/2010/anon/papers/parrot.pdf

7 10

Searchable Tor descriptor archive - GSoC 2013 project
by Kostas Jakeliunas 29 May '13

29 May '13

Greetings! I'm a student who will be working on the Searchable Tor descriptor archive as part of Google Summer of Code. Yay! I've been following Tor development for a while and hope that this opportunity will be my way of sneaking into the development kitchen of Tor. In any case, I hope to stay around for a longer time to come. The original GSoC project proposal is based on one of the Tor project ideas available [1] and is part of the Tor Metrics project [2]. The GSoC proposal itself is also available to read [3] (TXT; if there's any interest, I can work on reformatting.) My primary mentor is Karsten and my secondary mentor is Damian. I will quote the abstract from the proposal to sum up the high-level goals of this project: I'd like to create a more integrated and powerful descriptor archival > search and browse system. (The current tools are very restrictive and the > experience disjointed.) To do this, I'll write an archival browsing > application wherein the results are interactive: they may act as further > search filters. Together with a search string input tool which will have > more filtering options, the application will provide a more cohesive > archival browse & search experience and will be a more efficient tool. > So as of now, we have an array of tools for inspecting, searching for and getting aggregate data about running relays. (For an overview, see the Tools page in the Metrics portal. [4]) These tools include relay search, consensus info, exit-by-IP search, and quite a few more; furthermore, two Onionoo [5] based applications/tools: Atlas and Compass. This project would proposes to: - implement a more powerful backend that would allow one to search for all available relays since mid-2007 (I should have clarified in the previous discussions, and Karsten already includes this bit; i.e., since v2 statuses became available [6]; I guess this can also be discussed). "More powerful" here means, first and foremost, "all (>= v2) archival data" (relay descriptors and consensuses at the very least), and furthermore (at least per the original proposal), involving more complex queries: we'd be looking into, I think, minimally, combined AND/OR filters referring to a wider range of data fields available in the archival data and the ability to specify multiple date ranges. Referring to consensus-related data while searching for relays and vice versa would also be possible. (The capabilities would therefore also include those of exoneraTor.) - implement backend results which would, as of current standing, aim for Onionoo compatibility (again see protocol design in [5]), or perhaps supersede it while providing backwards compatibility (e.g. returning paginated lists of consensus-status-entries where a specified relay was present.) - (as per original proposal,) implement a more powerful archival descriptor search & browse tool (frontend) which would provide a more uniform "looking up relays" / "searching by using many criteria" / "further refining search in the results page" experience - "refining search results", i.e. adjusting filters would be semantically the same as entering search criteria in the beginning; hence a more interactive experience, a more powerful search/browse tool. The goals and design of the project have to be clarified, however. There is ongoing discussion (see another tor-dev thread [7] e.g.) whether perhaps the focus could be to create a backend which would speak the full Onionoo protocol and therefore be a potential replacement not only for relay search and exoneraTor, but also for other components: all presently-speaking Onionoo applications could be made to talk to the new backend, for example. The overall count of components will hopefully be reduced in any case, but ideally, we would end up with a much more integrated Tor Metrics (and maybe beyond) ecosystem. Many open questions, however - see again [7]. Obviously discussions are very welcome indeed! I'm wfn on OFTC (#tor-dev, #nottor), also reachable via XMPP < phistopheles(a)jabber.org>, and am very much up for any kind of chat. :) I'll be busy with exams in the first three weeks of June, though - but will find time for sure! Regards Kostas. [1] https://www.torproject.org/getinvolved/volunteer#metricsSearch [2] https://metrics.torproject.org/ [3] http://kostas.mkj.lt/gsoc2013.txt [4] https://metrics.torproject.org/tools.html [5] https://onionoo.torproject.org/ [6] https://metrics.torproject.org/data.html#relaydesc [7] https://lists.torproject.org/pipermail/tor-dev/2013-May/004940.html

1 0

New control function
by grarpamp 28 May '13

28 May '13

I'm implementing a new control command to print some info requested from the network and/or internal state, similar to getinfo. Found the function that it will call and am making those changes. Is there a suggested entry to the control command parser for new commands? Or a dummy template that one should follow?

2 2

Damian's Status Report - May 2013
by Damian Johnson 27 May '13

27 May '13

Hi all. This is a little early but considering the time sink GSoC has been I doubt I'll get much more done, so here's my status report for May. The few stem tasks I've snuck in included... * Added Ubuntu and Fedora to our download page [1], Fedora package is thanks to Juan Orti * Several testing fixes so our Jenkins tests runs now pass [2] * The Controller mangled non-unicode descriptor content when using python 3.x (caught thanks to aj00200, #8755) * Expanded our client usage tutorial [3] to use SocksiPy and include an example for polling Twitter (thanks to Ashish) * Integ tests now assert ownership on the tor process (patch by Jeremy Kushner, #8634) * The DescriptorReader mishandled relative paths (patch from Kostas, #8815) * Fix so the Controller cache is thread safe (patch by Akshit, #8607) * Running our integ tests littered the /tmp directory (patch by Ashish, #8622) * Improvements so we use the staticmethod decorator and new style exception catching (patches from Sean, #8824 and #8823) Cheers! -Damian [1] https://stem.torproject.org/download.html [2] https://jenkins.torproject.org/job/stem-tor-ci/ [3] https://stem.torproject.org/tutorials/to_russia_with_love.html

1 0

Re: [tor-dev] "Trawling for Tor Hidden Services: Detection, [...]"
by Jon Smithe 25 May '13

25 May '13

Hi, > As for the deanonymization attack, I think it is pretty novel in that it > uses a custom traffic signature to make the attack from > http://freehaven.net/anonbib/cache/hs-attack06.pdf more reliable, but > otherwise that is why we introduced guard nodes. I'm still pretty sure this only makes people more secure in theory; the basis appears to be that instead of connecting to a random node every time a circuit is built, you instead connect to only one entry/guard node for 30-60 days. I think on paper that reduces the probability, but it seems to presume an attacker that doesn't modify their attack and doesn't appear to account for a keyspace of sorts that has reduced by over 50% and the introduction of a "touch me here" flag that says you're not only a relay, but you're guaranteed to be an entry point relay. Looking at the numbers from a snapshot a few days ago: 839 exits, 865 guards, 1676 generic relays. Previously for entry you'd be looking at a 'keyspace' of n/(839+865+1676), with the obvious problem that connecting to them at random every time you create a circuit is going to increase the probability that you hop through a compromised node. However, you need a fairly large value of N to make that realistic, I think the number cited was approaching 50%, but I could be remembering that incorrectly. Now, you're at 865 _guaranteed_ entry points, which automatically reduces the required value for N and increases the value of each rogue node that gains the guard flag, which appears to be strictly based on bandwidth and stability. I'm hardly an expert on math or probability, but it seems like the 'weight' so to speak of a compromised node was never increased to account for the reduced number of nodes or guarantee that being a guard will always yield entry point traffic, which is to say that N is actually going to be some value multiplied by N instead of just N/C^2, (N*nWeight/C or possibly even N^nWeight/C). It at least seems like the probabilities were only worked for the state the network was in, and not for the state/behavior of the network after the modifications. So now you have a network where one can setup two boxes that are fast and stable and be guaranteed to receive exit and entry traffic and only needing about half as many boxes to reach the same 50% mark as before. The math behind this concept is not overly compelling or I'm just dumb, both are probable and neither are mutually exclusive, but if I were looking for a state-based backdoor, I'd imagine it to look a bit like this (which is not to imply that is the case here by any means). Jon On Thu, May 23, 2013 at 11:35 PM, <tor-dev-request(a)lists.torproject.org> wrote: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Hidden services performance (Petter Solberg) > 2. Re: Hidden services performance (Jeroen Massar) > 3. Chrome browser and sand boxing. (Andrew F) > 4. Re: Chrome browser and sand boxing. (Andrew Lewman) > 5. Re: Chrome browser and sand boxing. (Andrew F) > 6. "Trawling for Tor Hidden Services: Detection, Measurement, > Deanonymization" (Tom Ritter) > 7. Re: "Trawling for Tor Hidden Services: Detection, > Measurement, Deanonymization" (Mike Perry) > 8. Re: "Trawling for Tor Hidden Services: Detection, > Measurement, Deanonymization" (Nick Mathewson) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 23 May 2013 14:46:12 +0200 > From: Petter Solberg <pettersolberg88(a)gmail.com> > To: tor-dev(a)lists.torproject.org > Subject: [tor-dev] Hidden services performance > Message-ID: > <CAPnamRP==01+HkQQWbg_mQv0uF=f3hK02jvffXDJBW8UfyMueQ(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hi. > > We are two master thesis students at the Norwegian University of Science > and Technology (NTNU) which is looking into the performance of hidden > services. We have already set up 19 physical Linux servers connected by > gigabit Ethernet to be able to benchmark hidden services in a private tor > network. We have also some public servers that we plan to utilize ( > https://atlas.torproject.org/#search/disasterTor ). This mail is to inform > you of our work and a request for ideas in where some bottlenecks are and > if someone have done some previous research before. > > Without any modifications to Tor we have been able to have a throughput of > 180Mb/s through a single hidden service. > > > If you have some Ideas or a comment. Please reply. > > > > Regards > > Petter Solberg (pettso AT stud.ntnu.no ) & Bram Bezem ( bezem AT > stud.ntnu.mo ) >

2 1

Chrome browser and sand boxing.
by Andrew F 23 May '13

23 May '13

I have Googled and searched the Tor website, but I have not found any information on chrome's sandbox feature and Tor. Specifically with flash. Is there any information out their on this? Thanks

2 2

Hidden services performance
by Petter Solberg 23 May '13

23 May '13

Hi. We are two master thesis students at the Norwegian University of Science and Technology (NTNU) which is looking into the performance of hidden services. We have already set up 19 physical Linux servers connected by gigabit Ethernet to be able to benchmark hidden services in a private tor network. We have also some public servers that we plan to utilize ( https://atlas.torproject.org/#search/disasterTor ). This mail is to inform you of our work and a request for ideas in where some bottlenecks are and if someone have done some previous research before. Without any modifications to Tor we have been able to have a throughput of 180Mb/s through a single hidden service. If you have some Ideas or a comment. Please reply. Regards Petter Solberg (pettso AT stud.ntnu.no ) & Bram Bezem ( bezem AT stud.ntnu.mo )

2 1

[Fwd: Welcome to the "tor-dev" mailing list]
by bones44x6013＠tormail.org 22 May '13

22 May '13

Hello again, Now that after several hours of work, we have come full circle back to the same email address I used for my original contacting you? I was not allowed to contact you through this address because I was not a registered user of this system? Now that I am a registered user of this system, I still have the same questions for you...Is this your most secure form of communication and is it safe for me to send you this information via a MS word document here through this email? This information should only be viewed by top Tor Developers and Management it is not meant for general discussion or a general discussion board viewing. This is the reason for my asking again about this again. I am providing you with a new technique for the secure communications of bridge address network locations. This method is proven and will work for you. Bones44 ---------------------------- Original Message ---------------------------- Subject: Welcome to the "tor-dev" mailing list From: tor-dev-request(a)lists.torproject.org Date: Wed, May 22, 2013 4:25 pm To: bones44x6013(a)tormail.org -------------------------------------------------------------------------- Welcome to the tor-dev(a)lists.torproject.org mailing list! To post to this list, send your email to: tor-dev(a)lists.torproject.org General information about the mailing list is at: https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: https://lists.torproject.org/cgi-bin/mailman/options/tor-dev/bones44x6013%4… You can also make such adjustments via email by sending a message to: tor-dev-request(a)lists.torproject.org with the word `help' in the subject or body (don't include the quotes), and you will get back a message with instructions. You must know your password to change your options (including changing the password, itself) or to unsubscribe. It is: zenuokna Normally, Mailman will remind you of your lists.torproject.org mailing list passwords once every month, although you can disable this if you prefer. This reminder will also include instructions on how to unsubscribe or change your account options. There is also a button on your options page that will email your current password to you.

2 1

Move RecommendedTBBVersions from check.torproject.org to www.torproject.org
by Micah Lee 22 May '13

22 May '13

I just opened a ticket for this: https://trac.torproject.org/projects/tor/ticket/8940 Right now the way to check for the current TBB version is by loading https://check.torproject.org/RecommendedTBBVersions. It would be really helpful if there were a way to check the current version of TBB from a URL at https://www.torproject.org/. The main website has a .onion address, and all of the Tor mirrors listed at https://www.torproject.org/getinvolved/mirrors.html.en are mirrors of the main site, not check. So if you want to only use the hidden service or only use a mirror (because *.torproject.org is blocked on your network), there's no way to easily know the current recommended version. -- Micah Lee @micahflee

1 0

building from source in a 64-bit windows environment..
by not me 20 May '13

20 May '13

Hi, It seems fairly self-evident that tor hasn't been build for 64-bit windows and questionable whether it's ever been built in an environment that doesn't utilize mingw. There are literally hundreds of thousands of warnings about integer truncation, at least some of which seem somewhat problematic (64-bit pointer being stored in a 32-bit integer (why?! why?!)), a small amount of signed/unsigned comparisons or sign extensions, and a bunch of calls to functions without including the proper header files or using the correct function name ('open()' vs '_open()' without including io.h) and so on. This of course is ignoring all the calls to str{,n}cpy/{v,}s{,n}printf/etc and posix function names long since deprecated (fileno() vs _fileno()) and so on. Errors relating to MSVC finally including a stdint header (cstdint) and typedef's for intptr_t not producing the right size primitive, etc. I realize that many/most/all of the developers here are going to see no problem with using mingw, but if you spend some time looking underneath the hood and reviewing the code generated by mingw v. msvc both in terms of security features and general performance, I can't feel that its a prudent or responsible choice to produce anything under windows using GNUs development tools. So I'd prefer to just eliminate all such 'fixes'. Moving to 32-bit isn't an option either, I'd have to switch out far too many dependencies that frankly are a bit more important, and well I need 64-bit for some aspects and it's only been a decade since AMD released their first 64-bit chip. So, has anyone out there successfully built tor with a 64-bit msvc, and if so, you don't happen to have a list of voodoo tricks you performed or a diff or similar? Thanks.

6 22

RFC patch: systemd socket activation
by Marti Raudsepp 20 May '13

20 May '13

Hi list, The attached patch implements support for systemd socket activation. For people who don't know what that is: systemd is an "init" system for Linux. Socket activation means that systemd binds all the sockets in advance, and only spawns Tor once somebody attempts to connect. More information here: http://0pointer.de/blog/projects/socket-activation.html I rarely use Tor, so there's no reason to have it running all the time (wasting battery on my laptop), but it's also annoying to launch it manually every time. Socket activation is ideal for this use case. There are 3 changes to the startup process: 1. Before loading the configuration, Tor identifies all sockets passed in by systemd and creates pending_socket_t objects. I considered reusing connection_t, but that seemed to require way more modification to the code. 2. After parsing configuration, when Tor would otherwise create new listeners, it first tries to match up the address/port to existing pending sockets. If a pending socket does not match, it opens a new one as usual. 3. After configuration parsing is done, Tor closes all remaining unmatched systemd sockets and logs a warning for each one. This infrastructure can also be used to support "launch-on-demand" with launchd on OS X, but I have no experience with that. Known problems: * TCP and UDP sockets work, but Unix sockets are not currently yet implemented. * It's impossible to support hibernation as is for systemd sockets -- the systemd daemon still keeps a reference to the listener socket even after we close it, and it's impossible to re-bind the port later. * Closing unmatched sockets is a bad idea for the same reason: systemd still keeps it open and connections hang forever. Perhaps a better solution is to keep the socket and simply reject all connections, ditto hibernating connections? I have added the source of sd-daemon.c into Tor -- it's easier to manage this way and we don't introduce any new library dependencies (it's 804 LoC total). This approach is also encouraged by systemd itself. The code turns into a no-op when built on Windows. Full patch is attached, also available as individual commits from my GitHub clone (branch "systemd"): https://github.com/intgr/tor/tree/systemd Regards, Marti

3 4

Re: [tor-dev] Questions pertaining to client to directory authority
by Jon Smithe 20 May '13

20 May '13

Hello, I receive a daily digest, so I am replying to everything at once. > http, https://gitweb.torproject.org/torspec.git/blob/HEAD:/dir-spec.txt > > section 1 line 184 "All directory information is uploaded and > downloaded with HTTP." > > If you search through the document for http, you will find most of the > uris and related info youre looking for. Well look at that, thank you. That's what I get for skimming a bit too much. > From: Roger Dingledine <arma(a)mit.edu> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Questions pertaining to client to directory > authority communications > Message-ID: <20130519212036.GK31806(a)moria.seul.org> > Content-Type: text/plain; charset=us-ascii [...] > Yeah -- I'm afraid there's a lot to learn at this point. Perhaps when > you figure out the bootstrapping process to your satisfaction, you'll > write up a summary and then we can correct it as needed and have a better > answer for the next person? As pointed out above, I missed some things while reading the specifications, I hadn't quite made it to the bottom of the 3rd version of the directory spec, which is where all the URIs are and is largely what I was looking for. Piecing together all of the URIs and their exact contexts through the code was proving to be daunting, although surprisingly I got most of what I was looking for correctly. Either way, assuming I feel like I understand things well enough to explain them at some point, I would think at least throwing together some type of flow diagram would be fairly painless and I imagine others would indeed benefit from it. For anyone reading this in the interim, I actually found starting with the v1 specification and working towards the v3 was beneficial. I imagine a lot is outdated, but it somehow managed to make me take note of other things I had missed in the v3 spec. > Each directory authority actually has two ports baked into Tor. E.g., > "moria1 orport=9101 no-v2 " > "v3ident=D586D18309DED4CD6D57C18FDB97EFA96D330566 " > "128.31.0.39:9131 9695 DFC3 5FFE B861 329B 9F1A B04C 4639 7020 CE31", > > has 9131 as its DirPort (answering naked http requests), and 9101 as > its ORPort (doing the Tor TLS handshake). Note that clients by default > connect to the ORPort and then tunnel their http request through it > (see the 'begindir' relay cell command), both for authentication and to > prevent simple DPI-based blocking. Aha, so I imagine with moria1 the issue is that I'm connecting directly to port 9131 instead of tunneling it through the ORPort. I just retested to make sure I hadn't lost my mind and indeed directly connecting to port 9131 eventually just returns a single byte ('\0'). This seems to be the case with turtles as well, however the other authorities that serve the dirport over the traditional HTTP port (80) seem to not have this authentication/DPI evasion behavior. > That's for asking v2 directory questions, not v3. > > We disabled it because of > https://trac.torproject.org/projects/tor/ticket/6783 I suspected that might be the case. Thanks for the answer. > Building what basically amounts to a botnet of old Tors, all clamoring > for obsolete directory information, has certainly proved a learning > experience. :) I can imagine you've had a few headaches over the lifetime of the project :) > Try the v3 protocol, not the v2 protocol. > > I just made > https://trac.torproject.org/projects/tor/ticket/8913 > > > (tor26 does have the no-v2 flag set which might be the issue?) > > (Hm? No it doesn't.) Indeed you are correct, I must have gotten the lines crossed while reading them somewhere and mixed it up with another of the authorities. Either way, despite my criss-cross, my hinting suspicion was correct: trying to speak v2 to v3 servers (off-hand I'm not positive which authority I was connected to when I tested/wrote that) > I think the specs do a pretty good job of explaining what is supported, > but as you say they don't specify how you should use the protocol. > > Some of that is in path-spec.txt. Indeed, I spoke too soon > But see also https://trac.torproject.org/projects/tor/ticket/7106 I will be mindful of this as I progress, it's quite evident by all of the various measurements tor takes that there is a lot of care in the behavior of the various components. . > > I have other questions about aspects of the protocol, but I will mostly > > save those until I understand the basic blocks of it better. But to > > exemplify somewhat, it does seem that the introduction of guard nodes would > > cause an inverse of desired effect; there appears to be about 1000-1100 > > guard nodes versus a several thousand relays, and about 800-900 exit nodes > > so it would seem that mitigating the attack where an attacker controlled C > > number of nodes is essentially pointless as one would only need to control > > a set number of guard and exit nodes and can more or less ignore the relays > > in between, so whereas you needed say C/N nodes previously, one would only > > need Cg/Ng (Cg controlled guards / Ng Number of guards). > > It seems you're leaving out Ce/Ne, and/or assuming that the adversary > controls/observes the destination also. Well I was more assuming that any adversary that could inject enough guard nodes would also be able to inject 'enough' exit nodes, as my understanding is that this is an actual flag they control directly. > I agree that the guard notion is counterintuitive, and also it's not > perfect, but I think it's way better than not doing it. I've not had a chance to read the links you provided, so I am still speaking from my relatively feeble comprehension, but essentially it seems like the guards more or less condense the number of entry points to the network? Whereas there are thousands of potential relays that could be used as an entry point, there is about a thousand guard nodes. So, and again I am probably not entirely comprehending something, but it seems that we've just made the requisite value for C smaller? (Again assuming that anyone who could get a large volume of relays of any kind could reasonably also control a large volume of exit nodes) > You might like http://freehaven.net/anonbib/#ccs07-doa > (though it doesn't come with analysis of the guard design, and there's > still some debate about how the guard design changes things). > You might also like Mike Perry's work on Path Bias detection. See > https://trac.torproject.org/projects/tor/ticket/5458 and also look > in the changelog for the string "Bias". Will do, thanks! > 1 guard is better than 3 guards in this respect. But both 1 and 3 are > way better than 0. While I agree with this notion, I guess what I am asking is: 'are 3 guard nodes better than X generic entry nodes for values of X that are significantly larger than 3' (pure example. obviously there is more than 3 guard nodes). As I'm understanding things, the concept was introduced to combat the problem discussed in path-spec.txt section 5 (C/N^2), but it seems like the introduction of guard nodes, because there are so much fewer of them than regular relays and because they are guaranteed to be an entry point into the network in essence just makes the value of N smaller and thus the number of C required to mount at least half of the attack smaller. More over it also removes the 'guess work' as intermediate relays between the guard and exit more or less become irrelevant? Rephrased, does converting the problem to C/N instead of C/N^2 still hold true if in the process we make the values of N and thus C significantly smaller, at what point does C/N^2 present a better probability? I'll read more, I am obviously missing something. Thanks a lot for the links and information. Jon

1 0

Questions pertaining to client to directory authority communications
by Jon Smithe 19 May '13

19 May '13

Hello! I have been reading through the various tor specifications trying to understand how this all works, so please forgive any ignorance of the protocol on my part. There seems to be a fair amount of gaps about specifically how various communications take place; for instance if we consider the very beginning of the communication chain, Directory Authorities, we have the dir-spec.txt file which outlines rather well what type of information can be retrieved from the directories, but not what communication protocol is actually being used. It appears that the usage of HTTP is fairly inherent, but this seems like it is only a partial answer as only some of the trusted authorities seem to speak HTTP on the address & port combination compiled into tor, moreover at least some of the authorities do not appear to implement the specification entirely. For instance, the trusted authority at MIT, 'morial' is listening on port 9131, however attempting to retrieve the network status document from it via a GET request to /tor/status/all.z results in no output at all. I would assume if this was a matter of needing to connect via SSL that I would receive an error due to a bad handshake, but I get nothing back. This holds true for at least one of the other trusted authorities listening on a non-HTTP related port (turtles). So for those servers, exactly what protocol is being used and is it documented anywhere other than the source code? Then, for instance, if we connect to 'tor26', which does respond to HTTP requests and attempt to retrieve a v2 network status document via the /tor/status/all.z URI, we receive a 404 although it appears the document that should exist there exists on other URIs, it's not entirely clear if this is just outdated code, specific to particular versions of the protocol (tor26 does have the no-v2 flag set which might be the issue?) or what exactly. So the question is, are there accurate specifications anywhere that focus not only on the semantics of cryptography and rationale behind certain choices but also the specifics of how exactly the protocol works or am I 'stuck' with reading the source code? I suspect that it is the later, so my question would be is there anyplace where the control flow is somewhat documented? (As the flow is somewhat disjointed at least in part to the way libevent works and other such aspects that make it difficult to parse if you're not familiar already with how everything is interconnected). I have other questions about aspects of the protocol, but I will mostly save those until I understand the basic blocks of it better. But to exemplify somewhat, it does seem that the introduction of guard nodes would cause an inverse of desired effect; there appears to be about 1000-1100 guard nodes versus a several thousand relays, and about 800-900 exit nodes so it would seem that mitigating the attack where an attacker controlled C number of nodes is essentially pointless as one would only need to control a set number of guard and exit nodes and can more or less ignore the relays in between, so whereas you needed say C/N nodes previously, one would only need Cg/Ng (Cg controlled guards / Ng Number of guards). If we then factor in that it seems possible that a guard or relay can essentially indirectly control the route a circuit takes through the network by continually causing cell extensions to fail for all relays and exit nodes that they do not control, then the value for Cg would seem to not need to be overly large or at least not approach the values of C or Cg (C/N or Cg/Ng). This seems incredibly reasonable for attackers that have state level resources (What is 1,000 computers to China? Iran? ...the United States?) and because the algorithm for selecting guards appears to be based entirely on stability and bandwidth; metrics we can expect a government to have plenty of on hand. I understand that rotation is supposed to ease this somewhat, but at least according to the academic paper out of Waterloo that Roger co-authored, it would appear that this actually facilitates the compromise of more clients than eases the problem, with the most secure (in the lab) situation being a single guard. (I understand that in practice this will not be realistic as it creates a bottle-neck in a variety of ways, e.g. firewalls and DDoS). I suspect many of the questions I have will be answered as I better familiarize myself with the protocol, so of the few I've enumerated, they can be thought of exemplifications that I expect will hash themselves out as I progress through the source and specifications. Thanks a lot! Jon

3 2

Your server has not managed to confirm that its ORPort is reachable
by Christian Kujau 19 May '13

19 May '13

Hi, I had to reboot my bridge for a (Ubuntu) kernel upgrade but now it cannot confirm that the ORPort is accessible: May 17 20:20:36.000 [notice] Tor 0.2.4.12-alpha (git-a1bb0df9be95ce7a) opening log file. May 17 20:20:36.000 [notice] Not disabling debugger attaching for unprivileged users. May 17 20:20:36.000 [notice] Your Tor server's identity key fingerprint is '...' May 17 20:20:36.000 [notice] Configured hibernation. This interval began at 2013-05-13 10:00:00; the scheduled wake-up time was 2013-05-13 10:00:00; we expect to exhaust our quota for this interval around 2013-05-20 10:00:00; the next interval begins at 2013-05-20 10:00:00 (all times local) May 17 20:20:36.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip. May 17 20:20:37.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6. May 17 20:20:37.000 [notice] Configured to measure statistics. Look for the *-stats files that will first be written to the data directory in 24 hours from now. May 17 20:20:40.000 [notice] We now have enough directory information to build circuits. May 17 20:20:40.000 [notice] Bootstrapped 80%: Connecting to the Tor network. May 17 20:20:41.000 [notice] Bootstrapped 85%: Finishing handshake with first hop. May 17 20:20:41.000 [notice] Bootstrapped 90%: Establishing a Tor circuit. May 17 20:20:42.000 [notice] Registered server transport 'obfs3' at '0.0.0.0:30001' May 17 20:20:42.000 [notice] Registered server transport 'obfs2' at '0.0.0.0:20001' May 17 20:20:43.000 [notice] Tor has successfully opened a circuit. Looks like client functionality is working. May 17 20:20:43.000 [notice] Bootstrapped 100%: Done. May 17 20:20:43.000 [notice] Guessed our IP address as ... (source: ...). May 17 20:40:43.000 [warn] Your server (...:9001) has not managed to confirm that its ORPort is reachable. Please check your firewalls, ports, address, /etc/hosts file, etc. I have not changed my tor configuration (honest! :-)) and Tor 0.2.4.12-alpha (from deb.torproject.org) was running fine before. This particular bridge is running inside an Amazon EC2 instance and I can reach port 9001 from the outside: $ nc -w1 -vnz xx.18.xx.xxx 9001 Connection to xx.18.xx.xxx 9001 port [tcp/*] succeeded! And I can see that request on the bridge when tcpdump'ing :9001, so it's not a network issue. I'm not sure what "/etc/hosts" should have to do with it, but I haven't modified this either. I'm strace'ing the tor process now to see what it's doing but couldn't find anything suspicious so far. Any thoughts? Christian. -- BOFH excuse #139: UBNC (user brain not connected)

2 3

Tor Browser source tarball not available
by Jamie Nguyen 15 May '13

15 May '13

Hi, the source at this link is missing: https://www.torproject.org/dist/torbrowser/tor-browser-2.3.25-8-src.tar.gz I'd be very grateful if someone could upload it please. Kind regards, -- Jamie Nguyen

1 0

Status of Torouter project
by Griffin Boyce 14 May '13

14 May '13

Hello all, So I'm part of a team working on wireless mesh, and Torouter has come up a few times this week. Is it actively being developed? Given the state of the roadmap [1], I'd sort of assumed it was inactive or on hiatus, but others had heard differently. thanks, Griffin Boyce -- Technical Program Associate, Open Technology Institute #Foucault / PGP: 0xAE792C97 / OTR: saint(a)jabber.ccc.de

4 5

BridgeDB - Bridge Distribution Modifications
by Matthew Finkel 14 May '13

14 May '13

Hi all, Over the last few weeks I've been working with George and Aaron on updating BridgeDB's code with respect to how it handles pluggable transports. I've made some decent progress, but there are some questions that I'd like to ask (because I'm not sure I should be the one making the decision). I've also started updating the spec and there are some parts on which I'd like some clarification. I'll try to summarize the thoughts on the matter we/I have thus far. See [A] if you're unfamiliar with the BridgeDB code/spec/idea. 1) How should BridgeDB decide the number of transports, and types, it should hand out? - My current patch returns transports based on the ratio of how many there are compared to the other bridges, so that if we hand out four bridges and obfs2 bridges account for 3/10 of all running bridges, then BridgeDB will hand out (4*(3/10)) = 1.2 bridges with each request, on average. - I've also added an option into bridgedb.conf to set the (expected) minimum and maximum number of bridges which support a specific PT that BridgeDB should hand out per request. - I have a verification check that tries to force us to meet these values, however, with its current implementation it's not guaranteed, only probabilistic. I think this is okay for now. - So, is this enough? Do we want/need a deterministic method of supplying bridges with a supported set of transports? - Another option is to place each transport into its own subring and select from each of the subrings to ensure we meet the requirement. The more I've thought about this, the more I think this defeats the purpose of constructing the rings, though. - Last (for now), if a bridge supports multiple PTs, should we return all of them to the user or randomly select one or select one with a bias? We agreed that we really shouldn't do the first because that would just accelerate the ability of a censor to block more bridges. The middle option works, but given that many bridges now support obfs2 and obfs3, is it a good idea to, again, probabilistically return each type (roughly) half the time? 2) Should we prefer to distribute PT bridges over regular bridges which have their ORPort on 443? - Right now returning ORPorts on 443 is the highest priority and transports are a secondary best-effort operation. 3) Unless I incorrectly understand the code, the bridges never rotate. The bridge interval is set to NoSchedule(), which means it returns a static time. Is there a reason for this? This is counter to the spec. Just wondering. :) (I had some other points I wanted to raise, but I'm blanking on them now. I think this is a good start, though.) Please also let me know and correct anything I may have gotten wrong. Thanks everyone, and thanks to George and Aaron for their help, as well. - Matt A. For those who don't know the details of the code, the simplified version is as follows: 1) All bridges send their bridge descriptors and misc information to the Bridge Authority. 2) Bridge Authority provides a network status file containing all known bridges described by their name, fingerprint, digest, time of publication, IP addr, ORPort, DirPort. Bridge Auth also provides a bridge descriptor file also specifying the bridges IP addr, ORPort, and fingerprint. Last, it supplies an extra-info file that contains all the extra info that the bridges provide - mainly their transports, in our case. 3) BridgeDB parses all of these files and associates the information to a single instance of a bridge. 4) BridgeDB assigns each running bridge to a distributor (website, email, etc) based on an hmac of the bridge's ID. Once assigned, the bridge is inserted into the distributors list of bridges. 5) BridgeDB then further organizes the bridges assigned to each distributor by moving them into rings and subrings. - A ring is simply a sorted list of an hmac of the bridges' ID which, when traversed, wraps around to the beginning if it ever reaches the end. - The hmac of the bridge's ID is used to retrieve the actual bridge instance from a hash, which is stored along side the ring. 6) Some distributors, such as https, are 'initialized' with a few rings based on filters. - https starts out with a ring containing all bridges assigned to it, a ring only containing bridges which support IPv4 connections, and a ring only containing bridges which support IPv6 connections. - Every ring also contains two subrings (currently). One subring is the subset of bridges from the parent ring which have their ORPort listening on port 443. The other subring is the subset of bridges from the parent ring which have the stable flag set. - For example, - Cluster 1 Ring - subring (stable) - subring (https) - Cluster 2 Ring - subring (stable) - subring (https) - IPv4 Cluster 1 Ring - subring (stable) - subring (https) - IPv4 Cluster 2 Ring - subring (stable) - subring (https) - IPv6 Cluster 1 Ring - subring (stable) - subring (https) - IPv6 Cluster 2 Ring - subring (stable) - subring (https) 7) When BridgeDB receives a request for bridges from its website, it forwards the query on to the IP distributor. The details will include if a specific PT was requested, IP version bridge supports, country within which the bridge should not be blocked, requesing IP address, and interval. 8) The distributor then decides on the "area" of the IP address, currently the /24 mask, and then finds the "cluster" within that area (by taking the first eight bytes of an hmac of the area and using the result (modulus "the number of clusters")). A filter is then constructed based on the requested information. If a ring already exists that satisfies exactly these filters then that is then constructed based on the requested information. If a ring already exists that satisfies exactly these filters then that is used. Else a new ring (with subrings) is constructed to satisfy this request. The distributor also computes the position in the ring as the hmac of the interval and the area. 9) Once the correct ring exists, it determines how many bridges it can find in the ring's subrings to satisfy the request. This is done by taking the previously computed position and finding it in the list of bridges ID's hmacs and then selecting the next consecutive "requested number of bridges" from the list (wrapping around to the beginning, if necessary). The same is then done for the main ring. The results from these searchs are then joined and the first "requested number of bridges" unique keys are selected from the list. This list is then sorted and returned, propagating back up to the user. 10) Similar actions are taken by the other distributors. For example, the email distributor doesn't use an "area" to decide which bridges to distribute, it uses the normalized requesting/source mail address. 11) Misc: - Because the rings are sorted by an hmac of the bridge's ID, I expect that they will be uniformly distributed around the ring. As such, I don't expect there to be a bias for one type of bridge/transport/ORPort over any other. (Is this incorrect?)

2 2

txtorcon 0.8.0
by meejah 14 May '13

14 May '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 txtorcon 0.8.0 is now tagged and released. [1] txtorcon is a Python+Twisted-based asynchronous implementation of the Tor control protocol. It includes a state-tracking abstraction, configuration abstraction, Twisted endpoint support for hidden services, 96%+ unit-test coverage and many examples. New in this release: . issue #18: slight *API change* ICircuitListener.circuit_failed, circuit_closed, IStreamListener.stream_failed, stream_closed and stream_detach all now include any keywords in the notification method (these either lacked flags, or only included some) . launch_tor() can take a timeout (starting with a patch from hellais); . cleanup from aagbsn; . more test coverage; . run tests cleanly without graphviz (from lukaslueg); . issue #26: fix from lukaslueg; . pep8 and whitespace targets plus massive cleanup (now pep8 clean, from lukaslueg); . issue #30: fix reported by webmeister making ipaddr actually-optional; . example using synchronous web server (built-in SimpleHTTPServer) with txtorcon (from lukaslueg); . TorState can now create circuits without an explicit path; . passwords for non-cookie authenticated sessions now use a password callback (that may return a Deferred); . fixes for AddrMap in case Tor ticket #8596 is implemented; Code and built documentation may be obtained: git clone git://github.com/meejah/txtorcon.git https://github.com/meejah/txtorcon https://txtorcon.readthedocs.org/en/latest/index.html You may easily install it: pip install txtorcon easy_install txtorcon The sha256sum output is: 87fcb3bce3ac3b48583f637bf69445bd50071411dc3fded89df7600d870a3094 txtorcon-0.8.0.tar.gz thanks, meejah [1] -- (Note the tagging took place last month, and I've only now updated PyPI, ReadTheDocs, freecode, emailed this, ...) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJRjy+yAAoJEJ0UOBRRgEVQXrMH/0Kl3DmVLyqo3aTBNv1d50RL yi8u5LTZLNtuYiyJsxqTFyq8zNhfEwIPoXbsqHyNuv0hv8s6Qn4OkXh47/crhukO GpnNNTldvSQIpOfNw+ftCYF/NC0X5pOjJo/Rz1XUkFyms50cqq17i+MWvcAvaFnW Ols6+7ulh8u61ElafVxnwn1zgitkpUOOsnF8a27iXryFJ/iZFFROnCigJv2eM8pj SqcwKbVp9h1VtztoYhtEcjEPBm1NnrYGoRIjh9D3RQfEVnXgM2fU4HoGhFAqYj7M wFZv0DZBvL2s/XwV8PEi4Y4+fTJZTXE7bbHctMWjUWjhlMfMniq2oud4hZJ5ugY= =MDAq -----END PGP SIGNATURE-----

1 1

Tor Launcher UI feedback follow up
by Mark Smith 13 May '13

13 May '13

Thank you to everyone who provided feedback on the Tor Launcher UI. It has been very helpful to us and we made a lot of changes based on it. The most significant change was the addition of an initial question to the "first run" settings wizard, which allows people to skip all of the detailed questions and quickly connect. Take a look here: http://trial.pearlcrescent.com/tor/torlauncher/2013-05-08/SetupWizard/scree… As Mike pointed out, we are trying to get to alpha ASAP so we can deliver much smaller TBB packages – without Vidalia. For that reason, some of the improvements that people suggested will be left out for now (e.g., automated probing for proxy or firewall settings). -- Mark Smith Pearl Crescent, LLC http://pearlcrescent.com/

9 12

Local onionoo cache
by David Fifield 13 May '13

13 May '13

Karsten, could I ask you to take a quick look at this code? https://trac.torproject.org/projects/tor/ticket/7549#comment:14 https://trac.torproject.org/projects/tor/attachment/ticket/7549/onionoo-que… It's a daemon that keeps a local cache of potential exit relays, sourced from onionoo. It listens on a local port and sends "EXIT" when asked about an IP address that is possibly an exit. The idea behind this code was to have a fast local database that we can rapidly query from the flash proxy facilitator, in order to prevent Tor users from being flash proxies themselves. Is this code the most straightforward way you can think to achieve the goal? Do you have any suggestions on the use of onionoo? I wonder if something like the daemon exists already. The code looks reasonable, though I would make some changes before merging it. I want to gauge whether detection of exits is worth the additional code. David Fifield

4 6

Tor Censorship Detector: Can I help?
by Sam Burnett 13 May '13

13 May '13

Hi, I'd like to help improve the Tor Censorship Detector. I've read some background material and think I understand the basics of George Danezis' detection algorithm [1, 2]. Is anyone still working on this? Two tickets from a year ago talk about experimenting with various detection algorithms and turning one of them into a standalone utility [3, 4]. Has anything happened since then? My background: I'm a graduate student at Georgia Tech studying network censorship circumvention and measurement. Although I've met Tor developers on various occasions, I haven't directly contributed to the project; I'd like to change that. Thanks! Sam [1] https://lists.torproject.org/pipermail/tor-dev/2011-September/002923.html [2] https://metrics.torproject.org/papers/detector-2011-08-11.pdf [3] https://trac.torproject.org/projects/tor/ticket/3718 [4] https://trac.torproject.org/projects/tor/ticket/4180

3 3

Re: [tor-dev] Iran
by monet＠tormail.org 10 May '13

10 May '13

Is any IP6 in Iran? IP6 is blocked too?

1 0