- tor-dev - lists.torproject.org

Tenative schedule for Tor 0.2.4 [Or, why I might reject your patches in November->March]
by Nick Mathewson 07 Sep '12

07 Sep '12

Hi, all! Last year, I announced a tenative schedule for 0.2.3.x. We didn't stick to it, and things have gone a little pear-shaped with getting 0.2.3.x stabilized, but I think with a few tweaks we can use something similar to get a good schedule out for 0.2.4.x. My goals remain about what they were before: get release out faster by getting better at saying "no" to features after a release window. My target is March or April 2013. To that end: October 10, 2012: Big feature proposal checkpoint. Any large complicated feature which requires a design proposal must have its first design proposal draft by this date. November 10, 2012: Big feature checkpoint. If I don't know about a large feature by this date, it might have to wait. November 10, 2012: Big feature proposal freeze. Any small feature which requires a design proposal must have its design proposal finished by this date. December 10, 2012: Big feature merge freeze. No big features will be merged after this date. December 10, 2012: Small feature proposal freeze. Any small feature which requires a design proposal must have its design proposal finished by this date. January 10, 2013: Feature merge freeze. No features after this date. I mean it. Feb 20, 2013: Buggy feature backout date. Any feature which seems intractably buggy by this date may be disabled, deprecated, or removed until the next release. On the meaning of "feature": I'm probably going to argue that some things that you think are bugfixes are features. I'm probably going to argue that your security bugfix is actually a security feature. I'm probably even going to argue that most non-regression bugfixes are features. Let's try to get a release out *early* in 2013 this time, come heck or high water. (This is all subject to change, but let's not push it.) [0] https://lists.torproject.org/pipermail/tor-dev/2011-July/002851.html happy hacking, -- Nick

4 4

Parallel release series for directory authorities?
by Nick Mathewson 06 Sep '12

06 Sep '12

Hi, all! Here's an idea I had for directory authorities and the 0.2.3 release series. "As you know Bob," Tor 0.2.3 will be stable very soon, and I'm hoping not to take any more patches for it except for important security issues. I want 0.2.4 to come out very early next year. But in the meantime, to support 0.2.4, and to deploy security/reliability features to the network early, there are some directory consensus features that we'd want to deploy before the rest of 0.2.4 is done and stable. In particular, I'd like directories to be able to reach consensus on IPv6 addresses (6363), and I'd like to try out the code that Mike wrote to make the bandwidth voting system more robust against dishonest during its startup phase (2286). So here's my plan: Make a "maint-0.2.3-da" branch based on 0.2.3, plus feature patches for directory authorities. It gets patches from the stable series (0.2.3), plus directory-authority features under development. It would get merged-forward into 0.2.4, to make sure that 0.2.3-da and 0.2.4 behavior stay in sync. This way we can get new authority features working in time to support 0.2.4 clients, but without forcing too many authorities to run 0.2.4 before it's ready, and without forcing us to make feature changes in 0.2.3 proper. Thoughts? yrs, -- Nick

7 8

(FWD) Re: known attacks on Tor
by Roger Dingledine 05 Sep '12

05 Sep '12

Hi folks, Here's an email I wrote to a researcher who is working on categorizing anonymity attacks. I figured I should share it with you in case it's useful in some way. It's also related to my talk at https://www.cosic.esat.kuleuven.be/ecrypt/provpriv2012/program.html and I expect to use it as background for my discussions at the upcoming Dagstuhl: http://www.dagstuhl.de/no_cache/en/program/calendar/semhp/?semnr=12381 --Roger ----- Forwarded message from Roger Dingledine <arma(a)mit.edu> ----- > If you have any suggestions about which paper on each attack is most > likely to provide such an explanation, please send them to me as soon > as possible. > > > - "Traffic confirmation attack". If he can see/measure the traffic flow > > between the user and the Tor network, and also the traffic flow between > > the Tor network and the destination, he can realize that the two flows > > correspond to the same circuit: > > http://freehaven.net/anonbib/#SS03 > > http://freehaven.net/anonbib/#timing-fc2004 > > http://freehaven.net/anonbib/#danezis:pet2004 > > http://freehaven.net/anonbib/#ShWa-Timing06 > > http://freehaven.net/anonbib/#murdoch-pet2007 > > http://freehaven.net/anonbib/#ccs2008:wang > > http://freehaven.net/anonbib/#active-pet2010 It depends in what way you want to become more precise. I think the #SS03 paper might have the simplest version of the attack ("count up the number of packets you see on each end"). The #timing-fc2004 paper introduces the notion of a sliding window of counts on each side. The #murdoch-pet2007 one looks at how much statistical similarity you can notice between the flows when you are only sampling a small fraction of packets on each side. > > - "Congestion attack". An adversary can send traffic through nodes or > > links in the network, then try to detect whether the user's traffic > > flow slows down: > > http://freehaven.net/anonbib/#torta05 > > http://freehaven.net/anonbib/#torspinISC08 > > http://freehaven.net/anonbib/#congestion-longpaths Section 2 and the first part of Section 3 in #congestion-longpaths is probably your best bet here. It actually provides a good pretty overview of related work including the passive correlation attacks above. If by 'more precise' you mean you want to know exactly what the threat model is for this attack, I'm afraid it varies by paper. In #torta05 they assume the adversary runs the website, and when the target user starts to fetch a large file, they congest (DoS) relays one at a time until they see the download slow down. In #congestion-longpaths they assume the adversary runs the exit relay as well, so they know the middle relay, and the only question is which relay is the guard (first) relay. In #torspinISC08 on the other hand, they preemptively try to DoS the whole network except the malicious relays, so the target user will end up using malicious relays for her circuit. > > - "Latency or throughput fingerprinting". While congestion attacks > > by themselves typically just learn what relays the user picked (but > > don't break anonymity as defined above), they can be combined with > > other attacks: > > http://freehaven.net/anonbib/#tissec-latency-leak > > http://freehaven.net/anonbib/#ccs2011-stealthy > > http://freehaven.net/anonbib/#tcp-tor-pets12 These are three separate attacks. In #tissec-latency-leak, they assume the above congestion attacks work great to identify Alice's path, and then the attacker builds a parallel circuit using the same path, finds out the latency from them to the (adversary-controlled) website that Alice went to, and then subtracts out to find the latency between Alice and the first hop. #ccs2011-stealthy actually proposes a variety of variations on these attacks. They show that if Alice uses two streams on the same circuit, the two websites she visits can use throughput fingerprinting to realize they're the same circuit. They also show that by looking at the throughput Alice gets from her circuit, you can rule out a lot of relays that wouldn't have been able to provide that throughput at that time. And finally, they show that if you build test circuits through the network and then compare the throughput your test circuit gets with the throughput Alice gets, you can guess whether your circuit shares a bottleneck relay with Alice's circuit. Where "show" should probably be in quotes, since it probably works sometimes and not other times, and nobody has explored how robust the attack is. #tcp-tor-pets12 has the adversary watching Alice's local network, and wanting to know whether she visited a certain website. The adversary exploits vulnerabilities in TCP's window design to spoof RST packets between every exit relay and the website in question. If they do it right, the connection between the exit relay and the website cuts its TCP congestion window in response, leading to a drop in throughput on the flow between the Tor network and Alice. In theory. It also works in the lab, sometimes. I also left out http://freehaven.net/anonbib/date.html#esorics10-bandwidth which uses a novel remote bandwidth estimation algorithm to try to estimate whether various physical Internet links have less bandwidth when Alice is fetching her file. In theory this lets them walk back towards Alice, one traceroute-style hop at a time. In practice they need an Internet routing map (these are notoriously messy for the same reasons the Decoy Routing people are realizing), and also Alice's flows have to be quite high throughput for a long time. > > - "Website fingerprinting". If the adversary can watch the user's > > connection into the Tor network, and also has a database of traces of > > what the user looks like while visiting each of a variety of pages, > > and the user's destination page is in the database, then in some cases > > the attacker can guess the page she's going to: > > http://freehaven.net/anonbib/#hintz02 > > http://freehaven.net/anonbib/#TrafHTTP > > http://freehaven.net/anonbib/#pet05-bissias > > http://freehaven.net/anonbib/#Liberatore:2006 > > http://freehaven.net/anonbib/#ccsw09-fingerprinting > > http://freehaven.net/anonbib/#wpes11-panchenko > > http://freehaven.net/anonbib/#oakland2012-peekaboo #oakland2012-peekaboo aims to be a survey paper for the topic, so it's probably the right one to look at first. > > - "Correlating bridge availability with client activity." > > http://freehaven.net/anonbib/#wpes09-bridge-attack If you run a relay and also use it as a client, the fact that the adversary can route traffic through you lets him learn about your client activity. Section 1.1 summarizes: 2. A bridge always accepts connections when its operator is using Tor. Because of this, an attacker can compile a list of times when a given operator was either possibly or certainly not using Tor, by repeatedly attempting to connect to the bridge. This list can be used to eliminate bridge operators as candidates for the originator of a series of connections exiting Tor. We demonstrate empirically that typically, a small set of linkable connections is sufficient to eliminate all but a few bridges as likely originators. 3. Traffic to and from clients connected to a bridge interferes with traffic to and from a bridge operator. We demonstrate empirically that this makes it possible to test via a circuit-clogging attack [17, 15] which of a small number of bridge operators is connecting to a malicious server over Tor. Combined with the previous two observations, this means that any bridge operator that connects several times, via Tor, to a web-site that can link users across visits could be identified by the site's operator. > > I tried to keep this list of "excepts" as small as possible so it's not > > overwhelming, but I think the odds are very high that if the ratpac comes > > up with other issues, I'll be able to point to papers on anonbib that > > discuss these issues too. For example, these two papers are interesting: > > http://freehaven.net/anonbib/#ccs07-doa Traditionally, we calculate the risk that Alice's circuit is controlled by the adversary as the chance that she chooses a bad first hop and a bad last hop. They're assumed to be independent. But if an adversary's relay is chosen anywhere in the circuit yet he *doesn't* have both the first and last hop, he should tear down the circuit, forcing Alice to make a new one and roll the dice again. Longer path lengths (once thought to make the circuit safer) *increase* vulnerability to this attack. I think the guard node design helps here, but whether that's true is an area of active research. > > http://freehaven.net/anonbib/#bauer:wpes2007 If you lie about your bandwidth, you can get more traffic than you "should" get based on bandwidth investment. In theory we've solved this by doing active bandwidth measurement: https://blog.torproject.org/blog/torflow-node-capacity-integrity-and-reliab… but in practice it's not fully solved: https://trac.torproject.org/projects/tor/ticket/2286 --Roger ----- End forwarded message -----

1 0

Tor bufferbloat
by David Goulet 05 Sep '12

05 Sep '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi everyone, I discovered, during a presentation at this year LinuxCon in San Diego, the problem of "bufferbloat" which is encountered in network bottlenecks. https://en.wikipedia.org/wiki/Bufferbloat http://www.bufferbloat.net/ There is been a good amount of work done so far to fix this problem ending up with the CoDel algorithm (http://queue.acm.org/detail.cfm?id=2209336) The Linux kernel is about to accept this policy as default. Anyhow, I looked around for some recent studies on the Tor network about such congestion issues but I can't find any information on if this phenomenon is observed on the Tor network. Someone knows about that? Thanks a lot! David -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJQR59ZAAoJEELoaioR9I02nkkH/0NYYnaca6N5pbDDbu96klff BGVmxcO8TEbbdI41HBWaJJHyUSi1CDbfZXcK6eaqRgDhj/yGcTN4YKgU/fP7LQFw /rp+MnWxPt7uqfrso+jSgvpDX6FS8r9cXwG6aMd9V39CuLRoVHYnNAMQrwHMyXud bMk0cHe71FGcyW+BGxZwrbjtvP3gcIZH9r/LNGaJepRzn+vB0NcwVpUG08D50Nxe M/3Lhokrc/IqaFi0+FX5fUoc8TALsjgjPZ672DdsY1CMwxbPMxpBKadJ9TWA906U QNxZb/lOa/fSWrnjY3nDaFJNppmyqYgoC1iJNJiCiJC8Jbxgj0/gyQjhY1d11Qc= =yFyg -----END PGP SIGNATURE-----

2 1

Damian's Status Report - August 2012
by Damian Johnson 04 Sep '12

04 Sep '12

Hi all. I spent most of August, like the prior month, traveling. This time I attended Toorcamp [1] and went on vacation. Both were pleasant and relaxing, but not terribly conducive to coding so I don't have much to report... * Descriptor CSV Export Functionality Naif proposed export functionality for Tor server descriptors a while back [1], which Eric and Megan took the first stab at [2]. I ended up revising this quite a bit [4], but it turned out nicely. * Caching Expansion and Test Prompt Added caching for GETCONF and static GETINFO queries. I also added a handy little script for a debugging interpretor. It kicks off Tor, then provides an interactive python prompt with a controller instance to it (optionally shutting Tor down afterward). Ravi code reviewed these changes [5] and volunteered to do code reviews of my future work as well. * Consensus Parsing Over the last few weeks Ravi's been working on a patch to parse network status documents [6]. It's functional, but missing unit tests and deviates from the parsing style and strict validation done for the other descriptor types so I'm taking a turn with the code. Thus far I'm done revising and adding tests for router status entries, and now working on the document. Changes are available in the 'document-parsing' branch of my repo [7]. Parsing these documents is a far larger task than I thought (especially if you include v2 documents and microdescriptors), so working on this branch will probably keep me occupied for much of September. * Controller Expansion Ravi has gone on a hacking binge, adding support for USEFEATURE [8], SIGNAL [9], EXTENDCIRCUIT [10], and SETCIRCUITPURPOSE [11]. Cheers! -Damian [1] http://www.toorcamp.org/ [2] https://trac.torproject.org/6171 [3] https://trac.torproject.org/6512 [4] https://gitweb.torproject.org/stem.git/commitdiff/7022021c4207a0066cb932616… [5] https://trac.torproject.org/6631 [6] https://trac.torproject.org/6569 [7] https://gitweb.torproject.org/user/atagar/stem.git/shortlog/refs/heads/docu… [8] https://trac.torproject.org/6417 [9] https://trac.torproject.org/6659 [10] https://trac.torproject.org/6666 [11] https://trac.torproject.org/6670

1 0

Tor on IPv6 roadmap and status page
by Linus Nordberg 02 Sep '12

02 Sep '12

Hi, The roadmaps/Tor/IPv6 [1] wiki page has been started with the goal of communicating the status of the work with Tor on IPv6. Additionally, searching for Trac tickets with the keyword "ipv6" [2] might give a picture of what's going on. Asking here or highlighting ln5 on #tor-dev is of course another perfectly fine option. [1] https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/IPv6 [2] https://trac.torproject.org/projects/tor/query?keywords=~ipv6 -- Linus

4 5

Linus' August 2012 status report
by Linus Nordberg 01 Sep '12

01 Sep '12

Hi, What happened in August. Got most of #4564 working and parts of it merged to master, including - Directory authorities vote for IPv6 public relays (micro descriptors not yet merged) - Relays bind to and publish IPv6 addresses - Clients use IPv6 relays in consensus (not using micro descriptors) Created an IPv6 status and road map page roadmaps/Tor/IPv6 [1]. Tickets worked on include 4455, 4620, 5535, 6362, 6363, 6364, 6423, 6514, 6621. Made an interview for the Swedish paper Ny Teknik. What's happening in September. September is the month of getting Tor on IPv6 deployed! Primarily the client to non-private bridges parts and hopefully also client to relays. See above mentioned IPv6 road map page [1] for details. [1] https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/IPv6 -- Linus

1 0

Karsten's status report August 1--31
by Karsten Loesing 01 Sep '12

01 Sep '12

Hi everyone, August was a long month for me, and I feel like I spent more time reviewing other people's code than writing my own. That's really cool! We have a bunch of new metrics-related code now, and I hope there will be even more code contributions for me to review next month! So, below is what I did in August. Compared to last month's report, I left out how much of my developer time I spent on these tasks and added more links instead. Best, Karsten Added three new graphs to the metrics website: the first two graphs show our progress in operating 125 fast exits [1] (first half of #6498), which is a deliverable for sponsor J [2]; the third graph shows advertised bandwidth and bandwidth history by relay flags [3] (#6671). Also fixed problems with cutting off enough days from graphs for which we don't have enough data yet (#6593) and removed GetTor graphs and related code to reduce future maintenance effort (#6395). The gentle reader may have noticed that there was no mention of code reviewing in the paragraph above, but just own code writing. That's correct, the metrics website is still a one-man show. I'm planning to split up the metrics website beast into a component that produces JSON-formatted graph data [4] and ask other people to write components visualizing the data, like Walter Kim's metrics website prototype [5]. Hopefully there will be more code contributions if people don't have to mess with Java servlets in the current metrics website code. Reviewed Sathya's great contributions to what we're now calling Compass [6]. Compass is a website and command-line tool to filter the set of currently running relays and group by same country or same AS. It was originally written by delber to analyze Tor network diversity. Sathya improved Compass by adding options to show only fast exits and almost fast exits for the sponsor J project to add 125 fast exits [2], and he wrote the website part. Since Compass went live, we had a lot of tickets opened by cypherpunks that we're almost unable to handle [7]. Patches welcome, Sathya and I will review them! Improved graphs in Atlas [8] to show the displayed quantity on the y axis (#5388) and added graphs on various probabilities to be selected by clients (#5455). Arturo reviewed and cleaned up these patches. Speaking of code contributions, there are quite a few open tickets [9] that would benefit from JavaScript developers wanting to help out! Reviewed Pyonionoo [10], the Wesleyan students' Python rewrite of Onionoo, the data provider behind Compass and Atlas. More precisely, they were working on the front-end portion of Onionoo which gives out JSON-formatted data, not on the number-crunching back-end that Sathya is about to rewrite in Python (#6452). Basic functionality of Pyonionoo works, but there are a few defects (#6707, #6708) and one enhancement (#6709) that we'll have to work on before deploying it. Of course, we have more open tickets for the Java and/or Python Onionoo [11]. Converted even more old tech reports into the new Tor Tech Report format [12] (#5405). The goal here is to have a single place for all reports we produce and that other researchers can use to cite our work. Set up Rob's Tor simulator Shadow [13] to see if there are any usability issues that would prevent researchers from using it. Turns out it's not just a very powerful tool to evaluate changes to the Tor code, but also doesn't require its users to be rocket scientists. I'm planning to play more with Shadow in September and walk through the instructions in Shadow's wiki [14] to find any remaining pitfalls. And then I'll encourage Roger and Rob to blog about Shadow. [1] https://metrics.torproject.org/fast-exits.html [2] https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorJ [3] https://metrics.torproject.org/network.html#bandwidth-flags [4] https://metrics.torproject.org/graphs/relays-by-country [5] http://tigerpa.ws/tor_metrics/ [6] https://compass.torproject.org/ [7] https://trac.torproject.org/projects/tor/query?component=Compass&status=!cl… [8] https://atlas.torproject.org/#details/D223399907113A1F216AAA64997BC1D4CFA8E… [9] https://trac.torproject.org/projects/tor/query?component=Atlas&status=!clos… [10] https://gitweb.torproject.org/pyonionoo.git [11] https://trac.torproject.org/projects/tor/query?component=Onionoo&status=!cl… [12] https://research.torproject.org/techreports.html [13] http://shadow.cs.umn.edu/ [14] https://github.com/shadow/shadow/wiki

1 0

Nick's August 2012 status report
by Nick Mathewson 01 Sep '12

01 Sep '12

THINGS I DID IN AUGUST I spent a few days vacationing with my family in Maine; took a day off for my birthday, and generally chilled out. I merged lots of bugfixes and patches -- some mine, some other people's, some requiring rewriting and review, some not. This included some security-related stuff that lost me a little sleep. It also involved hacking on some neat patches from Stewart Smith (with help from Jim Meyering) to revamp our build system. We put out a new 0.2.2. I fixed the more important (fixable) bugs in 0.2.3 (notably 6507 and 6244 and 6404), and pushed back on some others. IMO we're now ready to release the (probably) last 0.2.3.x-rc. I merged more patches into 0.2.4; see the new changes files there for more info. Lots of these were ipv6-related stuff from Linus. I started a draft for BEGIN2 cells so I can try to get a more future-proof IPv6 protocol implementation. This is blocked while I teach myself how DNSSEC works once and for all. I tried to get a little more testing on alternatives to our current (bogus, deteriorating) geoip supplier (see #6266) but didn't get anything beyond my older inconclusive results. I started working on an implementation for proposal 200, and a revised 202. With help from some libevent users, I fixed an SSL bug that might have been one of the (probably several) things making SSL bufferevents unstable. The fix appears in libevent 2.0.20-stable. I agreed to fly out to SF to accept the EFF's pioneer awards on Tor's behalf. I had an anniversary and a birthday. I proposed a crazy idea about a separate 0.2.3.x branch for DAs. I chatted a little about metaformats, and spent a while catching up on the ones I didn't know the details of. Currently I'm thinking YAML is a hot mess, and ProtocolBuffers is not too bad actually. There was this insane stegotorus bug that I sort of tracked down. That is, I can tell you which libevent commit to revert in order to make the problem go away (that is, 3467f2fa3bbff0dab6865), but I can't for the life of me figure out what's wrong with it. My current guess is that it provokes an issue somewhere else. A couple of friends/acquaintances approached me about working for Tor as a project manager. I told them to email jobs(a)torproject.org. I don't think we've heard from them yet. It would rock if we had a faster release schedule. I've been trying to find projects our size that do it right. This coming month I must help Linus get our September IPv6 deliverables deployed. I need to help Andrea get her refactored channel abstraction merged, and make sure she's got more stuff to do (and more interesting stuff to do!) as she transitions to full-time. I must get that blog post I'm cowriting with Steven finished, and our paper for November started. And I need to write some of my own code too. I also need to finalize a plan and a schedule for 0.2.4 and send them to the world. "when in danger or in doubt," -- Nick

1 0

[GSoC] [Stegotorus] Recent activities
by vmon 31 Aug '12

31 Aug '12

Hey Zack, I know you are super busy packing for your big move, but I thought as tomorrow is the hard deadline for GSoC, is not a bad ideas to give you some final update. So most of last week I spent on the conflict that occurred between libcurl and libevent having their eyes on the same socket. I tried different ugly and uglier solutions but finally settled on this one which is not that bad: http://stackoverflow.com/questions/12021217/how-to-ask-libcurl-not-to-liste… After that, I'm passing all time line tests now. However, when I look at the client log, I see that some of the sequence no are missing in the recv: like this 203.7169 [debug] recv: <3.81> receiving block 42 <d=0 p=9 f=DAT> T:203.7170: ckt 3 <ntp 1 outq 0>: recv 42 <d=0 p=9 f=DAT> and the next recv is like this: 203.7926 [debug] recv: <3.99> receiving block 60 <d=0 p=1 f=DAT> T:203.7927: ckt 3 <ntp 1 outq 0>: recv 60 <d=0 p=1 f=DAT> Is this for sure an error and loss of data? or there is a normal situation that such a thing can happen? For example when the whole block can't be sent over one connection? (otherwise how do the tl tests all pass?). Further more, I merged the payload scraper and stegotorus. I this way, http_apache checks for the payload database and if it's not there it calls the scraper (it needs apache to be installed on the same machine). These all are commited on the github. I started writing a unit test that fetch a webpage directly (using curl) and through stegotorus and check if they are the same. I thought curl is best approximation of a browser that is available to us. But as I don't think I can finish that till tomorrow, I'm going to spend the remaining time on documenting the 'uri transport protocol' and deal with small issues. After we are done with the evaluation, I'll go back to unit testing. Thanks for your help. Cheers, vmon

2 2