September 2012 - tor-dev - lists.torproject.org

Damian's Status Report - August 2012
by Damian Johnson 03 Sep '12

03 Sep '12

Hi all. I spent most of August, like the prior month, traveling. This time I attended Toorcamp [1] and went on vacation. Both were pleasant and relaxing, but not terribly conducive to coding so I don't have much to report... * Descriptor CSV Export Functionality Naif proposed export functionality for Tor server descriptors a while back [1], which Eric and Megan took the first stab at [2]. I ended up revising this quite a bit [4], but it turned out nicely. * Caching Expansion and Test Prompt Added caching for GETCONF and static GETINFO queries. I also added a handy little script for a debugging interpretor. It kicks off Tor, then provides an interactive python prompt with a controller instance to it (optionally shutting Tor down afterward). Ravi code reviewed these changes [5] and volunteered to do code reviews of my future work as well. * Consensus Parsing Over the last few weeks Ravi's been working on a patch to parse network status documents [6]. It's functional, but missing unit tests and deviates from the parsing style and strict validation done for the other descriptor types so I'm taking a turn with the code. Thus far I'm done revising and adding tests for router status entries, and now working on the document. Changes are available in the 'document-parsing' branch of my repo [7]. Parsing these documents is a far larger task than I thought (especially if you include v2 documents and microdescriptors), so working on this branch will probably keep me occupied for much of September. * Controller Expansion Ravi has gone on a hacking binge, adding support for USEFEATURE [8], SIGNAL [9], EXTENDCIRCUIT [10], and SETCIRCUITPURPOSE [11]. Cheers! -Damian [1] http://www.toorcamp.org/ [2] https://trac.torproject.org/6171 [3] https://trac.torproject.org/6512 [4] https://gitweb.torproject.org/stem.git/commitdiff/7022021c4207a0066cb932616… [5] https://trac.torproject.org/6631 [6] https://trac.torproject.org/6569 [7] https://gitweb.torproject.org/user/atagar/stem.git/shortlog/refs/heads/docu… [8] https://trac.torproject.org/6417 [9] https://trac.torproject.org/6659 [10] https://trac.torproject.org/6666 [11] https://trac.torproject.org/6670

1 0

Tor on IPv6 roadmap and status page
by Linus Nordberg 02 Sep '12

02 Sep '12

Hi, The roadmaps/Tor/IPv6 [1] wiki page has been started with the goal of communicating the status of the work with Tor on IPv6. Additionally, searching for Trac tickets with the keyword "ipv6" [2] might give a picture of what's going on. Asking here or highlighting ln5 on #tor-dev is of course another perfectly fine option. [1] https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/IPv6 [2] https://trac.torproject.org/projects/tor/query?keywords=~ipv6 -- Linus

4 5

Linus' August 2012 status report
by Linus Nordberg 01 Sep '12

01 Sep '12

Hi, What happened in August. Got most of #4564 working and parts of it merged to master, including - Directory authorities vote for IPv6 public relays (micro descriptors not yet merged) - Relays bind to and publish IPv6 addresses - Clients use IPv6 relays in consensus (not using micro descriptors) Created an IPv6 status and road map page roadmaps/Tor/IPv6 [1]. Tickets worked on include 4455, 4620, 5535, 6362, 6363, 6364, 6423, 6514, 6621. Made an interview for the Swedish paper Ny Teknik. What's happening in September. September is the month of getting Tor on IPv6 deployed! Primarily the client to non-private bridges parts and hopefully also client to relays. See above mentioned IPv6 road map page [1] for details. [1] https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/IPv6 -- Linus

1 0

Karsten's status report August 1--31
by Karsten Loesing 01 Sep '12

01 Sep '12

Hi everyone, August was a long month for me, and I feel like I spent more time reviewing other people's code than writing my own. That's really cool! We have a bunch of new metrics-related code now, and I hope there will be even more code contributions for me to review next month! So, below is what I did in August. Compared to last month's report, I left out how much of my developer time I spent on these tasks and added more links instead. Best, Karsten Added three new graphs to the metrics website: the first two graphs show our progress in operating 125 fast exits [1] (first half of #6498), which is a deliverable for sponsor J [2]; the third graph shows advertised bandwidth and bandwidth history by relay flags [3] (#6671). Also fixed problems with cutting off enough days from graphs for which we don't have enough data yet (#6593) and removed GetTor graphs and related code to reduce future maintenance effort (#6395). The gentle reader may have noticed that there was no mention of code reviewing in the paragraph above, but just own code writing. That's correct, the metrics website is still a one-man show. I'm planning to split up the metrics website beast into a component that produces JSON-formatted graph data [4] and ask other people to write components visualizing the data, like Walter Kim's metrics website prototype [5]. Hopefully there will be more code contributions if people don't have to mess with Java servlets in the current metrics website code. Reviewed Sathya's great contributions to what we're now calling Compass [6]. Compass is a website and command-line tool to filter the set of currently running relays and group by same country or same AS. It was originally written by delber to analyze Tor network diversity. Sathya improved Compass by adding options to show only fast exits and almost fast exits for the sponsor J project to add 125 fast exits [2], and he wrote the website part. Since Compass went live, we had a lot of tickets opened by cypherpunks that we're almost unable to handle [7]. Patches welcome, Sathya and I will review them! Improved graphs in Atlas [8] to show the displayed quantity on the y axis (#5388) and added graphs on various probabilities to be selected by clients (#5455). Arturo reviewed and cleaned up these patches. Speaking of code contributions, there are quite a few open tickets [9] that would benefit from JavaScript developers wanting to help out! Reviewed Pyonionoo [10], the Wesleyan students' Python rewrite of Onionoo, the data provider behind Compass and Atlas. More precisely, they were working on the front-end portion of Onionoo which gives out JSON-formatted data, not on the number-crunching back-end that Sathya is about to rewrite in Python (#6452). Basic functionality of Pyonionoo works, but there are a few defects (#6707, #6708) and one enhancement (#6709) that we'll have to work on before deploying it. Of course, we have more open tickets for the Java and/or Python Onionoo [11]. Converted even more old tech reports into the new Tor Tech Report format [12] (#5405). The goal here is to have a single place for all reports we produce and that other researchers can use to cite our work. Set up Rob's Tor simulator Shadow [13] to see if there are any usability issues that would prevent researchers from using it. Turns out it's not just a very powerful tool to evaluate changes to the Tor code, but also doesn't require its users to be rocket scientists. I'm planning to play more with Shadow in September and walk through the instructions in Shadow's wiki [14] to find any remaining pitfalls. And then I'll encourage Roger and Rob to blog about Shadow. [1] https://metrics.torproject.org/fast-exits.html [2] https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorJ [3] https://metrics.torproject.org/network.html#bandwidth-flags [4] https://metrics.torproject.org/graphs/relays-by-country [5] http://tigerpa.ws/tor_metrics/ [6] https://compass.torproject.org/ [7] https://trac.torproject.org/projects/tor/query?component=Compass&status=!cl… [8] https://atlas.torproject.org/#details/D223399907113A1F216AAA64997BC1D4CFA8E… [9] https://trac.torproject.org/projects/tor/query?component=Atlas&status=!clos… [10] https://gitweb.torproject.org/pyonionoo.git [11] https://trac.torproject.org/projects/tor/query?component=Onionoo&status=!cl… [12] https://research.torproject.org/techreports.html [13] http://shadow.cs.umn.edu/ [14] https://github.com/shadow/shadow/wiki

1 0

Nick's August 2012 status report
by Nick Mathewson 01 Sep '12

01 Sep '12

THINGS I DID IN AUGUST I spent a few days vacationing with my family in Maine; took a day off for my birthday, and generally chilled out. I merged lots of bugfixes and patches -- some mine, some other people's, some requiring rewriting and review, some not. This included some security-related stuff that lost me a little sleep. It also involved hacking on some neat patches from Stewart Smith (with help from Jim Meyering) to revamp our build system. We put out a new 0.2.2. I fixed the more important (fixable) bugs in 0.2.3 (notably 6507 and 6244 and 6404), and pushed back on some others. IMO we're now ready to release the (probably) last 0.2.3.x-rc. I merged more patches into 0.2.4; see the new changes files there for more info. Lots of these were ipv6-related stuff from Linus. I started a draft for BEGIN2 cells so I can try to get a more future-proof IPv6 protocol implementation. This is blocked while I teach myself how DNSSEC works once and for all. I tried to get a little more testing on alternatives to our current (bogus, deteriorating) geoip supplier (see #6266) but didn't get anything beyond my older inconclusive results. I started working on an implementation for proposal 200, and a revised 202. With help from some libevent users, I fixed an SSL bug that might have been one of the (probably several) things making SSL bufferevents unstable. The fix appears in libevent 2.0.20-stable. I agreed to fly out to SF to accept the EFF's pioneer awards on Tor's behalf. I had an anniversary and a birthday. I proposed a crazy idea about a separate 0.2.3.x branch for DAs. I chatted a little about metaformats, and spent a while catching up on the ones I didn't know the details of. Currently I'm thinking YAML is a hot mess, and ProtocolBuffers is not too bad actually. There was this insane stegotorus bug that I sort of tracked down. That is, I can tell you which libevent commit to revert in order to make the problem go away (that is, 3467f2fa3bbff0dab6865), but I can't for the life of me figure out what's wrong with it. My current guess is that it provokes an issue somewhere else. A couple of friends/acquaintances approached me about working for Tor as a project manager. I told them to email jobs(a)torproject.org. I don't think we've heard from them yet. It would rock if we had a faster release schedule. I've been trying to find projects our size that do it right. This coming month I must help Linus get our September IPv6 deliverables deployed. I need to help Andrea get her refactored channel abstraction merged, and make sure she's got more stuff to do (and more interesting stuff to do!) as she transitions to full-time. I must get that blog post I'm cowriting with Steven finished, and our paper for November started. And I need to write some of my own code too. I also need to finalize a plan and a schedule for 0.2.4 and send them to the world. "when in danger or in doubt," -- Nick

1 0