August 2012 - tor-dev - lists.torproject.org

Damian's Status Report - July 2012
by Damian Johnson 05 Aug '12

05 Aug '12

Hi all. My July was mostly spent traveling, both for Defcon [1] and a couple visits with my family on Vashon (for a funeral and Strawberry Festival). August will be much the same. I'll be gone... * 8/8 - 8/12 for Toorcamp [2] * 8/17 - 9/3 for a family trip Besides that and cat wrangling for the GSoC midterms, here's the things I did this month... * Implemented an ExitPolicy class [3]. Sathyanarayanan did the initial port of the class from arm, but that implementation turned out to have several issues (no IPv6 support, masking was incomplete, didn't strictly conform to the spec) so I ended up rewriting it. The result was pretty nice... https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/exit_policy.py https://gitweb.torproject.org/stem.git/blob/HEAD:/test/unit/exit_policy/pol… https://gitweb.torproject.org/stem.git/blob/HEAD:/test/unit/exit_policy/rul… * Not tor related, but submitted five patches to ReviewBoard... * Re-enabling lexer guessing based on content [4] * Supporting dashboard impersonations [5] * Better supporting mixed newline types in diff reconstruction [6] * Crashing issue when authentication fails [7] * Better handle perforce symlinks [8] * Variety of small things... * Skipping tests on OSX that make numerous sockets to unblock development there [9] * Unsuccessful attempt to fix Windows specific date formatting issue [10] * Forgot to account for tor's debugger disabling in stem's proc tests [11] * Supporting the transport line in extrainfo descriptors [12] * Expanding the Version class and tor spec to include git commit ids * Variety of pylint fixes [13] * Discussion about is_alive() handling and quitting the socket [14] * Ravi has also been busy, finishing several controller methods... * GETCONF support [15] * SETCONF and RESETCONF support [16] * LOADCONF and SAVECONF support [17] * SETFEATURE support, though this one's still incomplete [18] Cheers! -Damian [1] http://www.atagar.com/arm/log.php#defcon2012 [2] http://www.toorcamp.org/ [3] https://trac.torproject.org/5454 [4] http://reviews.reviewboard.org/r/3156/ [5] http://reviews.reviewboard.org/r/3207/ [6] http://reviews.reviewboard.org/r/3208/ [7] http://reviews.reviewboard.org/r/3209/ [8] http://reviews.reviewboard.org/r/3221/ [9] https://trac.torproject.org/5917 [10] https://trac.torproject.org/6206 [11] https://trac.torproject.org/6286 [12] https://trac.torproject.org/6257 [13] https://gitweb.torproject.org/stem.git/commitdiff/fd2da10 [14] https://trac.torproject.org/6234 [15] https://trac.torproject.org/6114 [16] https://trac.torproject.org/6239 [17] https://trac.torproject.org/6356 [18] https://trac.torproject.org/6417

1 0

OnionOO protocol questions
by Norman Danner 04 Aug '12

04 Aug '12

Hi Karsten, A few questions on the Onionoo protocol specification: If multiple parameters are specified in the GET request, does that imply a logical and or a logical or (or something else)? E.g., suppose the GET request has type=relay and lookup=FFFF... . To me it seems like that should return the relay with fingerprint FFFF..., if there is such a relay. But it could also mean return all relays as well as the router with fingerprint FFFF... . The search parameter is described as "Return only relays with the parameter value matching the beginning of a nickname, (possibly $-prefixed) fingerprint, or IP address..." Do you mean that the parameter value matches the beginning of a nickname, the beginning of a (possibly $-prefixed) fingerprint, or is one of the relay IP addresses? For ordering, what do you mean by "possible fields for ordering are: consensus_weight." This is really two questions. How is the consensus_weight ordering defined when producing a summary document (which doesn't include a consensus weight field)? Can clients not request that summary documents be ordered by any of the 'n', 'f', 'a', or 'r' fields? Thanks! - Norman -- Norman Danner - ndanner(a)wesleyan.edu - http://ndanner.web.wesleyan.edu Department of Mathematics and Computer Science - Wesleyan University

2 4

[GSoC] Stegotorus
by vmon 04 Aug '12

04 Aug '12

Hi there, The payload strategies can be changed easily now by inheriting different class from PayloadServer. Current strategy is now in TracePayloadServer. For ApachePayloadServer strategy, server side is complete. for the client side, I need a handshake in steg level so the server sends the client, url dictionary. This needs to manipulate the program in level of Chop protocol. I spent few days to add this capability where chop send a reply handshake to the client (based what is in the paper) I haven't find it yet. Time to ask my mentor. Cheers, vmon

3 2

Sathya's status report - July 2012
by Sathyanarayanan Gunasekaran 04 Aug '12

04 Aug '12

July 2012 - Attended Tor dev meeting and hackfest in Florence in the first week of July. - Talked to arma, Karsten and asn about helping out with certain metrics tasks(#6460, #6232) to figure out the safety of the Tor network. Hopefully, I can extend this to something bigger and hack on it for my undergrad thesis. - Hacked on a entropy-over-time script(#6232) with help from Karsten, phw and asn. - Pair programmed with neena, helping him with his GSoC work(the consensus and votes descriptor parser). - Discussed pyonionoo components on the tor-dev mailing list[0]. - Hacked on the web service for pyonionoo for a bit at the hackfest. Megan is currently refactoring it and making it awesome. - Started work on the pyonionoo backend(#6452) - - Completed the network status document parser for summary documents - Taught pyonionoo about stem to parse server descriptors for details documents - Next up - Refactor this and add bandwidth documents. - Finished ExitPolicy parser for stem(#5454), which Damian ended up rewriting mostly. - Discussed the proper way to check if the Tor control socket is closed or not(#6234) which led to "Tor should have a PING command to check"(#6354), but arma and nickm are of the opinion that we shouldn't have any more pre-auth commands since it makes Tor more complex / less safe. Stem now makes a best-effort call (errors ignored) since we just want the socket to shut down and we don't need the QUIT to succeed to do that. - Discussed about setting up a code review tool(#6402). I've set up an ec2 instance with ReviewBoard and will hopefully set up the repos with some help from Damian soon-ish. [0] - https://lists.torproject.org/pipermail/tor-dev/2012-July/003731.html

1 0

Ticket #5749
by SiNA Rabbani 03 Aug '12

03 Aug '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have attached a patch: 0001-Faravahar-added.patch.asc to this ticket. Faravahar is very much ready! The patch is gpg signed, let me know if there are any other actions required on my part. All the best, SiNA - -- "If I had eight hours to chop down a tree, I?d spend the first six of them sharpening my axe." -Abraham Lincoln -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJQHDhXAAoJEJPBwXYLR9Vt2Q0P/jRRUvrjB0AcoFagw9rUZzFr 1Os0RYPk1j9HXz30xJy9u+3sLQAaRQovpgvRJJv6GY0Jb+RCI9A4k6ExaHO4nxHq OJPhW6HcWa+w6QcFxIb4GizYVKthxcxcBPfVGPXvzH/U/gkGiuJYGNTXeCUhZNi9 JI5gtfOriwp7pspcXyufLZcAcjGE+8LKlwnEhqNeMhw6bGvqWFuueb5ZYLQwuOn/ ngtLFsD1NnAXgTAJY9wFS9tTmDX3wISKzMq726lKieOBqOH2v4+nkusoq8CUNfjK NPq8MsrtHEtxsEdR5nMy2PDROKHsUNclO9mV8+W/K5nRDjJU9wzgBlX+fttNZ5H6 AH9+1f6Tr4Z1KzLfNfBrwoq4kCgkIIvCkYl4xL+0RboFuITZuqqKwE0tA0MDMh6X uAKSkB7WsXrVbKpYVfppD9Z2blKmJ80gLvOWkuiptoe9mI/EhmfnzlwdGYzcp+80 A3ovAI7JADCQNYQjaVST1nudVy0nUGua957JKgry6P/GM83nj1p7krIKs0FzQtly pG6tyuLySFKFyhfdZ3jkpi/fJwrffDgXVrT2yXh4hljTaHd5ym6WjFrHcnoBdLLx 895jZhaFh65rBpLpq1yYa4yoO0+J0MKTRy5qDEgzZoq6Y5TjagEk3tKCmIF62dW7 18iv/9Dp/+6eZbTOjnN4 =UQ4G -----END PGP SIGNATURE-----

1 0

Karsten's status report July 1--31
by Karsten Loesing 03 Aug '12

03 Aug '12

Hi all, here's what I did in July. The numbers in [] are the ticket number and percentage of my total developer time in July. Best, Karsten Briefly looked into Maxmind's GeoIP database that started labeling relays as coming from the "A1 -- Anonymous Proxy" country. We'll need to fix that, or users won't be able to exclude relays by country, and our country-based usage statistics will deteriorate. We switched back to the May database for now, but that's only a short-term solution. [#6266, 2%] Made the consensus-health script print out diffs of recommended version sets. [#6261, 1%] Fixed a problem with Tomcat running out of memory on the metrics server, which was related to Java's leap second bug. [#6270, 2%] Looked up the number of crypto operations on a typical relay for designing a hardware Tor device. [#6369, 5%] Started converting the first five old tech reports to a common Tor Tech Report format. Zack was really helpful here by writing an improved LaTeX template and tweaking the existing reports. [#5405, 8%] Helped delber write a Python script to compute statistics about current relays. This script is the second really useful front-end for the Onionoo service. [#6329, 12%] Extracted probabilities of relays to be selected for a given position in order to add more graphs to Onionoo. The next step will be to extend Atlas to display the new graph data. [#5755, 31%] Helped Sathya write a script to extract data on network entropy and degree of anonymity over time. Ran the script on local tiny number cruncher, which kept it busy for 12 hours, and plotted graphs. [#6232, 14%] Fixed an issue with too frequent consensus-health messages reporting expiring certs. [N/A, 1%] Extracted and graphed cumulative probabilities of clients selecting among the biggest k exits. [#6443, 13%] Discussed and prepared removing GetTor statistics from all metrics codebases. [#6395, 3%] Made metrics website graph colors easier to distinguish for people with color-deficient vision. [#6463, 2%] Reviewed Walter's new interactive metrics graphs on http://tigerpa.ws/tor_metrics/ and fixed a few metrics bugs while doing so. [N/A, 6%]

1 0

We have funding for another browser hacker
by Mike Perry 02 Aug '12

02 Aug '12

Hey everyone, We're looking to add another Firefox developer to our team to help improve Tor Browser. For details on the position, experience needed, and how to apply, please see the job posting: https://www.torproject.org/about/jobs-browserhacker.html.en -- Mike Perry

1 0

[GSoC] Pluggable Transports in Python Status Update - More Transports
by Brandon Wiley 02 Aug '12

02 Aug '12

So far I've implemented Dust and obfs3 and I'm putting the finishing touches on obfs2. I've also rewritten the pluggable transport API and rewritten the transport plugins to use this new API. The API now provides 4 events: start, encodedReceived, decodedReceived, and end. The plugins implement callbacks for these events as well as an __init__(self, decodedSocket, encodedSocket) method that provides virtual sockets to the plugins. The virtual sockets have read and write methods. Control is transferred to the plugin when one of the event callbacks is called, in at which time the plugin can read and write data to the socket buffers. When the callback returns, control is transferred back to the framework, which takes care of reading and writing between the buffers and the actual sockets. The next step is testing of all of the plugins and documenting the plugin API. In particular I expect that the python obfs2 implementation will not work with the C obfs2 implementation due to undocumented assumptions in the protocol implementation, so these will need to be worked out. Code: http://github.com/blanu/pyptlib and http://github.com/blanu/py-obfsproxy Blog: http://stepthreeprivacy.org/

2 1

George's status report: July 2012
by George Kadianakis 02 Aug '12

02 Aug '12

Hi, this is a list of some Tor stuff I did during July: - Got #3589 merged in 0.2.4! - Worked with Arturo on daphn3, an OONI plugin which, given a censored TCP conversation, tries to bruteforce the fingerprint used by the DPI box [0]. Unfortunately, it's still not quite ready for real life use! - Helped around with the problem of measuring the safety of the Tor network [1]. - Helped Brandon with his GSoC project [2]. - Talked with George Danezis about anomaly detection and his automatic censorship detection tool [3]. He has many ideas on improving his current anomaly detection model but they don't seem to work well in practice. I will try to read some stuff about anomaly detection in case I can help out. Meanwhile, Philipp Winter is interested in applying a different anomaly detection model to censorship detection and seeing how well it will work. [0]: https://speakerdeck.com/u/hellais/p/ooni-and-daphn3 [1]: https://blog.torproject.org/blog/research-problem-measuring-safety-tor-netw… https://trac.torproject.org/projects/tor/ticket/6460 [2]: http://stepthreeprivacy.org/ [3]: https://lists.torproject.org/pipermail/tor-dev/2011-September/002923.html https://trac.torproject.org/projects/tor/ticket/6067 https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-censorship-events

1 0

How to best support tor 0.2.3.x in Tor Browser Bundle?
by Mike Perry 01 Aug '12

01 Aug '12

Quick recap from #tor-dev IRC convo with Nick, Roger, and Andrew: We need to get a usable tor 0.2.3.x-rc bundle out RSN so we can declare it "stable", but there are concerns that using Firefox 14 with this will continue to cause unexpected problems and otherwise scare people away from testing tor 0.2.3.x enough. However, I still need to have a place to commit TBB-alpha patches, and have three bugfixes (including a fix for a FF14 crash bug that was discovered by tor-qa) that I'd like to get into the alpha series. Also, if we don't provide Firefox Rapid Release with regular alpha testing, we're going to be really, really sad when everything breaks at once in November with the next Firefox ESR. One option is to create a temporary "rc" branch of torbrowser.git's maint-2.2 to build tor 0.2.3.x but with the rest of a "TBB stable" bundle with Vidalia 0.2.x and Firefox 10.x ESR, and leave maint-2.3 as "TBB alpha". A second option is to create a more permanent "TBB beta" series, with whatever software smells like it is getting close to stable at a given point in time. A third option is to just keep doing English-only maint-2.3 builds back-to-back until tor-qa stops reporting crash bugs or strange issues. However, we need input from Erinn to decide the best approach. If the "rc" fork (or a permanent beta branch) messes up the build process or introduces issues with build automation work, perhaps it is not the right way to go, and we should just keep doing english-only maint-2.3 releases back-to-back until FF14 is more stable (Note: it works OK for me now in my test builds with the crash fix applied). So Erinn, which one is least painful for you? Also, does anyone else have any other input or suggestions on how to approach this problem? We're probably going to run into this issue periodically due to our different components stabilizing at different points... -- Mike Perry

4 4