- tor-dev - lists.torproject.org

Damian's Status Report - October 2012
by Damian Johnson 02 Nov '12

02 Nov '12

Ahhh, the beginning of winter. That very special time of the year when Seattle skies are perpetual overcast and gloomy. When crowds flock to the coffee shops and the best thing to do with your time is hook up to a caffeine IV and code. ... on a side note it has been a really productive month. ======================================== * Stem Website Stem now has a far more developer friendly website! https://stem.readthedocs.org/en/latest/index.html Barring the obvious new sections I revised vast swaths of stem's API documentation. Feedback welcome! The tutorial section is just starting, but I have some ideas for expanding it. ======================================== * Network Status Document Parsing After a couple months of work by Ravi and me the network status document handling is *finally* done. It turned out to be a... very big feature branch. Announcement: https://lists.torproject.org/pipermail/tor-dev/2012-October/004092.html Docs: https://stem.readthedocs.org/en/latest/api/descriptor/networkstatus.html Code: https://gitweb.torproject.org/stem.git/commitdiff/ea356b7?hp=5cd457b With this done we're very nearly at feature parity with metrics-lib (and I suspect a bit past it in terms of testing). In finishing this up I also spotted an undocumented oddity with microdescriptor 'directory-signature' lines. [1] ======================================== * GSoC Mentor Summit Met with the developers of numerous other open source projects. I was really impressed at how many worthwhile conversations were crammed into such a short conference. Highlights include... * Arc from Python had distutils suggestions, thanks to which stem and arm will soon have both Python 2.x and 3.x support. * OSU's Open Source Labs are entertaining the thought of running Tor relays. * Discussed TorBirdy and other Tor projects with Sukhbir. He mentioned an issue with trac permissions that is now fixed. [2] * Talked with Terri from Python about Mailman 3, which would be a great answer for the requests we get to have a Tor forum. [3] * Attended a talk led by Marina from Gnome about their outreach program for women. We'll likely be taking part in it this year. For the full trip report see... https://lists.torproject.org/pipermail/tor-dev/2012-October/004115.html ======================================== Other changes include... * Cleaning up orphaned pyc files as a part of running our tests (feature suggested by Ravi) [4] * Ported arm's str_tools module to stem and added unit tests [5][6] * Code reviewed Ravi's attach_stream addition, waiting on the revisions [7] * Arm troubleshooting on irc with zack and ultramage. Also fixed a broken link on arm's site spotted by wh6iQ. * A new contributor, Eoin, spotted some great issues and contributed patches. Tickets include... * python 2.5 compatibility bugs [8] * count of the number of skipped tests [9] * error with the whitespace checker [10] * spelling corrections [11] * Sathyanarayanan spotted some nice bugs including... * on python 2.5-2.6 a missing microdescriptor consensus wasn't causing the related test to be skipped [12] * integ tests for the process module would fail if tor was already running [13] ======================================== Next up: Ravi and I are working on tor event handling, the last major feature stem is missing as a controller library. After that we plan to port arm over, then tidy up loose ends in preparation for stem's initial release. Cheers! -Damian [1] https://trac.torproject.org/7072 [2] https://trac.torproject.org/7186 [3] https://trac.torproject.org/3592 [4] https://trac.torproject.org/7103 [5] https://trac.torproject.org/7158 [6] https://stem.readthedocs.org/en/latest/api/util/str_tools.html [7] https://trac.torproject.org/7152 [8] https://trac.torproject.org/7244 [9] https://trac.torproject.org/7246 [10] https://trac.torproject.org/7263 [11] https://trac.torproject.org/7245 [12] https://trac.torproject.org/7250 [13] https://trac.torproject.org/7251

1 0

circuitmux_ewma.c
by Gisle Vanem 01 Nov '12

01 Nov '12

These gcc extensions: /*** EWMA circuitmux_policy_t method table ***/ circuitmux_policy_t ewma_policy = { .alloc_cmux_data = ewma_alloc_cmux_data, .free_cmux_data = ewma_free_cmux_data, Doesn't work well with MSVC. Here is a patch: --- ..\..\Git-latest\src\or\circuitmux_ewma.c 2012-11-01 15:53:23.125000000 +0100 +++ circuitmux_ewma.c 2012-11-01 17:52:55.371093500 +0100 @@ -200,15 +200,22 @@ /*** EWMA circuitmux_policy_t method table ***/ -circuitmux_policy_t ewma_policy = { .alloc_cmux_data = ewma_alloc_cmux_data, - .free_cmux_data = ewma_free_cmux_data, - .alloc_circ_data = ewma_alloc_circ_data, - .free_circ_data = ewma_free_circ_data, - .notify_circ_active = ewma_notify_circ_active, - .notify_circ_inactive = ewma_notify_circ_inactive, - .notify_set_n_cells = NULL, /* EWMA doesn't need this */ - .notify_xmit_cells = ewma_notify_xmit_cells, - .pick_active_circuit = ewma_pick_active_circuit +#ifdef __GNUC__ + #define STRUCT_INIT(member,val) member = val +#else + #define STRUCT_INIT(member,val) val +#endif + +circuitmux_policy_t ewma_policy = { + STRUCT_INIT (.alloc_cmux_data, ewma_alloc_cmux_data), + STRUCT_INIT (.free_cmux_data, ewma_free_cmux_data), + STRUCT_INIT (.alloc_circ_data, ewma_alloc_circ_data), + STRUCT_INIT (.free_circ_data, ewma_free_circ_data), + STRUCT_INIT (.notify_circ_active, ewma_notify_circ_active), + STRUCT_INIT (.notify_circ_inactive, ewma_notify_circ_inactive), + STRUCT_INIT (.notify_set_n_cells, NULL), /* EWMA doesn't need this */ + STRUCT_INIT (.notify_xmit_cells, ewma_notify_xmit_cells), + STRUCT_INIT (.pick_active_circuit, ewma_pick_active_circuit) }; /*** EWMA method implementations using the below EWMA helper functions ***/ --gv

2 2

Reminder of November, December deadlines for Tor 0.2.4
by Nick Mathewson 01 Nov '12

01 Nov '12

Hi, all! >From the Tor 0.2.4 schedule at https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/024 : """November 20, 2012: Big feature proposal freeze. Any big feature which requires a design proposal must have its design proposal finished by this date. If it needs changes after this date, then maybe we didn't design it well enough, and we probably shouldn't merge it. December 10, 2012: Big feature merge freeze. No big features will be merged after this date. December 15, 2012: Small feature proposal freeze. Any small feature which requires a design proposal must have its tiny little design proposal finished by this date. January 10, 2013: Feature merge freeze. No features after this date. I mean it. ... The definition of "big patch" is deliberately vague. Note that the cutoffs above are merge freezes, not submission freezes.""" (I've just added 10 days to that first deadline, and 5 days to the third.) Also, I'm currently thinking that, to give these deadlines teeth, I shouldn't self-enforce, since I've veered between "pushover" and "insane zealot" in the past in ways that didn't make a lot of sense. Rather, I'm going to say that in cases of doubt ("Is this a little feature") and cases where I'd like to violate freezes ("surely this can go in on December 12"), I'm not going to merge patches on my sole discretion any more. Rather, I'm going to say that at least two of me, Roger, and Andrea ought to agree to violate a freeze before we do so. (I will try not to forget and mess this up.) With any luck, this will let us get closer to a reasonable 6 month release cycle. yrs, -- Nick

1 0

[Patch] tools\tor-fw-helper\tor-fw-helper-natpmp.c
by Gisle Vanem 01 Nov '12

01 Nov '12

The code in tools\tor-fw-helper\tor-fw-helper-natpmp.c does things wrong on Winsock where e.g. errno isn't set on socket-errors and 'fd' is (always?) >= FD_SETSIZE. Patch attached. --gv

2 1

gsathya's October
by Sathyanarayanan Gunasekaran 01 Nov '12

01 Nov '12

I was mostly busy with giving the required exams for grad school and generally worrying about. I did manage to do a bunch of Tor stuff - - Implemented DisableNetwork in Orbot. Orbot stops making new connections and drops existing ones once you lose network connectivity, saving battery life. Also, one of the features I implemented during GSoC11 is now finally merged into master -- Orbot now shows usage statistics(bandwidth). - Wrote a whole bunch of tests for Pyonionoo -- which in hindsight weren't really good. I will mostly need to refactor them and make them more modular. - Added more functionality to Pyonionoo -- /bandwidths and /details work now - Hacked on #1854[0] - Set up a pyobfsproxy bridge to help test. - Found some bugs in stem - #7133, #7250, #7251 - Generally hung around #tor providing support. Hopefully I can do this a lot more. [0] - https://trac.torproject.org/projects/tor/ticket/1854

1 0

use tor controller to detect circuit/path change
by esolve esolve 31 Oct '12

31 Oct '12

Hi, I'm new to Tor controller, Tor client usually open 3 circuits, one for current use and the other 2 for standby. Tor will change the current circuit/path in use from time to time, depending on the network load dynamics I'm wondering whether it is possible to use Tor controller to detect when the circuit/path in use is changed and obtain the list of IP addresses of those relay nodes. thanks!

4 6

[Stegotorus] Re: Do you have time to poke at ST
by vmonmoonshine＠gmail.com 25 Oct '12

25 Oct '12

Hello Zack, Sorry for being bit out of touch as I'm traveling and my access to Internet is sporadic. I'm still working on Stegonotorus. I developed the webpage_tester (now on f--webpage-tester branch on github), that is curl using Stegotorus to download anything on the web and compare it with the original. It is based on assumption that is my best approximation for a browser. I did that in a hope to catch the bug, but I haven't. Basically I failed to browse anything unreliably. I can download 20 megabytes or so with no error. It takes a long time, but It doesn't fail. My payload server is the apache payload server but that shouldn't make a huge difference. I think my payloads even have smaller capacity and less efficient than your fake ones. Anyway, I'm going to tweak the tester proxy to drops packets at random, to force the ack into play, I'll get back to you with more update. Cheers, vmon

1 0

New tech report: Counting daily bridge users
by Karsten Loesing 24 Oct '12

24 Oct '12

Hi everyone, I just finished a new tech report titled "Counting daily bridge users": https://research.torproject.org/techreports/counting-daily-bridge-users-201… >From the abstract: "As part of the Tor Metrics Project, we want to learn how many people use the Tor network on a daily basis. Counting users in an anonymity network is, obviously, a difficult task for which we cannot collect too sensitive usage data. We came up with a privacy-preserving approach for estimating directly connecting user numbers by counting requests to the directory mirrors and deriving approximate user numbers from there. In this report we describe a modified approach for estimating the number of users connecting via bridges by evaluating directory requests made to bridges. We compare this new approach to our current approach that estimates bridge user numbers from total unique IP addresses seen at bridges. We think that results from the new approach are closer to reality, even though that means there are significantly fewer daily bridge users than originally expected." Feedback is much appreciated and could influence our implementation of user number estimates in the future (#7154). Thanks, Karsten

1 0

Re: [tor-dev] Automating Bridge Reachability Testing (#6414)
by Isis 23 Oct '12

23 Oct '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Karsten! Oh sheesh. I did not see it...I will have to figure out why. That is slightly worrying. So, I am rushing to meet the final deadline, but I still think it is doable. I have mostly finished up my OONI work for the month, and I planned to spend the remainder of this month working on the bridge test. I have finished most of the actual Tor connection code, as well as one of the four basic packet level scans (the icmp8 one). Two of the other packet level scans, the TCP SYN and ACK ones, are pretty much copies of the icmp8 one with a couple lines changed, and they shouldn't be a big deal. There is still the vanilla TLS handshake test/scan/thing, which has not been started yet, and will take a bit more time than the others because Python notoriously has problems with SSL bindings and libraries, so I'll need to do a bit of research on newer ones and updates and see which is the best to use now. I hear that tlslite[1] is the current best choice; if anyone else has any input on this it would be very helpful. :) There were a couple minor hups: 1) When George asked me to test pluggable transports, this required significantly more refactoring than I previously thought was necessary. 2) Arturo redesigned the OONI testing framework API again to use a completely different structure, which was supposed to be backwards compatible and turned out not to be (though I believe that my recent OONI commits fixed that). However, I have been fighting the framework already, because the main scripts in OONI (/ooni/oonicli.py and /ooni/ooniprobe.py) control the reactor, and also expect static iterations through single test and single control functions for each asset (an asset in this case would be one bridge address). The bridge testing is rather dynamic (I would like it to be able to evaluate an approximate danger level to running the next test) and so the framework is kind of troublesome. Also, because the framework handles calling the reactor (in Twisted, the reactor is a sort of event scheduler), and it also expects a rather linear progression of defer.Deferreds (in Twisted, those are standin objects which execute callbacks when they get results from some previous deferred/callback), it would be nicer if I had full control of these myself without needing to hack around the parent scripts. I think it's wise that OONI deals with these things for the testwriter in most cases, because the testwriter shouldn't be expected to be an expert in using Twisted. However, I also think that, in the long term, OONI shouldn't prohibit people who know what they are doing or are doing odd things from being able to do so. As a result, I've decided (for now), to use bits are parts of the OONI code before the recent refactoring, and later (after the deliverable) I will work on adding flags to OONI to give the test script full control of the reactor and deferreds, as well as evaluating whether or not the bridge test is even compatible with the new API. I do not want to get caught up in dealing with this right now, I just want to have it all working and deployable in a way that I know will work. 3) The indirect scans are becoming quite complicated to automate in any sane fashion. I still would like to continue working on this, as I'm quite enjoying the difficulty, but due to their temporary and volatile nature (they will change frequently depending on the blocking methods of a particular country and the currently available in-country bounces/proxies/whatever-thing-the-indirect-scan-uses), as well as the fact that many of these methods are still undiscovered, I think it is safe to add them as specialty cases after the fact without impacting overall general testing. There is one in particular that I would like to finish before the deadline because I am quite proud of it and am having a lot of fun working on it, but I'm first going to concentrate on wrapping up the active scans. There are other things which I've marked as helpful things to do, but which are not necessarily part of this deliverable: 1) Having a parser for bridge descriptors to turn them into test inputs, and vice versa. 2) Having some undiscoverable method for setting up lots of IPv6 bridges on one OR (Tor currently only allows up to eight, I believe) and having these be discoverable by bridgedb and no one else. I was thinking of this while talking with Aaron, because he reminded me that people on IPv6 have tons of IPs available, and I was thinking that if we configured some type of one-way hash function, we could say that a bridge descriptor for 2001:db8::1:1 should actually mean multiple descriptors for 2001:db8::fa98:38d2 2001:db8::e099:2188 2001:db8::88aa:3b7 or something, derived from the output of hashing the original descriptor with the OR's key or something else. This would help distribute bridges in the future quite a bit, though it doesn't do much for the current bridge situation. Anyone wanting to help with the above two things, or with an idea for another indirect scan, or with feedback on anything I'm working on, should feel free to contact me and it will be greatly appreciated. :D .-- .. ... .... - -- . .-.. ..- -.-. -.- <(A)3 isis agora lovecruft .-.-. [1] https://github.com/trevp/tlslite On Fri 12 Oct 2012 at 14:56, thus spake Karsten Loesing: > Hi Isis, > > did you see this email? > > Sorry for being pushy, but we're less than three weeks away from the > deadline, which makes me kinda nervous. > > Thanks, > Karsten > > > On 10/8/12 9:14 AM, Karsten Loesing wrote: > > Hi Isis, > > > > can you give me a brief status update on the sponsor F year 2 > > deliverable "Automating Bridge Reachability Testing", ticket #6414? > > > > In particular, how did the two project-internal milestones on September > > 15 and October 7 go? > > > > Do you mind updating #6414 with the status? > > > > The overall deadline is on November 1, which is in 3.5 weeks. How > > realistic do you think it is that we'll have automated bridge > > reachability tests by then? > > > > Thanks, > > Karsten > > > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBCgAGBQJQeJ4pAAoJEKOttnos24s1GlkP/1ODucFe9q9a7KaKisgVqBcx Cg1X3yGI+dsWvKtMwzgNqT/hcS+95GAYQiaa8fDlbiLY3nCWEsGYSOTjsbAmp04O qPCdBiSFE4VqiZy4PYUHUkf8qZTYkXqAiMnyuuLVJINJ+BHLAdwEmEWB77ncQUxV PvG6sxQ3GY64ZqRaSHwudf3H8mEPN/tNWRIXubnvAGM16EV0hWb51uORwo6CG86m lKwpBqbSX5uDgG+B6HCeZps0Zv7cDypI3dtdd8hJHncUpFmfuj2y+Oad1M5DVbMC 4UMvCv/4tMPoPyjTYjWgLQRHEFiMdMwQcdPntnF6XqUBMLvd6v1fk1IxaaqNCDah JWxK8ymUu90FLmmiyaSHQE/tfkFoEDCXz3XzR8xQA1zI2aq0hHdxA+G2JmW8yadQ wt0N4EeCzK154r4+YRC1eJLrsSOhmA7SaAE6A+/iEOgasdMY0YWchTDnlUbUB5iv acbMVITSGqmcRMos5xi450V98Qw1nW3UoBj5wi8uh0BWqnhqfomqL6oloQEdgAze MqtCZ91adKKXntIsEZW0C/klBYg5mZIxdtmAW5eO/j+ogLB/L0/hX4/TZ6OH5i/N 7jJWAF0MbSWI2Q1Gk+oFFulib5m3XWzqg5Yq/+8VfE5n1R5xM1FgBwC932SWLOFZ BV5c9ntZ+iJlbhpdL5yJ =jk13 -----END PGP SIGNATURE-----

4 5

Trip Report: GSoC Mentor Summit
by Damian Johnson 22 Oct '12

22 Oct '12

As per conferencing tradition Friday was spent on travel and meeting the other attendees. Some of the highlights for me were... * David from KDE Besides demoing some KDE eye candy we discussed their project infrastructure. KDE is a federation of smaller projects and had over sixty students this year (ten times the number mentored by Tor). Their project's scale has led to some unusual infrastructure decisions. For instance, they have a partly decentralized git infrastructure where pushes go to a single master host and pulls are from any of several mirrors. The config they use to do this leads to some... odd behavior. For instance a 'git pull' updates your tracking branch but not the origin branch reference. The result is that to do a pull for realz you need to call *both* pull and fetch. No doubt they also get fun behavior from mirroring delays... We also talked a bit about post-review and defaults they could set to better support their setup. KDE has the largest public ReviewBoard instance, but the above git setup makes it a bit confusing to use. * Sukhbir from Debian In 2011 Sukhbir applied to us for GSoC to work on TorBirdy. We loved his proposal, but due to prior commitments he ended up working with Debian instead. Since then he has become a GSoC mentor for Debian and involved with the Tor by implementing his earlier proposal for TorBirdy. Sukhbir's interested in getting even more involved with Tor so we discussed other projects that might interest him, and ways that we could better publicize TorBirdy on our site. * Arc from Python When I found out that Python had a mentor at the summit I made a mental note to hunt him down and ask about packaging best practices. After an unexpected discussion about rugby I found out that it's actually easy to support both python's 2.x and 3.x series by including a 2to3 conversion at build time. This can be done via either distutils or distribute... distutils: http://python3porting.com/2to3.html#running-2to3-on-install distribute: http://packages.python.org/distribute/python3.html I also asked him to look into his crystal ball for when python 3 would take over the world and he said 'Next year. Ubuntu and Fedora are ready and willing to make the switch. The last main holdout is Gnome. They tried to migrate but work there isn't finished yet.' ======================================== Saturday was the first day of the unconference. After an amusingly confused attempt to have each of the couple hundred attendees shake each other's hand there were sessions. Some were a little interesting, but I spent more time on the hallway track since that's the real benefit of the summit. The only useful tidbits I got from the talks were... - Do outreach early. The successful GSoC students who stick around tend to be the ones that get involved before the application phase. We should try harder to recruit college students to hack on tor, with the carrot that this'll give them a leg up when applying for the program. OpenHatch (http://openhatch.org/) might be something to look into for this. This would be a nice task for a community manager if we get one. - Google Code In (https://code.google.com/gci/) is a program somewhat similar to GSoC where highschool students become involved with open source. Last year they had 18 organizations and this year they're narrowing it down to 10. I was already highly tentative about having us apply and now that I've heard more I'm sure we don't have enough bandwidth for the hand-holding this would require. As for the hallway track... * Adriano and Luis from Umit Last year Adriano showed me Open Monitor (http://www.openmonitor.org/) a censorship detector written in python. Sounds familiar? I thought so too, and tried a few times to get them to talk with Ooni Probe and vice versa without success. My impression is that they're UI developers (a skillset we sorely lack in the tor project) with a rather unscalable backend, while Ooni Probe's backend is far more mature but lacks any sort of UI for rendering real time censorship information. I made another stab at getting the two projects to talk, after which the meeting took a weird turn with Adriano arguing that 'some censorship is good'. Evidently they decided that Open Monitor won't look for censorship concerning 'porn or terrorism'. I argued that this was a slippery slope and that censorship monitoring shouldn't try to pass a moral judgment on the content being censored, but after a time it was clear that we were talking past each other. I still think that we should leverage their UI expertise, but that's up to the Ooni Probe devs. * Open Source Lab Met with a couple administrators from the OSU's Open Source Labs (https://osuosl.org/) They provide hosting for several of the largest open source projects including Apache and the Linux Foundation. Mostly we talked about amusing legal threats they get for hosting the phpBB project. Evidently lawyers are quite skilled at clicking the 'this is a phpBB forum' link followed by 'hosted by the OSL' before sending their angry emails. We also talked a bit about setting up non-exit relays. They might be pretty receptive to this if we want to follow up. * Sumana and Rob from Wikimedia Unsurprisingly Wikipedia occasionally has issues with spammers using Tor. We talked about some possible options, such as requiring accounts for Tor users to edit with a sort of proof of work in account creation to make ban evasion more of a pita. * Terri from Python Mailman 3 is coming, and with it an interface that *doesn't* look like it came from the 1980s! Most importantly for us, the new version of Mailman provides a forum interface, letting email and forum users communicate by whichever method they prefer. This would be a good answer to our forums ticket (https://trac.torproject.org/3592) She estimates that it'll be ready in six months or so. ======================================== Flying out on Sunday cut my day short, but there was one session that I thought was interesting. Gnome and Wikimedia are launching a program similar to GSoC to encourage more women to get involved with open source. It runs later this winter. One gotcha is that Google's not involved so mentoring orgs need to cover the $5k stipend. I like the idea. Is this something we want to take part in? If so then I'd be happy to administer the non-financial parts of it. Cheers! -Damian

2 1

Re: [tor-dev] Extending Pyonionoo to provide statistics
by Damian Johnson 19 Oct '12

19 Oct '12

Hi Karsten. This is actually really good timing. As you probably noticed from tor-commits@ I just pushed support for network status documents (v2 and v3 votes, consensus, and microdescriptor consensuses). It's the combination of a month of work from Ravi and a couple more from me. Stem's descriptor parsing functionality is one of its most well developed features and I'm anxious for it to start to get some users. The feature gap between stem and metrics-lib is now pretty small, so I'd love for it to start to take over some of metrics-lib's responsibilities (and hopefully in turn get direct involvement from you and Sathyanarayanan so it'll better meet your needs). > We would start with daily users, both direct and bridge users, and later add > aggregate statistics about relays and bridges, and after that torperf > performance statistics. Gotcha. As I understand it the document types that stem is still missing are... * microdescriptors * bridge pool assignments * exit list entry * torperf output So what you'll need from me is primarily the ability to parse torperf output? Or is there another document type that I'm missing? > I think we'd need Damian's help for the descriptor-parsing side Happy to help, though I'm not entirely sure yet what kind of additional parsing support you'll need. > Sathya, Damian, can you already say how much of 2013 you will be around > for doing Tor stuff and how much time you think you could spend on this > project? For my part a fair bit. This is, of course, a hobby that I do outside of a full time job so work and life might get in the way. That said, any project that includes collaborating with other developers to make use of stem goes to the top of my todo list. > I'm thinking one ticket for implementing > usage statistics in Pyonionoo, a second ticket for the censorship > detector integration which depends on the first ticket, and two more > tickets for network statistics and torperf statistics integration into > Pyonionoo. I'd probably suggest the first two tickets only for trying > to get funding from sponsor F year 3 and save the other two tickets for > later. Hmm, aren't those last two tickets the only ones where I'd be involved? It sounds like this project mostly concerns pyonionoo so guess we should wait for input from Sathyanarayanan... Cheers! -Damian PS. Sathyanarayanan: I was gonna ping you separately but might as well hijack this thread - would you mind giving stem's new networkstatus module a try in pyonionoo? You can find documentation for it at... https://stem.readthedocs.org/en/latest/stem.descriptor.html#module-stem.des… ... let me know if you run into any issues! PPS. Grrr, their pydocs have some rendering bugs... >:( On Fri, Oct 12, 2012 at 9:57 AM, Karsten Loesing <karsten(a)torproject.org> wrote: > Hi Sathya, hi Damian, > > I have been working on improving bridge usage statistics in the past > weeks, and I discussed my results with George in the context of > improving the censorship detector. We concluded that we need to write a > fair amount of new code and that it would be nice to integrate this new > code into Pyonionoo. > > Let me explain in more detail: I'm not yet done with the bridge usage > analysis, but early results are that we can calculate daily bridge users > similar to how we calculate daily direct users. We need a more precise > implementation than what we have in metrics-web though. So, we either > need to redesign the complex beast called metrics-web, or start from > scratch, ideally in a programming language that attracts more potential > contributors. > > The censorship detector takes the output of metrics-web's daily direct > usage numbers and tries to detect sudden drops or increases in usage. > It would be good to also look at bridge usage for censorship events, and > it would be quite important to have a tighter integration of the > censorship detector into the usage-calculating process and look at raw > numbers there. Right now, the vast majority of events that the > censorship detector reports are just false positives. > > We first thought about implementing this in a new Python project that > has several modules: the first module would calculate daily users, the > second would determine possible censorship events, the third would > export data via JSON, the fourth would graph results on a website, and > the fifth would send out email notifications. > > However, that codebase would share a lot of functionality with > Onionoo/Pyonionoo. We could instead extend Pyonionoo and Atlas/Compass > to not only provide network status data, but also statistics about the > Tor network and its usage. We would start with daily users, both direct > and bridge users, and later add aggregate statistics about relays and > bridges, and after that torperf performance statistics. Once we're > there we can retire a large part of metrics-web. > > This project is huge, which is why I'm thinking we should apply for > funding. I expect the project to take at least 6 months of developer > time, in addition to the time to make Pyonionoo a full replacement of > Onionoo. Before we ask for money, we should have a very rough idea > who's going to work on this project. Once we have the money, we'll > actually have to do it, and I can't do this all by myself. > > I think we'd need Damian's help for the descriptor-parsing side and > general help with Python and Sathya to make Pyonionoo modular enough to > provide all these different features. I'd either write or help write > the user-counting code, and I think George would help with integrating > and improving the existing censorship-detecting code. > > Sathya, Damian, can you already say how much of 2013 you will be around > for doing Tor stuff and how much time you think you could spend on this > project? If the answer is "maybe not very much", by all means, please > say so. > > If there's at least some interest in this project, I'd create Trac > tickets in the next few days. I'm thinking one ticket for implementing > usage statistics in Pyonionoo, a second ticket for the censorship > detector integration which depends on the first ticket, and two more > tickets for network statistics and torperf statistics integration into > Pyonionoo. I'd probably suggest the first two tickets only for trying > to get funding from sponsor F year 3 and save the other two tickets for > later. > > Thanks, > Karsten

3 2

Even more notes on relay-crypto constructions
by Nick Mathewson 19 Oct '12

19 Oct '12

I should share with the list an update of where I am with a design for an improved relay crypto protocol. For background and motivation, please see the last thread on the topic [Prop202]. There are three main questions remaining for me in choosing among new relay crypto protocols. Basically, they are: "Am I comfortable with this system?", "Among systems I'm comfortable with, how good is this system?", and "Do I know how to implement this system?" Unfortunately, the stuff I am currently comfortable with and know that I could implement is not nearly as good as the stuff that I'm _nearly_ comfortable enough to use and I don't know how to implement. Let's talk about some designs in detail, using the same terminology as proposal 202. Note: As usual, this is probably largely wrong. If you find it on a mailing list archive years from now, please don't try to learn cryptography from it. What I'm looking for most right now is places where my reasoning is wrong, places where I'm mistaken about what tradeoffs I need need to make, and places where there's research that I should have read or remembered. I. MAC-AND-PAD DESIGNS These are the least involved to implement: all you need is an AEAD construction and a PRNG. Those are both pretty well-understood: you can build a good PRNG out of any stream cipher, and you can build the AEAD mode out of a stream cipher and a MAC, or out of various other constructions. As a wrinkle: it would be good to have a system that generates "two MACs for the price of one." That's because in our topology, we need the ability to address relay cells to any node in the circuit, and one easy way to do that here is to have a construction that generates two MACs, and uses one for "cells that should arrive here" and another for "cells that we should relay". We also want shorter MACs: 16 bytes per hop is excessive for our needs. Authenticators like GCM whose output is the raw result of a polynomial evaluation aren't safe to truncate: Ferguson [Ferguson] has a result showing that it's easier than it should be to perform forgery against truncated-MAC GCM constructions. On the other hand, polynomial authenticators seem safe to truncate (generally) if the encryption step comes _after_ the polynomial evaluation. See for example [Poly1305-Trunc]'s discussion of truncated Poly1305 -- I hope it's right. So to be concrete, let me suggest a few modes of operation. I believe I'm competent to implement these: AES-GCM+AES: AES-GCM, except that we add (modulo 2^128) the output of AES(nonce) to each GCM output, in hopes that it will make GCM safe to truncate. (This is probably crazy somehow.) AES-CTR + HMAC-SHA512/256. AES-CTR + Poly1305. Poly1305 requires nonces, but we can use a counter for those. Salsa20 + Poly1305. For a padding PRNG, we could use any stream cipher we like. In each case, we'd want to authenticate not only the content of the cell, but also the previous cell's authenticator and the position of the cell within the stream. AES-GCM+AES is going to be the fastest on CPUs with specific support for AES and GCM, but not on other architectures until/unless more chips grow instructions specialized for AES and GF(2^128). Salsa20+Poly1305 should be fastest on other architectures. This entire category of designs still has the problems that it had before: it leaks the length of the circuit to the last node in the circuit, and consumes a fairly large portion of each cell (for one truncated mac per hop). Further, it's going to be a bit fiddly (though not impossible) to get it to work with rendezvous circuits. II. WIDE-BLOCK DESIGNS Turn we now to designs where we encrypt each block of the cipher using a 509-byte SPRP, such that each block's SPRP is keyed dependently on the original key and on the encrypted value of the previous block. There be dragons here. Let's talk about some ways we could build that. I'll start by talking about wide-block encryption, then talk about getting the "unrecoverability" property where any missing or corrupted block makes all future blocks unrecoverable. The wide-block SPRP I've used before is LIONESS [Bear-Lion]. It requires two keyed hash operations and two stream cipher operations, so it's going to be at best half the speed of the mac-and-pad designs above: a little worse, maybe, since it forces us to rekey with every block. Might that be acceptable? Right now, AES is not a huge fraction of our runtime, and a relay does AES on each cell three times (TLS,relay crypto,TLS) and SHA1 on each cell twice (TLS,TLS). An exit does AES on each cell twice (TLS,relay) and SHA1 on each cell twice (TLS,relay). So for relays, with a naive SHA1-AES LIONESS, we'd be increasing the stream-cipher operations by 25% and the digests by 100%. On an exit, we'd be increasing the stream-cipher operations by 33% and the digests by 100%. If we were to go with LIONESS, we'd surely want to look into faster, better keyed-hash algorithms than SHA1. I don't know whether one of the polynomial MACs above would be good enough, or whether we need other cryptographic digest properties that those don't give us and we'd need to turn to SHA256 or something. I've heard some suggestions that we should look into BEAR or LION instead, but I'm leery of the idea. They are faster, requiring one fewer application of their primitives, but they are PRPs, not SPRPs -- meaning I think that they don't give you Ind-CCA2 [Bear-Lion, section 6]. I don't know a way to exploit this in the Tor network, but deviating from an ideal block cipher seems to me like one of those ideas that's practically begging for disaster. Can we get faster than LIONESS? Indeed we can! There are a pile of constructions. If I understand correctly, and I'm not missing anything, the ones we might want to use fall into two broad categories: those that (like LIONESS) split the message into a short bit and a long bit, then treat them differently; and those that apply a transformation on the message, encrypt the message, and transform it again. The first category (split, then frob each part based on the other once or twice) would appear to be more of a patent minefield, thanks to the XCB patent. The ones to look at here are [HCTR] and [HCH], which require two applications of a universal hash function (can be polynomial-based) and approximately one encryption pass. HCH is a little more unlike XCB. Neither is exactly trivial. We could build either one out of whatever field and stream cipher we wanted, as above, though bitsliced AES-CTR wouldn't be efficient in this case: we aren't generating enough stream at once for bitslicing to pay off. The second category (frob, encrypt, frob) is pretty elegant IMO. The best-explained of these I've seen so far are in a paper by Palash Sarkar [Efficient-Tweakable], though the earlier TET construction [TET] might also be cool. For these, you need an invertible block-wise (Almost) (Xor-)Universal hash function, typically implemented with GF(2^128). I'm not sure if you could use a different field. The multiplication operations here appear to be multiplication by a primitive element, and multiplication by a per-key element. The encryption step can be realized with a somewhat unorthodox counter-mode stream cipher, or a ciphertext-stealing ECB approach. I don't know what you'd need to do to substitute in an orthodox stream cipher for the one used in iHCH. Sarkar seems to see iHCH as a successor to HCH, which is a little worrisome given that HCH is a spiritual descendant of the patented XCB, but to me the two constructions (HCH, iHCH) look practically nothing alike except for their use of a counter mode step. I am a little intimidated by the idea of trying to implement one of these ciphers. Above, I haven't taken one of our requirements into account: that any change to a single cell must make all future cells unrecoverable. There are modes that are supposed to prevent this, and applying them to a decent wide-block cipher might solve the issue. IGE is one of them [IGE], but it turns out to be broken by an attacker who knows some plaintext. The Accumulated Block Chaining [ABC] construction is supposed to fix that; I'm not too sure whether it's correct or efficient. As a blunt-force approach to the problem, we could rekey between each block, using new keys based on a MAC of the last block. This would be pretty inefficient for primitives that need any serious amount of per-key computation. Finally, we could look into constructions that produce an extra secret output incidental to their regular operation, and which are easily rekeyed for each operation. In this whole field, we need to keep an eye out for the patents on CMC, EME, and XCB. "Joy." III. CHOICE OF CIPHERS AND OTHER PRIMITIVES I am hearing exactly two recommendations for encryption primitives nowadays: AES and Salsa20 (and its family). Nobody's recommending anything else as far as I can tell. AES is still the IBM of the block cipher world, which "Nobody Ever Got Fired For Using." It's a bit of a pain to use it in software, though. Its most obvious implementations have timing attacks due to table lookups [DJB-Timing]. OpenSSL has three fast x86 implementations: one of them uses vector permutations, one uses bit-slicing, and one uses the aesni instructions to invoke the chip's built-in AES capabilities. On high-end Intel chips, these take about 12-25, 7-9, and 1.5 cycles per byte respectively. The bitsliced one really needs to encrypt 4KiB at a time (yes, kibibytes), which makes it fine for counter mode but not so good for CBC. (Those numbers are from the OpenSSL source.) Key setup times are cheap for all of these (I think!), but the bitsliced implementation gets expensive if you change the key (or the stream position). Salsa20 (and its descendant, ChaCha) is the Obvious Second Choice for people who don't want to use AES. It is faster than AES on every platform except those with hardware support for AES; 4-7 cycles per byte seems typical for high-end Intel stuff. You would probably have to go out of your way to implement it in a way that was vulnerable to timing side-channel attacks. The only arguments for its insecurity that I'm aware of are that although it has fewer known flaws than AES, it has received less attention than AES, therefore is likelier to _unsuspected_ problems. The counterargument there is that there _are_ several dozen cryptographers who have tried to attack it (or attack reduced-round variants of it), several of whom have also done successful results against AES. [DJB-Comm] Per-key setup time is basically nonexistent. There are a lot of constructions out there that want to do multiplication in GF(2^128) as a basic operation. That turns out to be less than totally straightforward, though. On newer intel chips, you've got a "multiply without carry" instruction that can supposedly let you get to around 2 cycles per byte. On other platforms, you're reduced to worse trickery to try to get good performance without timing side-channels, for an amount of trickery dependent on what operation you're trying to do exactly. The easiest GF(2^128) operation to implement safely and quickly is multiplication by a known compile-time value. (It's easy enough that even I could do it.) Next easiest is multiply a large number of values by the same run-time-fixed value -- you do precomputation on the value in order to generate some tables. (The scary, fast variants use big tables; the still-a-little-scary variants use 256-byte tables, and get performance on high-end Intel boxes around 7-10 cycles per byte when doing GCM.) Full-on multiplication of two arbitrary GF(2^128) values is slowest still. As a further wrinkle with GF(2^128), OpenSSL doesn't seem to actually expose its "multiply in GF(2^128)" functions as far as I can tell: we would need to snarf such code from elsewhere. Oh! ARMv8 has an optional crypto instruction set that supports AES, SHA256, and GF(2^128) multiplication [ARMv8]. If it looks like most of the ARMs we care about are going to get it, that could factor into our planning. IV. HOW MUCH DOES PERFORMANCE MATTER HERE ANYWAY? A fine question. The right place too look would IMO be at profiles on a busy non-exit non-AESNI server, to see how much time is spent in AES there. Make sure you look at all the AES variants, since it's not uncommon to see more than one used at once. A ridiculously large amount of that time will be in bn_sqr4x_mont and its unindicted co-conspirators , but expect this to get cut a great deal when we move to ECC for TLS DHE and for circuit extension. So if we're willing to make a lot of assumptions, then you can figure out a worst-case performance impact like this. Take the fraction of the time spent in bn_and_friends and remove them from the profile. (Assume that circuit creation dominates over TLS renegotiation because hey why not.) Then assume that the AES calls used for relay crypto's counter mode turn into whatever new relay encryption thing we're proposing. See how much that change costs Tor's performance overall. V. ACKNOWLEDGMENTS Thanks to everybody who's schooled me about this stuff, especially Daniel J. Bernstein, Ian Goldberg, and Robert Ransom. Assume that the good ideas are here due to their patience and that the bad ideas are here due to my slowness on the uptake. Thanks to Susan Born for a much-needed copy-edit. Assume that every correctly structured sentence is one she fixed, and that the bad ones I messed up after her editing. VI. References [Prop202] https://lists.torproject.org/pipermail/tor-dev/2012-June/003649.html [Ferguson] Niels Ferguson, "Authentication weaknesses in GCM", 2005. http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/comments/CWC-GCM/Fergu… [DJB-Timing] Daniel J. Bernstein, "Cache-timing attacks on AES", 2004. http://cr.yp.to/papers.html#cachetiming [DJB-Comm] Daniel J. Bernstein, personal communication [Poly1305-Trunc] http://osdir.com/ml/encryption.poly1305/2005-09/msg00007.html [ARMv8] "ARMv8 Instruction Set Overview", 2011, http://board.flatassembler.net/download.php?id=5698 [Bear-Lion] Ross Anderson and Eli Biham. "Two Practical and Provably Secure Block Ciphers: BEAR and LION". http://www.cl.cam.ac.uk/~rja14/Papers/bear-lion.pdf [Efficient-Tweakable] Palash Sarkar, "Efficient Tweakable Enciphering Schemes from (Block-Wise) Universal Hash Functions". 2008. http://eprint.iacr.org/2008/004.pdf [IGE] Gligor and Donescu, "On Message Integrity in Symmetric Encryption" has the good IGE analysis: http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/proposedmodes/ige/ige-… The original IGE proposal was by Carl Campell in an old NIST publication that I can't find online; the paper above has a reference for it if you want to chase it more. [ABC] Lars R. Knudsen, "Block Cipher Chaining Modes of Operation". http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/proposedmodes/abc/abc-… [HCTR] Peng Wang, Dengguo Feng, and Wenling Wu. "HCTR: A variable-input-length enciphering mode." 2005. http://delta.cs.cinvestav.mx/~debrup/hctr.pdf [HCH] Debrup Chakraborty and Palash Sarkar. "HCH: A new tweakable enciphering scheme using the hash-encrypt-hash approach." http://biblioteca.cinvestav.mx/indicadores/texto_completo/cinvestav/2006/13… [TET] Shai Halevi. "Invertible universal hashing and the TET encryption mode." 2007. http://www.iacr.org/archive/crypto2007/46220405/46220405.pdf

5 10

Proposal: Increase Acceptable Consensus Age
by Mike Perry 17 Oct '12

17 Oct '12

Also available at: https://gitweb.torproject.org/user/mikeperry/torspec.git/blob/tolerate-old-… ---------------------------------- Title: Increase Acceptable Consensus Age Author: Mike Perry Created: 01-10-2012 Status: Open Target: 0.2.4.x+ Overview This proposal aims to extend the duration that clients will accept old consensus material under conditions where the directory authorities are either down or fail to produce a valid consensus for an extended period of time. Motivation Currently, if the directory authorities are down or fail to consense for 24 hours, the entire Tor network will cease to function. Worse, clients will enter into a state where they all need to re-bootstrap directly from the directory authorities, which will likely exacerbate any potential DoS condition that may have triggered the downtime in the first place. The Tor network has had such close calls before. In the past, we've been able to mobilize a majority of the directory authority operators within this 24 hour window, but that is only because we've been exceedingly lucky and the DoS conditions were accidental rather than deliberate. If a DoS attack was deliberately timed to coincide with a major US and European combined holiday such as Christmas Eve, New Years Eve, or Easter, it is very unlikely we would be able to muster the resources to diagnose and deploy a fix to the authorities in time to prevent network collapse. Description Based on the need to survive multi-day holidays and long weekends balanced with the need to ensure clients can't be captured on an old consensus forever, I propose that the consensus liveness constants be set at 3 days rather than 24 hours. This requires updating two consensus defines in the source, and one descriptor freshness variable. The descriptor freshness should be set to a function of the consensus freshness. See Implementation Notes for further details. Security Concerns: Using an Old Consensus Clients should not trust old consensus data without an attempt to download fresher data from a directory mirror. As far as I could tell, the code already does this. The minimum consensus age before we try to download new data is two hours. However, the ability to accept old consensus documents does introduce the ability of malicious directory mirrors to feed their favorite old consensus document to clients to alter their paths until they download a fresher consensus from elsewhere. Directory guards (Proposal 207) may exacerbate this ability. This proposal does not address such attacks, and seeks only a modest increase in the valid timespan as a compromise. Future consideration of these and other targeted-consensus attacks will be left to proposals related to ticket #7126[1]. Once those proposals are complete and implemented, raising the freshness limit beyond 3 days should be possible. Implementation Notes There appear to be at least three constants in the code involved with using potentially expired consensus data. Two of them (REASONABLY_LIVE_TIME and NS_EXPIRY_SLOP) involve the consensus itself, and two (OLD_ROUTER_DESC_MAX_AGE and TOLERATE_MICRODESC_AGE) deal with descriptor liveness. Two additional constants ROUTER_MAX_AGE and ROUTER_MAX_AGE_TO_PUBLISH are only used when submitting descriptors for consensus voting. FORCE_REGENERATE_DESCRIPTOR_INTERVAL is the maximum age a router descriptor will get before a relay will re-publish. It is set to 18 hours. OLD_ROUTER_DESC_MAX_AGE is set at 5 days. TOLERATE_MICRODESC_AGE is set at 7 days. The consensus timestamps are used in networkstatus_get_reasonably_live_consensus() and networkstatus_set_current_consensus(). OLD_ROUTER_DESC_MAX_AGE is checked in routerlist_remove_old_routers(), router_add_to_routerlist(), and client_would_use_router(). It is my opinion that we should combine REASONABLY_LIVE_TIME and NS_EXPIRY_SLOP into a single define, and make OLD_ROUTER_DESC_MAX_AGE a function of REASONABLY_LIVE_TIME and FORCE_REGENERATE_DESCRIPTOR_INTERVAL: #define REASONABLY_LIVE_TIME (3*24*60*60) #define NS_EXPIRY_SLOP REASONABLY_LIVE_TIME #define OLD_ROUTER_DESC_MAX_AGE \ (REASONABLY_LIVE_TIME+FORCE_REGENERATE_DESCRIPTOR_INTERVAL) Based on my review of the above code paths, these changes should be all we need to enable clients to use older consensuses for longer while still attempting to fetch new ones. 1. https://trac.torproject.org/projects/tor/ticket/7126 -- Mike Perry

2 1

tor decrypt packet
by esolve esolve 16 Oct '12

16 Oct '12

HI, I capture packets on the tor client using tcpdump and I want to decrypt the captured packets for analysis. I think there are two steps 1 obtain the session keys 2 use some tools to decrypt the packets Are there any ways, tools, methodology to decrypt the packets? thanks!

1 0

Re: [tor-dev] Proposal 211: Internal Mapaddress for Tor Configuration Testing
by Nick Mathewson 15 Oct '12

15 Oct '12

On Thu, Oct 11, 2012 at 5:38 AM, Mike Perry <mikeperry(a)torproject.org> wrote: > Also at: > https://gitweb.torproject.org/user/mikeperry/torspec.git/blob/mapaddress-ch… > > --------------------------------------------------------------- > > Title: Internal Mapaddress for Tor Configuration Testing > Author: Mike Perry > Created: 08-10-2012 > Status: Open > Target: 0.2.4.x+ > > > Overview > > This proposal describes a method by which we can replace the > https://check.torproject.org/ testing service with an internal XML > document provided by the Tor client. > > Motivation > > The Tor Check service is a central point of failure in terms of Tor > usability. If it is ever out of sync with the set of exit nodes on the > Tor network or down, user experience is degraded considerably. Moreover, > the check itself is very time-consuming. Users must wait seconds or more > for the result to come back. Worse still, if the user's software *was* > in fact misconfigured, the check.torproject.org DNS resolution and > request leaks out on to the network. > > Design Overview > > The system will have three parts: an internal hard-coded IP address > mapping (127.84.111.114:80), a hard-coded mapaddress to a DNS name > (selftest.torproject.org:80), and a DirPortFrontPage-style simple > HTTP server that serves an XML document for both addresses. The use of XML and HTTP here are both reasons for some unhappiness. Both of them pull in a fair amount of complexity that I'd prefer not to need. (Yes, Tor already has a sort of an HTTP implementation, but at least clients aren't currently required to run what amounts to a local HTTP server.) I seriously wonder whether the benefits of HTTP (easier to access from within a locked-down web browser environment) aren't actually the _defects_ of HTTP here: it's easier to poke it from a web page. I understand that your design takes some steps to prevent browser-based attacks on this, but I'm not currently sure how to become sure that that it solves them all. Right now, I'm nervous. > Upon receipt of a request to the IP address mapping, the system will > create a new 128 bit randomly generated nonce and provide it > in the XML document. > > Requests to http://selftest.torproject.org/ must include a valid, > recent nonce as the GET url path. Upon receipt of a valid nonce, > it is removed from the list of valid nonces. Nonces are only valid > for 60 seconds or until SIGNAL NEWNYM, which ever comes first. So, I'm not totally sure what the nonce field is for. The idea as I understand it is that when you connect to the IPv4 address, you get a nonce, and later when you connect to the hostname, you provide that nonce, and Tor tells you "yes" if you gave it the same nonce. What does that protect against? My first thought is that you're trying to prevent the case where a malicious local DNS server maps "selftest.torproject.org" to some IP address in their control, and then just runs a server at that IP address to say "yes I'm Tor". But that doesn't make sense, since you could just make one of those that said "yes I'm Tor" no matter what you say for the nonce. Also, how useful is the followup DNS check? If it's checking that DNS leaks aren't happening... You're going to need torbrowser or something of equivalent complexity for this to work at all; isn't it easier then for torbrowser to make sure that it set up SOCKS ? > The list of pending nonces should not be allowed to grow beyond 10 > entries. This means that any webpage could flush out the list of pending nonces. Does that matter? > The timeout period and nonce limit should be configurable in torrc. > > Design: XML document format for http://127.84.111.114 > [...] > Security Considerations > > XML was chosen over JSON due to the risks of the identifier leaking > in a way that could enable websites to track the user[1]. Well, that's a nuclear-powered-flyswatter! If I read that page right, the problem with using JSON is that it can be parsed and executed as Javascript, and the advantage of XML is that it's unlikely to be syntactically correct javascript, then maybe instead we should If that's the issue, I'd strongly suggest that instead of going with a more complex data format, we could add a layer of encoding over the json, or use an even simpler format. > Because there are many exceptions and circumvention techniques > to the same-origin policy, we have also opted for strict controls > on dns-nonce lifetimes and usage, as well as validation of the Host > header and SOCKS4A request hostnames. Of course, this all comes down to the fact that we're using http. Can we spell out why we need HTTP for this? > 1. http://www.hpenterprisesecurity.com/vulncat/en/vulncat/dotnet/javascript_hi… > -- Nick

2 3

Re: [tor-dev] Proposal 210: Faster Headless Consensus Bootstrapping
by Nick Mathewson 15 Oct '12

15 Oct '12

On Thu, Oct 11, 2012 at 5:32 AM, Mike Perry <mikeperry(a)torproject.org> wrote: > Also at: > https://gitweb.torproject.org/user/mikeperry/torspec.git/blob/consensus-boo… > > ------------------------------------------------------------------------- > > Title: Faster Headless Consensus Bootstrapping > Author: Mike Perry > Created: 01-10-2012 > Status: Open > Target: 0.2.4.x+ > > > Overview and Motiviation > > This proposal describes a way for clients to fetch the initial > consensus more quickly in situations where some or all of the directory > authorities are unreachable. This proposal is meant to describe a > solution for bug #4483. > > Design: Bootstrap Process Changes > > The core idea is to attempt to establish bootstrap connections in > parallel during the bootstrap process, and download the consensus from > the first connection that completes. > > Connection attempts will be done in batches of three. Only one > connection will be performed to one of the canonical directory > authorities. Two connections will be performed to randomly chosen hard > coded directory mirrors. I misread this paragraph at first. I thought you were suggesting 3 parallel directory downloads when in fact you were discussing 3 parallel TLS connections, with only the first one that finishes actually getting a download. [...] > Design: Fallback Dir Mirror Selection Out of scope for this proposal; relevant for proposal 206. > Performance: Additional Load with Current Parameter Choices > > This design and the connection count parameters were chosen such that > no additional bandwidth load would be placed on the directory > authorities. In fact, the directory authorities should experience less > load, because they will not need to serve the consensus document for a > connection in the event that one of the directory mirrors complete their > connection before the directory authority does. To be clear, it's the part of this proposal that's shared with proposal 206 (directory sources) that would lower load on the authorities. > However, the scheme does place additional TLS connection load on the > fallback dir mirrors. Because bootstrapping is rare and all but one of > the TLS connections will be very short-lived and unused, this should not > be a substantial issue. How do we know that bootstrapping is rare? > The dangerous case is in the event of a prolonged consensus failure > that induces all clients to enter into the bootstrap process. In this > case, the number of initial TLS connections to the fallback dir mirrors > would be 2*C/100, or 10,000 for C=500,000 users. If no connections > complete before the five retries, this could reach as high as 50,000 > connection attempts, but this is extremely unlikely to happen in full > aggregate. > > However, in the no-consensus scenario today, the directory authorities > would already experience C/9 or 55,555 connection attempts. The > 5-retry scheme increases their total maximum load to about 275,000 > connection attempts, but again this is unlikely to be reached > in aggregate. Additionally, with this scheme, even if the dirauths > are taken down by this load, the dir mirrors should be able to survive > it. This looks like an argument of the form "The outcome would be horrible, but the current outcome is also horrible, so we wouldn't break stuff any worse." Right? I wonder if in this case the answer isn't to actually back off from fetching after N minutes or M servers, like a sane system. Or to treat "hey, that's not a good consensus!" as different from "couldn't connect to directory server" in terms of what it means for how we back off. -- Nick

2 1

Proposal 207: Directory guards
by Nick Mathewson 15 Oct '12

15 Oct '12

Filename: 207-directory-guards.txt Title: Directory guards Author: Nick Mathewson Created: 10-Oct-2012 Status: Open Target: 0.2.4.x Motivation: When we added guard nodes to resist profiling attacks, we made it so that clients won't build general-purpose circuits through just any node. But clients don't use their guard nodes when downloading general-purpose directory information from the Tor network. This allows a directory cache, over time, to learn a large number of IPs for non-bridge-using users of the Tor network. Proposal: In the same way as they currently pick guard nodes as needed, adding more guards as those nodes are down, clients should also pick a small-ish set of directory guard nodes, to persist in Tor's state file. Clients should not pick their own guards as directory guards, or pick their directory guards as regular guards. When downloading a regular directory object (that is, not a hidden service descriptor), clients should prefer their directory guards first. Then they should try more directories from a recent consensus (if they have one) and pick one of those as a new guard if the existing guards are down and a new one is up. Failing that, they should fall back to a directory authority (or a directory source, if those get implemented-- see proposal 206). If a client has only one directory guard running, they should add new guards and try them, and then use their directory guards to fetch multiple descriptors in parallel. Discussion: The rule that the set of guards and the set of directory guards need to be disjoint, and the rule that multiple directory guards need to be providing descriptors, are both attempts to make it harder for a single node to capture a route. Open questions and notes: What properties does a node need to be a suitable directory guard? If we require that it have the Guard flag, we'll lose some nodes: only 74% of the directory caches have it (weighted by bandwidth). We may want to tune the algorithm used to update guards. For future-proofing, we may want to have the DirCache flag from 185 be the one that nodes must have in order to be directory guards. For now, we could have authorities set it to Guard && DirPort!=0, with a better algorithm to follow. Authorities should never get the DirCache flag.

3 6

[patch] fix cparser/firm compiler warnings
by Christian Grothoff 15 Oct '12

15 Oct '12

Hi! We're trying to compile Tor with cparser and got some warnings and an error. src/or/dirserv.c:803:1: error: declaration 'was_router_added_t dirserv_add_extrainfo(extrainfo_t*, const char**)' is incompatible with 'int dirserv_add_extrainfo(extrainfo_t*, const char**)' (declared at line 96:12) src/or/dirserv.c:81:14: warning: unnecessary static forward declaration for 'char* format_versions_list(config_line_t*)' [-Wredundant-decls] src/or/dirserv.c:2186:8: warning: statement is empty [-Wempty-statement] src/or/dirserv.c:2210:8: warning: statement is empty [-Wempty-statement] src/or/dirserv.c:2752:51: warning: statement is empty [-Wempty-statement] src/or/dirserv.c:3009:51: warning: statement is empty [-Wempty-statement] src/or/dirserv.c:3280:112: warning: statement is empty [-Wempty-statement] src/or/dirserv.c:3483:54: warning: statement is empty [-Wempty-statement] src/or/dirserv.c:3603:129: warning: statement is empty [-Wempty-statement] 1 error(s), 47 warning(s) The attached patch fixes those the warnings that are unambiguously due to inconsistent style ("if (cond) {body};" -- removing extra semicolon) and fixes the warning due to inconsistently declared a return type (once declared as 'enum', once as 'int'). Is there a better place for sending patches that do not really require broad discussion? Happy hacking! Christian

3 2

txtorcon 0.6
by meejah＠meejah.ca 12 Oct '12

12 Oct '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 txtorcon 0.6 is now tagged and released. txtorcon is a Twisted-based Python asynchronous implementation of the Tor control protocol. It includes a state-tracking abstraction, configuration abstraction, Twisted endpoint support for hidden services, 96%+ unit-test coverage and many examples. New in this release: . debian packaging (mmaker); . psutil fully gone; . *changed API* for launch_tor() to use TorConfig instead of args; . TorConfig.save() works properly with no connected Tor; . fix incorrect handling of 650 immediately after connect; . pep8 compliance; . use assertEqual in tests; . messages with embdedded keywords work properly; . fix bug with setup.py + pip; . issue #15 reported along with patch by Isis Lovecruft; . consolidate requirements (from aagbsn); . increased test coverage and various minor fixes; . https URIs for ReadTheDocs Code and built documentation may be obtained: https://github.com/meejah/txtorcon https://txtorcon.readthedocs.org/en/latest/index.html git clone git://github.com/meejah/txtorcon.git You may also install it directly: pip install txtorcon easy_install txtorcon The sha256sum output is: 4f939e217ea3149175bb1285fa7296edc02ac245a71a60ec77b9cf48511e991e txtorcon-0.6.tar.gz thanks, meejah -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/> iQEcBAEBAgAGBQJQd5uqAAoJEMJgKAMSgGmnTCkIAOGT6QaE+Q8jaBOedE79j/B3 QYMXh1SSfrvxn2x/utmjIgRbvCrCE+lzkxv9T9GXXjAhKSRN2xyyhRxq/Mbxvucd FWb2zHyZIjsTcc//veHYNpRNEWORl1ZIG2kU0muHQPpxY5ZAvtEkW+RA5zMJbhHp LF/JyltsAEu1Ja6HMFsfPmlLZiOtWBkBi3MgGGenUVJsScrtGRNCjWutGEkXxKXe 9pw2W0IGH5YmBAUiAuSW6q4cnyPQroqjSgO+RFfKMVGJNAaSqgUE8UCkcCifyjjM 2cDtGNA50kU0obT8QLgsiRyavePCQSfn9Jr1y8/Ck45Toa9a4FOQbBlOOyQSgrY= =bORU -----END PGP SIGNATURE-----

1 0

Proposal: Internal Mapaddress for Tor Configuration Testing
by Mike Perry 11 Oct '12

11 Oct '12

Also at: https://gitweb.torproject.org/user/mikeperry/torspec.git/blob/mapaddress-ch… --------------------------------------------------------------- Title: Internal Mapaddress for Tor Configuration Testing Author: Mike Perry Created: 08-10-2012 Status: Open Target: 0.2.4.x+ Overview This proposal describes a method by which we can replace the https://check.torproject.org/ testing service with an internal XML document provided by the Tor client. Motivation The Tor Check service is a central point of failure in terms of Tor usability. If it is ever out of sync with the set of exit nodes on the Tor network or down, user experience is degraded considerably. Moreover, the check itself is very time-consuming. Users must wait seconds or more for the result to come back. Worse still, if the user's software *was* in fact misconfigured, the check.torproject.org DNS resolution and request leaks out on to the network. Design Overview The system will have three parts: an internal hard-coded IP address mapping (127.84.111.114:80), a hard-coded mapaddress to a DNS name (selftest.torproject.org:80), and a DirPortFrontPage-style simple HTTP server that serves an XML document for both addresses. Upon receipt of a request to the IP address mapping, the system will create a new 128 bit randomly generated nonce and provide it in the XML document. Requests to http://selftest.torproject.org/ must include a valid, recent nonce as the GET url path. Upon receipt of a valid nonce, it is removed from the list of valid nonces. Nonces are only valid for 60 seconds or until SIGNAL NEWNYM, which ever comes first. The list of pending nonces should not be allowed to grow beyond 10 entries. The timeout period and nonce limit should be configurable in torrc. Design: XML document format for http://127.84.111.114 To avoid the need to localize the message in Tor, Tor will only provide a XML object with connectivity information. Here is an example form: <tor-test> <tor-bootstrap-percent>100</tor-bootstrap-percent> <tor-version-current>true</tor-version-current> <dns-nonce>4977eb4842c7c59fa5b830ac4da896d9</dns-nonce> <tor-test/> The tor-bootstrap-percent field represents the results of the Tor client bootstrap status as integer percentages from bootstrap_status_t. The tor-version-current field represents the results of the Tor client consensus version check. If the bootstrap process has not yet downloaded a consensus document, this field will have the value null. The dns-nonce field contains a 128-bit secret, encoded in base16. This field is only present for requests that list the Host: header as 127.84.111.114. Design: XML document format for http://selftest.torproject.org/nonce <tor-test> <tor-bootstrap-percent>100</tor-bootstrap-percent> <tor-version-current>true</tor-version-current> <dns-nonce-valid>true</dns-nonce-valid> <tor-test/> The first two fields are the same as for the IP address version. The dns-nonce-valid field is only true if the Host header matches selftest.torproject.org and the nonce is current and valid. Upon receipt of a valid nonce, that nonce is removed from the list of valid nonces. Design: Request Servicing Care must be taken with the dns-nonce generation and usage, to prevent users from being tracked through leakage of nonce value to application content. While the usage of XML appears to make this impossible due to stricter same-origin policy enforcement than JSON, same-origin enforcement is still fraught with exceptions and loopholes. In particular: Any requests that contain the Origin: header MUST be ignored, as the Origin: header is only included for third party web content (CORS). dns-nonce fields MUST be omitted if the HTTP Host: header does not match the IP address 127.84.111.114. Requests to selftest.torproject.org MUST return false for the dns-nonce-valid field if the HTTP Host: header does not match selftest.torproject.org, regardless of nonce value. Further, requests to selftest.torproject.org MUST validate that 'selftest.torproject.org' was the actual hostname provided to SOCKS4A, and not some alternate address mapping (due to DNS rebinding attacks, for example). Design: Application Usage Applications will use the system in two steps. First, they will make an HTTP request to http://127.84.111.114:80/ over Tor's SOCKS port and parse the resulting XML, if any. If the request at this stage fails, the application should inform the user that either their Tor client is too old, or that it is misconfigured, depending upon the nature of the failure. If the request succeeds and valid XML is returned, the application will record the value of the dns-nonce field, and then perform a second request to http://selftest.torproject.org/nonce_value. If the second request succeeds, and the dns-nonce-valid field is true, the application may inform the user that their Tor settings are valid. If the second request fails, or does not provide the correct dns-nonce, the application will inform the user that their Tor DNS proxy settings are incorrect. If either tor-bootstrap-percent is not 100, or tor-version-current is false, applications may choose to inform the user of these facts using properly localized strings and appropriate UI. Security Considerations XML was chosen over JSON due to the risks of the identifier leaking in a way that could enable websites to track the user[1]. Because there are many exceptions and circumvention techniques to the same-origin policy, we have also opted for strict controls on dns-nonce lifetimes and usage, as well as validation of the Host header and SOCKS4A request hostnames. 1. http://www.hpenterprisesecurity.com/vulncat/en/vulncat/dotnet/javascript_hi… -- Mike Perry

2 1

Proposal: Faster Headless Consensus Bootstrapping
by Mike Perry 11 Oct '12

11 Oct '12

Also at: https://gitweb.torproject.org/user/mikeperry/torspec.git/blob/consensus-boo… ------------------------------------------------------------------------- Title: Faster Headless Consensus Bootstrapping Author: Mike Perry Created: 01-10-2012 Status: Open Target: 0.2.4.x+ Overview and Motiviation This proposal describes a way for clients to fetch the initial consensus more quickly in situations where some or all of the directory authorities are unreachable. This proposal is meant to describe a solution for bug #4483. Design: Bootstrap Process Changes The core idea is to attempt to establish bootstrap connections in parallel during the bootstrap process, and download the consensus from the first connection that completes. Connection attempts will be done in batches of three. Only one connection will be performed to one of the canonical directory authorities. Two connections will be performed to randomly chosen hard coded directory mirrors. If no connections complete within 5 seconds, another batch of three connections will be launched. Otherwise, the first connection to complete will be used to download the consensus document and the others will be closed, after which bootstrapping will proceed as normal. If at any time, the total outstanding bootstrap connection attempts exceeds 15, no new connection attempts are to be launched until existing connection attempts experience full timeout. Design: Fallback Dir Mirror Selection The set of hard coded directory mirrors from #572 shall be chosen using the 100 Guard nodes with the longest uptime. The fallback weights will be set using each mirror's fraction of consensus bandwidth out of the total of all 100 mirrors. This list of fallback dir mirrors should be updated with every major Tor release. In future releases, the number of dir mirrors should be set at 20% of the current Guard nodes, rather than fixed at 100. Performance: Additional Load with Current Parameter Choices This design and the connection count parameters were chosen such that no additional bandwidth load would be placed on the directory authorities. In fact, the directory authorities should experience less load, because they will not need to serve the consensus document for a connection in the event that one of the directory mirrors complete their connection before the directory authority does. However, the scheme does place additional TLS connection load on the fallback dir mirrors. Because bootstrapping is rare and all but one of the TLS connections will be very short-lived and unused, this should not be a substantial issue. The dangerous case is in the event of a prolonged consensus failure that induces all clients to enter into the bootstrap process. In this case, the number of initial TLS connections to the fallback dir mirrors would be 2*C/100, or 10,000 for C=500,000 users. If no connections complete before the five retries, this could reach as high as 50,000 connection attempts, but this is extremely unlikely to happen in full aggregate. However, in the no-consensus scenario today, the directory authorities would already experience C/9 or 55,555 connection attempts. The 5-retry scheme increases their total maximum load to about 275,000 connection attempts, but again this is unlikely to be reached in aggregate. Additionally, with this scheme, even if the dirauths are taken down by this load, the dir mirrors should be able to survive it. Implementation Notes: Code Modifications The implementation of the bootstrap process is unfortunately mixed in with many types of directory activity. The process starts in update_consensus_networkstatus_downloads(), which initiates a single directory connection through directory_get_from_dirserver(). Depending on bootstrap state, a single directory server is selected and a connection is eventually made through directory_initiate_command_rend(). There appear to be a few options for altering this code to perform multiple connections. Without refactoring, one approach would be to make multiple calls to directory_initiate_command_routerstatus() from directory_get_from_dirserver() if the purpose is DIR_PURPOSE_FETCH_CONSENSUS and the only directory servers available are the authorities and the fallback dir mirrors. The code in directory_initiate_command_rend() would then need to be altered to maintain a list of the dircons created for this purpose as well as avoid immediately queuing the directory_send_command() request for the DIR_PURPOSE_FETCH_CONSENSUS purpose. A flag would need to be set on the dircon to be checked in connection_dir_finished_connecting(). The function connection_dir_finished_connecting() would need to be altered to examine the list of pending dircons, determine if this one is the first to complete, and if so, then call directory_send_command() to download the consensus and close the other pending dircons. An additional timer would need to be installed to re-call update_consensus_networkstatus_downloads() or a related helper after 5 seconds. connection_dir_finished_connecting() would cancel this timer. The helper would check the list of pending connections and ensure it never exceeds 15. -- Mike Perry

2 1

Proposal 208: IPv6 Exits Redux
by Nick Mathewson 11 Oct '12

11 Oct '12

Filename: 208-ipv6-exits-redux.txt Title: IPv6 Exits Redux Author: Nick Mathewson Created: 10-Oct-2012 Status: Open Target: 0.2.4.x 1. Obligatory Motivation Section [Insert motivations for IPv6 here. Mention IPv4 address exhaustion. Insert official timeline for official IPv6 adoption here. Insert general desirability of being able to connect to whatever address there is here. Insert profession of firm conviction that eventually there will be something somebody wants to connect to which requires the ability to connect to an IPv6 address.] 2. Proposal Proposal 117 has been there since coderman wrote it in 2007, and it's still mostly right. Rather than replicate it in full, I'll describe this proposal as a patch to it. 2.1. Exit policies Rather than specify IPv6 policies in full, we should move (as we have been moving with IPv4 addresses) to summaries of which IPv6 ports are generally permitted. So let's allow server descriptors to include a list of accepted IPv6 ports, using the same format as the "p" line in microdescriptors, using the "ipv6-policy" keyword. "ipv6-policy" SP ("accept" / "reject") SP PortList NL Exits should still, of course, be able to configure more complex policies, but they should no longer need to tell the whole world about them. After this ipv6-policy line is validated, it should get copied into a "p6" line in microdescriptors. This change breaks the existing exit enclave idea for IPv6, but the exiting exit enclave implementation never worked right in the first place. If we can come up with a good way to support it, we can add that back in. 2.2. Which addresses should we connect to? One issue that's tripped us up a few times is how to decide whether we can use IPv6 addresses. You can't use them with SOCKS4 or SOCKS4a, IIUC. With SOCKS5, there's no way to indicate that you prefer IPv4 or IPv6. It's possible that some SOCKS5 users won't understand IPv6 addresses. With this in mind, I'm going to suggest that with SOCKS4 or SOCKS4a, clients should always require IPv4. With SOCKS5, clients should accept IPv6. If it proves necessary, we can also add per-SOCKSPort configuration flags to override the above default behavior. See also partitioning discussion in Security Notes below. 2.3. Extending BEGIN cells. Prop117 (and the section above) says that clients should prefer one address or another, but doesn't give them a means to tell the exit to do so. Here's one. We define an extension to the BEGIN cell as follows. After the ADDRESS | ':' | PORT | [00] portion, the cell currently contains all [00] bytes. We add a 32-bit flags field, stored as an unsigned 32 bit value, after the [00]. All these flags default to 0, obviously. We define the following flags: bit 1 -- IPv6 okay. We support learning about IPv6 addresses and connecting to IPv6 addresses. 2 -- IPv4 not okay. We don't want to learn about IPv4 addresses or connect to them. 3 -- IPv6 preferred. If there are both IPv4 and IPv6 addresses, we want to connect to the IPv6 one. (By default, we connect to the IPv4 address.) 4..32 -- Reserved. As with so much else, clients should look at the platform version of the exit they're using to see if it supports these flags before sending them. 2.4. Minor changes to proposal 117 GETINFO commands that return an address, and which should return two, should not in fact begin returning two addresses separated by CRLF. They should retain their current behavior, and there should be a new "all my addresses" GETINFO target. 3. Security notes: Letting clients signal that they want or will accept IPv6 addresses creates two partitioning issues that didn't exist before. One is the version partitioning issue: anybody who supports IPv6 addresses is obviously running the new software. Another is option partitioning: anybody who is using a SOCKS4a application will look different from somebody who is using a SOCKS5 application. We can't do much about version partitioning, I think. If we felt especially clever, we could have a flag day. Is that necessary? For option partitioning, are there many applications whose behavior is indistinguishable except that they are sometimes configured to use SOCKS4a and sometimes to use SOCKS5? If so, the answer may well be to persuade as many users as possible to switch those to SOCKS5, so that they get IPv6 support and have a large anonymity set. IPv6 addresses are plentiful, which makes caching them dangerous if you're hoping to avoid tracking over time. (With IPv4 addresses, it's harder to give every user a different IPv4 address for a target hostname with a long TTL, and then accept connections to those IPv4 addresses from different exits over time. With IPv6, it's easy.) This makes proposal 205 especially necessary here.

1 0

Proposal 206: Preconfigured directory sources for bootstrapping
by Nick Mathewson 11 Oct '12

11 Oct '12

Filename: 206-directory-sources.txt Title: Preconfigured directory sources for bootstrapping Author: Nick Mathewson Created: 10-Oct-2012 Status: Open Target: 0.2.4.x Motivation and History: We've long wanted a way for clients to do their initial bootstrapping not from the directory authorities, but from some other set of nodes expected to probably be up when future clients are starting. We tried to solve this a while ago by adding a feature where we could ship a 'fallback' networkstatus file -- one that would get parsed when we had no current networkstatus file, and which we would use to learn about possible directory sources. But we couldn't actually use it, since it turns out that a randomly chosen list of directory caches from 4-5 months ago is a terrible place to go when bootstrapping. Then for a while we considered an "Extra-Stable" flag so that clients could use only nodes with a long history of existence from these fallback networkstatus files. We never built it, though. Instead, we can do this so much more simply. If we want to ship Tor with a list of initial locations to go for directory information, why not just do so? Proposal: In the same way that Tor currently ships with a list of directory authorities, Tor should also ship with a list of directory sources -- places to go for an initial consensus if you don't have a somewhat recent one. These need to include an address for the cache's ORPort, and its identity key. Additionally, they should include a selection weight. They can be configured with a torrc option, just like directory authorities are now. Whenever Tor is starting without a consensus, if it would currently ask a directory authority for a consensus, it should instead ask one of these preconfigured directory sources. I have code for this (see git branch fallback_dirsource_v2) in my public repository. When we deploy this, we can (and should) rip out the Fallback Networkstatus File logic. How to find nodes to make into directory sources: We could take any of three approaches for selecting these initial directory sources. First, we could try to vet them a little, with a light variant of the process we use for authorities. We'd want to look for nodes where we knew the operators, verify that they were okay with keeping the same IP for a very long time, and so forth. Second, we could try to pick nodes for listing with each Tor release based entirely on how long those nodes have been up. Anything that's been a high-reliability directory for a long time on the same IP (like, say, a year) could be a good choice. Third, we could blend the approach and start by looking for up-for-a-long-time nodes, and then also ask the operators whether their nodes are likely to stay running for a long time. I think the third model is best. Some notes on security: Directory source nodes have an opportunity to learn about new users connecting to the network for the first time. Once we have directory guards, that's going to be a fairly uncommon ability. We should be careful in any directory guard design to make sure that we don't fall back to the directory sources any more than we need to. See proposal 207.

1 0

10 days left for proposals for hard stuff [Re: Reminder: Big/tricky/interesting features for 0.2.4 need proposals by 10 October]
by Nick Mathewson 08 Oct '12

08 Oct '12

On Mon, Sep 17, 2012 at 11:47 AM, Nick Mathewson <nickm(a)freehaven.net> wrote: > Hi, all! > > From https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/024 : > > "October 10, 2012: Big feature proposal checkpoint. Any large > complicated feature which requires a design proposal must have its > first design proposal draft by this date." > > There are only 23 days till October 10. If there's something you want > in 0.2.4.x, and it is the kind of thing that needs a design proposal, > and there is no proposal yet, it could be time to start writing! > > What needs a design proposal? Generally: anything that involves a > change to the Tor protocol; anything whose security implications are > nontrivial and need discussion; and anything that will change any Tor > specification. Err on the side of "it needs a proposal." > > What is a "large complicated feature"? Please assume I'm going to be > extremely grumpy here, and err on the side of "it is big". If the > writeup of the proposal that explains how it works, or why to do it > this way is going to take more than a few paragraphs, it is probably > 'big' or 'complex'. > > (Note to would-be system-gamers: Please don't send a sketchy > incomplete draft as a placeholder to get your foot in the door. That's > not cool. If you don't have a draft ready, the feature can wait till > 0.2.5.) > > (Note also: a feature proposal by this deadline is a necessary > condition for getting your big/tricky/complicated feature into 0.2.4, > but not a sufficient condition. It also needs to have a working > implementation on schedule.) > > (Note finally: This is not a promise to not merge stuff that violates > this deadline, but I sure will be trying not to merge such stuff.) > > yrs, > -- > Nick

3 3

resistance to rubberhose and UDP questions
by Eugen Leitl 06 Oct '12

06 Oct '12

I've had an IRC session with the designer of cjdns (on cjdns) who made a few interesting points, and suggestions. Comments? Verbatim chat snip below. 18:03 <@cjd> if you took the components from cjdns, you could build a TOR like protocol which used UDP if possible and made connections much faster 18:04 <+eleitl> I wonder why they didn't choose UDP 18:05 <@cjd> you need to fall back on tcp in case you're firewalled to hell 18:05 <+eleitl> Apparently, they're thinking about it https://blog.torproject.org/blog/moving-tor-datagram-transport 18:06 <@cjd> problem with tor is (correct me if I) 18:06 <@cjd> 'm wrong) 18:06 <@cjd> the directory is signed by the tor foundation 18:07 <@cjd> so they can sign a fake directory and run a bunch of directory servers and when Alice connects to their directory server, they give her a bunch of fake nodes 18:07 <@cjd> run their own botnet with fake tor nodes so your circuit is always owned 18:07 <+eleitl> I don't really know for sure, but there's intrinsic trust to Tor developers, yes. 18:08 <+eleitl> You can run your own Tor network, though. 18:08 <+eleitl> Some botnets do that. 18:08 <@cjd> I trust them to make the software right, esp. since I could check if they did. 18:09 <@cjd> But a little arm twisting can change someone's motives pretty fast. 18:09 <+eleitl> Maintaining signing secrets is a problem. 18:09 <+eleitl> They should have used a P2P design. 18:10 <@cjd> If someone (with government hat?) tells you they can make your life hell... I wouldn't fault them for doing what the man says. 18:10 <@cjd> *wouldn't fault you 18:10 <+eleitl> I'll try bugging some Tor developers about that scenario, and see how they squirm. 18:11 <+eleitl> Also, the UDP connection thing. 18:11 <@cjd> You can "stack" your circuit setup packets if you're using UDP 18:11 <@cjd> stack -> all headers in the same packet 18:12 <@cjd> cjdns does the same thing 18:13 <+eleitl> Can I use snippage from this chat verbatim, or will I need to rephrase? 18:14 <@cjd> sure go ahead 18:14 <+eleitl> Thanks! 18:14 <@cjd> can only speak for myself ofc 18:14 <+eleitl> Sure.

6 7