- tor-dev - lists.torproject.org

STEM: Tor2csv / Tor2xml / Tor2json ?
by Fabio Pietrosanti (naif) 15 Jun '12

15 Jun '12

Hi, i just would like to provide a suggestion for Stem use (maybe already done), now that it has a powerful cached-consensus/descriptors parsers. Would it possible to provide easy to use command line tools to access Tor's data in the following formats: - csv - xml - json So that anyone requiring to access tor's current network data can easily use a function / script trough a command line tool and get it in easy to be formatted output. That's just a suggestion, but it maybe very useful for anyone requiring to work on tor's network data in an easy/accessible way. -naif

2 1

Re: [tor-dev] Stem Testing Mocking Issue
by Damian Johnson 14 Jun '12

14 Jun '12

Hi Megan, thanks for the patch! What is an example of a standard library function with a builtin type? I'd like to exercise the use case that has been causing you trouble. This change has a couple issues, for instance it treats anything with the same name as a builtin like a builtin. It also calls setattr() on standard library functions if you try to mock it a second time. I've pushed an adaptation of your change to the 'mocking' branch of my personal repository... git://git.torproject.org/user/atagar/stem.git change... https://gitweb.torproject.org/user/atagar/stem.git/commitdiff/5029e4b9b92db… It shouldn't have the issues that I mentioned, though some remaining bits of the change confuse me. As you've found the mocking code is pretty tricky so I won't be able to make sense of it until I see what was breaking. Thanks! -Damian PS. Very nice commit message on b8b7c22. It helped me a lot in making sense of the change - thanks for that.

2 3

start_daemon and finish_daemon
by Zack Weinberg 14 Jun '12

14 Jun '12

Why is the process of going into daemon mode (on Unix) split into two functions, start_daemon and finish_daemon? zw

2 2

Dreamplug trouble
by Linus Nordberg 13 Jun '12

13 Jun '12

Hi, An hour ago I had a beautiful dreamplug laying next to me on my table, booting freedombox, making me happy. Now I have a beautiful dreamplug nicely mounted in a rack in a hall next to me, not booting at all, making me sad. --8<---------------cut here---------------start------------->8--- U-Boot 2011.06-02334-g8f495d9-dirty (May 31 2011 - 02:06:26) Marvell-DreamPlug SoC: Kirkwood 88F6281_A0 CPU running @ 1200Mhz L2 running @ 400Mhz SysClock = 400Mhz , TClock = 200Mhz DRAM: 512 MiB SF: Detected MX25L1606 with page size 256, total 1 MiB In: serial Out: serial Err: serial Net: egiga0, egiga1 88E1116 Initialized on egiga0 88E1116 Initialized on egiga1 Hit any key to stop autoboot: 0 *** ERROR: `ipaddr' not set ping failed; host 192.168.2.1 is not alive No link on egiga1 *** ERROR: `ipaddr' not set ping failed; host 192.168.2.1 is not alive (Re)start USB... USB: Register 10011 NbrPorts 1 USB EHCI 1.00 scanning bus for devices... 5 USB Device(s) found scanning bus for storage devices... Device NOT ready Request Sense returned 02 3A 00 3 Storage Device(s) found reading uImage 0 bytes read Wrong Image Format for bootm command ERROR: can't get kernel image! Marvell>> --8<---------------cut here---------------end--------------->8--- It finds 5 USB devices, one of which is a USB stick with the freedombox filesystem on it. Did I break it when I moved it? Ideas?

2 1

Proposal 188: Bridge Guards and other anti-enumeration defenses
by Nick Mathewson 12 Jun '12

12 Jun '12

Filename: 188-bridge-guards.txt Title: Bridge Guards and other anti-enumeration defenses Author: Nick Mathewson Created: 14 Oct 2011 Status: Open 1. Overview Bridges are useful against censors only so long as the adversary cannot easily enumerate their addresses. I propose a design to make it harder for an adversary who controls or observes only a few nodes to enumerate a large number of bridges. Briefly: bridges should choose guard nodes, and use the Tor protocol's "Loose source routing" feature to re-route all extend requests from clients through an additional layer of guard nodes chosen by the bridge. This way, only a bridge's guard nodes can tell that it is a bridge, and the attacker needs to run many more nodes in order to enumerate a large number of bridges. I also discuss other ways to avoid enumeration, recommending some. These ideas are due to a discussion at the 2011 Tor Developers' Meeting in Waterloo, Ontario. Practically none of the ideas here are mine; I'm just writing up what I remember. 2. History and Motivation Under the current bridge design, an attacker who runs a node can identify bridges by seeing which "clients" make a large number of connections to it, or which "clients" make connections to it in the same way clients do. This has been a known attack since early versions {XXXX check} of the design document; let's try to fix it. 2.1. Related ideas: Guard nodes The idea of guard nodes isn't new: since 0.1.1, Tor has used guard nodes (first designed as "Helper" nodes by Wright et al in {XXXX}) to make it harder for an adversary who controls a smaller number of nodes to eavesdrop on clients. The rationale was: an adversary who controls or observes only one entry and one exit will have a low probability of correlating any single circuit, but over time, if clients choose a random entry and exit for each circuit, such an adversary will eventually see some circuits from each client with a probability of 1, thereby building a statistical profile of the client's activities. Therefore, let each client choose its entry node only from among a small number of client-selected "guard" nodes: the client is still correlated with the same probability as before, but now the client has a nonzero chance of remaining unprofiled. 2.2. Related idea: Loose source routing Since the earliest versions of Onion Routing, the protocol has provided "loose source routing". In strict source routing, the source of a message chooses every hop on the message's path. But in loose source routing, the message traverses the selected nodes, but may also traverse other nodes as well. In other words, the client selects nodes N_a, N_b, and N_c, but the message may in fact traverse any sequence of nodes N_1...N_j, so long as N_1=N_a, N_x=N_b, and N_y=N_c, for 1 < x < y. Tor has retained this feature, but has not yet made use of it. 3. Design Every bridge currently chooses a set of guard nodes for its circuits. Bridges should also re-route client circuits through these circuits. Specifically, when a bridge receives a request from a client to extend a circuit, it should first create a circuit to its guard, and then relay that extend cell through the guard. The bridge should add an additional layer of encryption to outgoing cells on that circuit corresponding to the encryption that the guard will remove, and remove a layer of encryption on incoming cells on that circuit corresponding to the encryption that the guard will add. 3.1. An example This example doesn't add anything to the design above, but has some interesting inline notes. - Alice has connected to her bridge Bob, and built a circuit through Bob, with the negotiated forward and reverse keys KB_f and KB_r. - Alice then wants to extend the circuit to node Charlie. She makes a hybrid-encrypted onionskin, encrypted to Charlie's public key, containing her chosen g^x value. She puts this in an extend cell: "Extend (Charlie's address) (Charlie's OR Port) (Onionskin) (Charlie's ID)". She encrypts this with KB_f and sends it as a RELAY_EARLY cell to Bob. - Bob receives the RELAY_EARLY cell, and decrypts it with KB_f. He then sees that it's an extend cell for him. So far, this is exactly the same as the current procedure that Alice and Bob would follow. Now we diverge: - Instead of connecting to Charlie directly, Bob makes sure that he is connected to his guard, Guillaume. Bob uses a CREATE_FAST cell (or a CREATE cell, but see 4.1 below) to open a circuit to Guillaume. Now Bob and Guillaume share keys KG_f and KG_b. - Now Bob encrypts the Extend cell body with KG_f and sends it as a RELAY_EARLY cell to Guillaume. - Guillaume receives it, decrypts it with KG_f, and sees: "Extend (Charlie's address) (Charlie's OR Port) (Onionskin) (Charlie's ID)". Guillaume acts accordingly: creating a connection to Charlie if he doesn't have one, ensuring that the ID is as expected, and then sending the onionskin in a create cell on that connection. Note that Guillaume is behaving exactly as a regular node would upon receiving an Extend cell. - Now the handshake finishes. Charlie receives the onionskin and sends Guillaume "CREATED g^y,KH". Guillaume sends Bob "E(KG_r, EXTENDED g^y KH)". (Charlie and Guillaume are still running as regular Tor nodes do today). - With this extend cell, and with all future relay cells received on this circuit, Bob first decrypts the cell with KG_r, then re-encrypts it with KB_r, then passes it to Alice. When Alice receives the cell, it will be just as she would have received if Bob had extended to Charlie directly. - With all future outgoing cells that he receives from Alice, Bob first decrypts the cell with KA_f, and if the cell does not have Bob as its destination, Bob encrypts it with KG_f before passing it to Guillaume. Note that this design does not require that our stream cipher operations be transitive, even though they are. Note also that this design requires no change in behavior from any node other than Bob the bridge. Finally, observe that even though the circuit is one hop longer than it would be otherwise, no relay's count of permissible RELAY_EARLY cells falls lower than it otherwise would. This is because the extra hop that Bob adds is done with a CREATE_FAST cell, and so he does not need to send any RELAY_EARLY cells not originated by Alice. 4. Other ideas and alternative designs In addition to the design above, there are more ways to try to prevent enumeration. 4.1. Make it harder to tell clients from bridges Right now, there are multiple ways for the node after a bridge to distinguish a circuit extended through the bridge from one originating at the bridge. (This lets the node after the bridge tell that a bridge is talking to it.) One of the giveaways here is that the first hop in a circuit is created with CREATE_FAST cells, but all subsequent hops are created with CREATE cells. In the above design, it's no longer quite so simple to tell, since all of the circuits that extend through a bridge now reach its guards through CREATE_FAST cells, whether the bridge originated them or not. (If we adopt a faster circuit extension algorithm -- for example, Goldberg, Stebila, and Ustaoglu's design instantiated over curve25519 -- we could also solve this issue by eliminating CREATE_FAST/CREATED_FAST entirely, which would also help our security margin a little.) The CREATE/CREATE_FAST distinction is not the only way for a bridge's guard to tell bridges from orginary clients, however. Most importantly, a busy bridge will open far more circuits than a client would. More subtly, the timing on response from the client will be higher and more highly variable that it would be with an ordinary client. I don't think we can make bridges behave wholly indistinguishably from clients: that's why we should go with guard nodes for bridges. 4.2. Client-enforced bridge guards What if Tor didn't have loose source routing? We could have bridges tell clients what guards to use by advertising those guard in their descriptors, and then refusing to extend circuits to any other nodes. This change would require all clients to upgrade in order to be able to use the newer bridges, and would quite possibly cause a fair amount of pain along the way. Fortunately, we don't need to go down this path. So let's not! 4.3. Separate bridge-guards and client-guards In the design above, I specify that bridges should use the same guard nodes for extending client circuits as they use for their own circuits. It's not immediately clear whether this is a good idea or not. Having separate sets would seem to make the two kinds of circuits more easily distinguishable (even though we already assume they are distinguishable). Having different sets of guards would also seem like a way to keep the nodes who guard our own traffic from learning that we're a bridge... but another set of nodes will learn that anyway, so it's not clear what we'd gain. 5. Other considerations What fraction of our traffic is bridge traffic? Will this alter our circuit selection weights? Are the current guard selection/evaluation/replacement mechanisms adequate for bridge guards, or do bridges need to get more sophisticated?

7 7

Proposal 189: AUTHORIZE and AUTHORIZED cells
by George Kadianakis 12 Jun '12

12 Jun '12

Filename: 189-authorize-cell.txt Title: AUTHORIZE and AUTHORIZED cells Author: George Kadianakis Created: 04 Nov 2011 Status: Open 1. Overview Proposal 187 introduced the concept of the AUTHORIZE cell, a cell whose purpose is to make Tor bridges resistant to scanning attacks. This is achieved by having the bridge and the client share a secret out-of-band and then use AUTHORIZE cells to validate that the client indeed knows that secret before proceeding with the Tor protocol. This proposal specifies the format of the AUTHORIZE cell and also introduces the AUTHORIZED cell, a way for bridges to announce to clients that the authorization process is complete and successful. 2. Motivation AUTHORIZE cells should be able to perform a variety of authorization protocols based on a variety of shared secrets. This forces the AUTHORIZE cell to have a dynamic format based on the authorization method used. AUTHORIZED cells are used by bridges to signal the end of a successful bridge client authorization and the beginning of the actual link handshake. AUTHORIZED cells have no other use and for this reason their format is very simple. Both AUTHORIZE and AUTHORIZED cells are to be used under censorship conditions and they should look innocuous to any adversary capable of monitoring network traffic. As an attack example, an adversary could passively monitor the traffic of a bridge host, looking at the packets directly after the TLS handshake and trying to deduce from their packet size if they are AUTHORIZE and AUTHORIZED cells. For this reason, AUTHORIZE and AUTHORIZED cells are padded with a random amount of padding before sending. 3. Design 3.1. AUTHORIZE cell The AUTHORIZE cell is a variable-sized cell. The generic AUTHORIZE cell format is: AuthMethod [1 octet] MethodFields [...] PadLen [2 octets] Padding ['PadLen' octets] where: 'AuthMethod', is the authorization method to be used. 'MethodFields', is dependent on the authorization Method used. It's a meta-field hosting an arbitrary amount of fields. 'PadLen', specifies the amount of padding in octets. 'Padding', is 'PadLen' octets of random content. 3.2. AUTHORIZED cell format The AUTHORIZED cell is a variable-sized cell. The AUTHORIZED cell format is: 'AuthMethod' [1 octet] 'PadLen' [2 octets] 'Padding' ['PadLen' octets] where all fields have the same meaning as in section 3.1. 3.3. Cell parsing Implementations MUST ignore the contents of 'Padding'. Implementations MUST reject an AUTHORIZE or AUTHORIZED cell where the 'Padding' field is not 'PadLen' octets long. Implementations MUST reject an AUTHORIZE cell with an 'AuthMethod' they don't recognize. 4. Discussion 4.1. Why not let the pluggable transports do the padding, like they are supposed to do for the rest of the Tor protocol? The arguments of section "Alternative design: Just use pluggable transports" of proposal 187, apply here as well: All bridges who use client authorization will also need camouflaged AUTHORIZE/AUTHORIZED cell. 4.2. How should multiple round-trip authorization protocols be handled? Protocols that require multiple round-trips between the client and the bridge should use AUTHORIZE cells for communication. The format of the AUTHORIZE cell is flexible enough to support messages from the client to the bridge and the inverse. In the end of a successful multiple round-trip protocol, an AUTHORIZED cell must be issued from the bridge to the client. 4.3. AUTHORIZED seems useless. Why not use VPADDING instead? As noted in proposal 187, the Tor protocol uses VPADDING cells for padding; any other use of VPADDING makes the Tor protocol kludgy. In the future, and in the example case of a v3 handshake, a client can optimistically send a VERSIONS cell along with the final AUTHORIZE cell of an authorization protocol. That allows the bridge, in the case of successful authorization, to also process the VERSIONS cell and begin the v3 handshake promptly.

6 12

[GSoC] Update - Safe cookie support/General controller class for Stem
by Ravi Chandra Padmala 11 Jun '12

11 Jun '12

Hello I have finished working on implementing the safe cookie authentication support for Stem. Damian fixed a few things[1] and merged it into master. I'm now working on implementing the general controller class. GETINFO parsing was implemented by Damian as an example. I've implemented GETCONF parsing[2] for the controller. It's almost done. I'm also going to implement the other methods to handle the remaining control commands allocated to week 2 and 3 in my proposal[3]. Note: I'm behind my schedule by a little more than a week. I had an exam during the 2nd week, because of which I couldn't do much/any work that week. Though, I do have enough buffer time to make up for this, and, I'm trying to catch up with my schedule. 1. https://gitweb.torproject.org/stem.git/commit/560923cb7b572d02046c6ca2bd5eb… 2. https://trac.torproject.org/projects/tor/ticket/6114 3. https://www.torproject.org/about/gsocProposal/gsoc12-proposal-stemImproveme… -- neena

1 0

Stem Sphinx Documentation
by Damian Johnson 10 Jun '12

10 Jun '12

Hi Ravi, Beck, and everyone else hacking on stem. I just finished and merged a rewrite of our documentation into reStructuredText. The results are... very pretty. http://www.atagar.com/transfer/tmp/stem_html_12_06_05/ Next I plan to make a landing page for stem, and a cron process which will automatically keep a documentation page up to date with the current git master branch. Some pages that you might find especially relevant... Ravi http://www.atagar.com/transfer/tmp/stem_html_12_06_05/stem.html#module-stem… Beck http://www.atagar.com/transfer/tmp/stem_html_12_06_05/stem.html#stem.contro… Karsten http://www.atagar.com/transfer/tmp/stem_html_12_06_05/stem.descriptor.html#… Norman, Erik, and Megan http://www.atagar.com/transfer/tmp/stem_html_12_06_05/stem.util.html#module… Cheers! -Damian

2 4

How (not) to Build a Transport Layer for Anonymity Overlays
by Roger Dingledine 09 Jun '12

09 Jun '12

http://pade12.mytestbed.net/pade12-final6.pdf To be presented this coming Friday in London: http://pade12.mytestbed.net/technicalprogram/ --Roger

1 0

Stem Testing Code Question - Wesleyan Interns
by Erik I Islo 08 Jun '12

08 Jun '12

Hi Damian, Having gotten started looking at some of the testing code from Stem over the last day or so, Megan and I have come up with a question. While both of us are currently on #tor and #tor-dev (as ErikI and MeganC), we figured an email would be best for getting started as we have a bit more to say. In run_tests.py, under the if statement that catches the case of integration tests (line 268), there is a block of code that contains a number of references to 'targets.' To start, could you explain the formatting and specific purpose of the settings.cfg file? Basically, we are trying to understand what lines 274 through 309 are doing in run_tests.py. Thanks in advance, Erik & Megan

2 1