August 2013 - tor-dev - lists.torproject.org

Proposal 221: Stop using CREATE_FAST
by Nick Mathewson 31 Oct '13

31 Oct '13

Filename: 221-stop-using-create-fast.txt Title: Stop using CREATE_FAST Authors: Nick Mathewson Created: 12 August 2013 Target: 0.2.5.x Status: Open 0. Summary I propose that in 0.2.5.x, Tor clients stop sending CREATE_FAST cells, and use CREATE or CREATE2 cells instead as appropriate. 1. Introduction The CREATE_FAST cell was created to avoid the performance hit of using the TAP handshake on a TLS session that already provided what TAP provided: authentication with RSA1024 and forward secrecy with DH1024. But thanks to the introduction of the ntor onionskin handshake in Tor 0.2.4.x, for nodes with older versions of OpenSSL, the TLS handshake strength lags behind with the strength of the onion handshake, and the arguments against CREATE no longer apply. Similarly, it's good to have an argument for circuit security that survives possible breakdowns in TLS. But when CREATE_FAST is in use, this is impossible: we can only argue forward-secrecy at the first hop of each circuit by assuming that TLS has succeeded. So let's simply stop sending CREATE_FAST cells. 2. Proposed design Currently, only clients will send CREATE_FAST, and only when they have FastFirstHopPK set to its default value, 1. I propose that we change "FastFirstHopPK" from a boolean to also allow a new default "auto" value that tells Tor to take a value from the consensus. I propose a new consensus parameter, "usecreatefast", default value taken to be 1. Once enough versions of Tor support this proposal, the authorities should set the value for "usecreatefast" to be 0. In the series after that (0.2.6.x?), the default value for "FastFirstHopPK" should be 0. (Note that CREATE_FAST must still be used in the case where a client has connected to a guard node or bridge without knowing any onion keys for it, and wants to fetch directory information from it.) 3. Alternative designs We might make some choices to preserve CREATE_FAST under some circumstances. For example, we could say that CREATE_FAST is okay if we have a TLS connection with a cipher, public key, and ephemeral key algorithm of a given strength. We might try to trust the TLS handshake for authentication but not forward secrecy, and come up with a first-hop handshake that did a simple curve25519 diffie-hellman. We might use CREATE_FAST only whenever ntor is not available. I'm rejecting all of the above for complexity reasons. We might just change the default for FastFirstHopPK to 1 in 0.2.5.x-alpha. It would make early users of that alpha easy for their guards to distinguish. 4. Performance considerations This will increase the CPU requirements on guard nodes; their cpuworkers would be more heavily loaded as 0.2.5.x is more adopted. I believe that, if guards upgrade to 0.2.4.x as 0.2.5.x is under development, the commensurate benefits of ntor will outweigh the problems here. This holds even more if we wind up with a better ntor implementation or replacement. 5. Considerations on client detection Right now, in a few places, Tor nodes assume that any connection on which they have received a CREATE_FAST cell is probably from a non-relay node, since relays never do that. Implementing this proposal would make that signal unreliable. We should do this proposal anyway. CREATE_FAST has never been a reliable signal, since "FastFirstHopPK 0" is easy enough to type, and the source code is easy enough to edit. Proposal 163 and its successors have better ideas here anyway.

3 4

Status report - Stream-RTT
by ra 08 Oct '13

08 Oct '13

Hi all! During the last weeks I have been very busy working on my GSoC project which is about reducing the RTT of preemptively built circuits. There is now a single script called "rttprober"[0] that depends on a patched[1] Tor client running a certain configuration[2]. The goal is to measure RTTs of Tor circuits. It takes a few parameters as input: an authenticated Stem Tor controller for communication with the Tor client, the number of circuits to probe, the number of probes to be taken for each circuit and the number of circuits that should be probed concurrently. It outputs a tar file containing lzo-compressed serialized data with detailed node information, all circuit- and stream-events involved and the circuit build time for further analysis. Since the RTT-measurements are run in parallel with very short locks it is important not to overload Tor nodes. Therefore a single node is not probed more than once at a time. A first analysis of some measurements taken supports the original assumption that a Frechét distribution fits both the circuit build times[3] and round trip times[4]. I will continue gathering and analyzing measurement data and will hopefully be able to draw some conclusions from that. Best, Robert [0] https://bitbucket.org/ra_/tor- rtt/src/1127f6936086664981fc55b4dbc82b1570714140/rttprober.py?at=master [1] https://bitbucket.org/ra_/tor- rtt/src/1127f6936086664981fc55b4dbc82b1570714140/patches?at=master [2] https://bitbucket.org/ra_/tor- rtt/src/1127f6936086664981fc55b4dbc82b1570714140/torrc?at=master [3] http://postimg.org/image/je8k5yydt/ [4] http://postimg.org/image/ktk90vxm7/

3 9

RFC: Lightweight Obfuscated Datagram Protocol (LODP)
by Yawning Angel 13 Sep '13

13 Sep '13

Hello all, Inspired by a conversation with arma, I have been working on designing a UDP based protocol to serve as the bulk data transport for something along the lines of "obfs3, but over UDP". The spec document is approaching the point where I would like to have increased public review before I go and implement it. People who sit in #tor-dev probably have seen various revisions of this draft already. Some notes: * This is heavily inspired by SSU from the i2p project, and Dust by Brandon Wiley. * Ensuring that data gets delivered reliably and in order is outside the scope of this document. Such things are a well researched problem so my reference implementation will likely draw upon an existing protocol instead of reinventing the wheel here. * As far as existing work in this area, the only other protocols that attempt to do similar things are i2p's SSU and Dust. * LODP assumes that there is a shared secret between peers to attempt to thwart active probing attacks. * As part of my evil plans, "secure fingerprinting resistant UDP" is something useful to have. * Open questions/things that are subject to change are tagged with XXX. Questions, comments, feedback appreciated. -- Yawning Angel

1 1

IRC meeting to discuss sponsor F progress on Tue September 3, 18:00 to 19:00 UTC in #tor-dev
by Karsten Loesing 05 Sep '13

05 Sep '13

Hi all, I'd like to schedule an IRC meeting to discuss what progress we made on sponsor F deliverables in August. Time and place are: Tue September 3, 18:00 to 19:00 UTC in #tor-dev That's in (exactly) 4 1/2 days from now (sorry for the short notice). The time in other timezones is: 11:00 in San Francisco 14:00 in Boston 20:00 in Berlin 21:00 in Athens 23:30 in New Delhi People who should attend are: Roger, Nick, George, David, Ximin, Nathan, Matt, Colin, Andrew, Tom, and anyone else who wants to attend. If your name is on that list and you're working on sponsor F stuff, but the date or time doesn't work for you, please email me your progress and I'll report it for you at the meeting. Thanks, Karsten

3 4

Draft of proposal "Migrate HS identity keys to Ed25519"
by George Kadianakis 03 Sep '13

03 Sep '13

Greetz, I'm posting the draft of a proposal that specifies how to upgrade the identity keys of HSes (currently RSA-1024) to use Ed25519. This proposal is supposed to go along with a proposal that specifies how to hide HS descriptors from HSDirs. I'm going to post that second proposal in a few minutes. This proposal is incredibly drafty in the sense that I might have forgotten to specify things that need to be specified. On the other hand, "release early; release often" they say, so here it goes. Inlining: Filename: xxx-hs-ecc-id-keys.txt Title: Migrate HS identity keys to Ed25519 Author: George Kadianakis Created: 10 August 2013 Target: 0.2.5.x Status: Draft [More draft than Guiness.] 0. Overview: This proposal suggests the adoption of ECDSA keys as the long-term identity keys of Hidden Services. It also proposes the adoption of ECDSA keys for the per-introduction-point service key that was introduced in the v2 hidden service protocol. This proposal specifies the key generation procedure, paths where the keys should be stored, and flagday/migration procedures. XXX 1. Motivation: Hidden Services currently use an RSA-1024 keypair as their long-term identity keys. RSA-1024 is considered weak and should be replaced. Dan Bernstein's ed25519 ECDSA scheme provides 2^128 bits of security and better signature performance than RSA. Furthermore, the keysize is much smaller than the keys of RSA. 2. Related proposals This document is part of a proposal triptych. Upcoming sister proposals are: - "Stop HS address enumeration by HSDirs", where we propose an alternative HS descriptor format that will not allow HSDirs to learn the addresses of the Hidden Services it serves. - "Deployment strategy for recent HS proposals", where we specify how these proposals should be deployed without making people angry. 3. Changes to the current HS protocol 3.0. Overview of changes to the current HS protocol There are two places where RSA-1024 is used in the current HS: Longterm RSA-1024 "identity keys" are used by the HSes to authenticate themselves to clients. Multiple short-term RSA-1024 "service keys" are used by the HSes for each introduction point so that they don't reveal their identity key. This proposal specifies: a) The generation of the new ed25519 long-term identity keys and service keys (#KEYGEN) b) The new HS descriptor format (v3) that contains identity keys and service keys (#NEWDESC) c) A way for clients to upload and fetch v3 descriptors from HSDirs (#HSDIRV3) 3.1. Generation of ed25519 keypairs (#KEYGEN) 3.1.0. Generation of long-term ed25519 identity key Compliant Hidden Services need to generate a fresh ed25519 keypair and store the private key in: $HiddenServiceDirectory/hs_ed25519_privkey For deployment and migration reasons, the new ed25519 keys must be kept alongside with the old RSA-1024 keys. 3.1.1. Generation of short-term ed25519 service keys Hidden Services generate an ed25519 service key for each introduction point -- instead of the RSA-1024 key they currently generate. 3.2. New v3 Hidden Service descriptors 3.2.0. New v3 Hidden Service Descriptor Format (#NEWDESC) v2 Hidden Service descriptors contain both the long-term identity public key of the Hidden Service, and service keys for each introduction point. It also contains a signature of the descriptor. We need to change all these fields to carry our brand new ed25519 keys. For v3 Hidden Service descriptors, we keep the v2 format and perform the following changes: [*] The "permanent-key" field is replaced by "permanent-key-ed25519": "permanent-key-ed25519" NL an ed25519 public key [Exactly once] The public key of the hidden service which is required to verify the "descriptor-id" and the "signature-ed25519". In base64 format with terminating =s removed. [*] The "service-key" field is replaced by "service-key-ed25519': "service-key-ed25519" NL an ed25519 public key [Exactly once] The public key that can be used to encrypt messages to the hidden service. In base64 format with terminating =s removed. [*] The "signature" field is replaced by "signature-ed25519': "signature-ed25519" NL signature-string [At end, exactly once] A signature of all fields above with the private ed25519 key of the hidden service. In base64 format with terminating =s removed. 3.2.1. Uploading v3 HS descriptors to HSDirs (#HSDIRV3) A new type of Hidden Service Directory Server must be established which understands v3 Hidden Service descriptors. The Hidden Service follows the same publishing procedure as for v2 descriptors but instead of sending an HTTP 'POST' to "/tor/rendezvous2/publish", the HS sends the 'POST' request to "/tor/rendezvous3/publish". (This part of the proposal conflicts with the "Stop HS address enumeration by HSDirs" proposal.) 3.3. Fetching v3 HS descriptors from HSDirs (#HSDIRV3) When a client needs to fetch a v3 Hidden Service Descriptor from an HSDir, it follows the exact same procedure as for v2 descriptors but instead of sending an HTTP 'GET' to "/tor/rendezvous2/<z>", it sends an HTTP 'GET' to "/tor/rendezvous3/<z>". (This part of the proposal conflicts with the "Stop HS address enumeration by HSDirs" proposal) 3.4. Service keys and INTRODUCE cells In INTRODUCE cells, when v2 descriptors are used, PK_ID is defined as the hash of the service key. This means that we don't need to change the format of INTRODUCE cells since we can just use the hash of the ed25519 service key. # XXX will this cause problems?

3 5

Re: [tor-dev] Help with rebasing arma's n23-5 tor branch to current master
by Karsten Loesing 03 Sep '13

03 Sep '13

(Moving this discussion back to tor-dev@ with Charlie's permission.) On 8/6/13 4:10 AM, Charlie Belmer wrote: > Hey Karsten, > > I've been reading up on Shadow & Scallion to do some performance testing on > N23. Before I go too far down that road, I wanted to check in with you. > > Are there existing blueprints or best practices I should be following? Thanks for looking into this! So, I wouldn't call the following notes blueprints or best practices, but they are a description of what I would do to run Shadow simulations of your branch. First of all, be sure to read the Shadow wiki which has contains lots of interesting stuff: https://github.com/shadow/shadow/wiki The next step would be to get a vanilla Tor master (that your branch is based on) running in a minimal or tiny Shadow network. It's probably easiest to use a local Ubuntu VM for this. Or you can start an m1.large EC2 instance for the tiny network, or an even smaller instance for the minimal network. Once that simulation succeeds, you'll want to run another simulation using your branch, extract performance metrics using Shadow's analyze script, produce a PDF with graphs also using the analyze script, and then wonder if things are faster than before. The minimal or tiny networks are mostly there to learn that things are working as expected. You'll probably want larger networks for real performance results, but starting small to get everything working is probably easier. Sounds doable, I guess. However, here's where things start to get tricky: in order to simulate Tor master (or a branch based on it) in Shadow, you'll have to perform quite a few extra steps like compiling your own CMake and Clang/LLVM. Not really hard, but you need to get all pieces together before Shadow does what you want. I'm pasting detailed instructions that I used to simulate Steven's uTP branch in Shadow below as an example. Not everything applies to your branch, but it should give you the general idea. If something remains unclear, please feel free to ask! > Depending how it goes, I'd also look at re-trying adaptive 23 or tuning the > existing implementation. Neat! We should probably include Mashael in that discussion once you have a working and simulated n23-5 branch. Maybe she has ideas what parts need tweaking. I'm cc'ing her in this mail so she knows what you're working on. In any case, having a working rebased n23-5 branch is already a big step forward. Thanks! Karsten === START OF INSTRUCTIONS === Launch new EC2 instance running Ubuntu Server 13.04 64 bit Set ulimit -n to 25000, see Shadow wiki for details https://github.com/shadow/shadow/wiki/Preparing-your-machine#system-configs… $ sudo apt-get update $ sudo apt-get upgrade Clone and test-compile modified libutp $ sudo apt-get install build-essential git automake libssl-dev libevent-dev $ mkdir src $ cd src/ $ git clone https://github.com/sjmurdoch/libutp $ cd libutp/ $ make $ cd ../ Clone and test-compile Tor branch utp $ git clone https://git.torproject.org/tor.git $ cd tor/ $ git remote add sjm217 https://git.torproject.org/sjm217/tor.git $ git fetch sjm217 $ git checkout -b utp sjm217/utp $ ./autogen.sh $ LDFLAGS="-L/home/ubuntu/src/libutp" CFLAGS="-I/home/ubuntu/src/libutp" LIBS="-lutp -lrt" ./configure --disable-asciidoc $ make $ make distclean $ cd ../ Modify Tor to use uTP for all links (or "0 &&" for not using uTP for any link) diff --git a/src/or/channeltls.c b/src/or/channeltls.c index 0551b73..b7b36e1 100644 --- a/src/or/channeltls.c +++ b/src/or/channeltls.c @@ -418,7 +418,7 @@ channel_tls_connect(const tor_addr_t *addr, uint16_t port, /* Create a uTP connection */ tor_addr_to_sockaddr(addr, port, (struct sockaddr*)&sin, sizeof(sin)); tor_addr_to_str(addr_str, addr, sizeof(addr_str), 0); - if (!strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { + if (1 || !strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { log_info(LD_CHANNEL, "Trying uTP connection to %s", addr_str); tlschan->utp = UTP_Create(tor_UTPSendToProc, tlschan, (const struct sockaddr*)&sin, Install CMake 2.8.10 $ wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz $ tar xf cmake-2.8.10.2.tar.gz $ cd cmake-2.8.10.2/ $ ./configure $ make $ export PATH=~/src/cmake-2.8.10.2/bin:$PATH $ cd ../ $ which cmake # result: /home/ubuntu/src/cmake-2.8.10.2/bin/cmake Build Clang and LLVM from source $ wget http://llvm.org/releases/3.2/llvm-3.2.src.tar.gz $ wget http://llvm.org/releases/3.2/clang-3.2.src.tar.gz $ tar xaf llvm-3.2.src.tar.gz $ tar xaf clang-3.2.src.tar.gz $ cp -R clang-3.2.src llvm-3.2.src/tools/clang $ cd llvm-3.2.src/ $ mkdir build $ cd build/ $ cmake -DCMAKE_INSTALL_PREFIX=/home/ubuntu/.local ../. $ make $ make install $ export PATH=~/.local/bin/:$PATH $ cd ../../ $ which clang # result: /home/ubuntu/.local/bin//clang Install Shadow $ sudo apt-get install gcc xz-utils make automake autoconf cmake tidy libtidy-dev libglib2.0 libglib2.0-dev dstat pdftk python2.7 python-matplotlib python-numpy python-scipy $ git clone https://github.com/shadow/shadow.git $ cd shadow $ ./setup dependencies $ ./setup build -g -i /home/ubuntu/src/libutp -i /usr/include/x86_64-linux-gnu/c++/4.7 -l /home/ubuntu/src/libutp --tor-prefix /home/ubuntu/src/tor --tor-lib utp $ ./setup install $ export PATH=~/.shadow/bin/:$PATH Simulate tiny network in Shadow $ cd resource/examples/scallion/ $ tar xf tiny-4GB-m1.large.tar.xz $ cd tiny-4GB-m1.large/ $ scallion Graph simulation results Add one line to beginning of analyze.py: import matplotlib; matplotlib.use('Agg') For additional traffic simulations: $ grep -v "\[traffic-" scallion.log > scallion-notraffic.log $ cd ~/src/ $ mkdir results $ cd results/ $ python ~/src/shadow/contrib/analyze.py parse --output uTP-1 ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-1/scallion.log $ python ~/src/shadow/contrib/analyze.py parse --output uTP-0 ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-0/scallion.log $ python ~/src/shadow/contrib/analyze.py parse --output vanilla ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-vanilla/scallion.log $ python ~/src/shadow/contrib/analyze.py plot --title "sjm217's utp branch (#9166)" --prefix "sjm217-utp" --data vanilla/ "vanilla 0.2.4.4-alpha" --data uTP-1/ "uTP for all links" --data uTP-0/ "uTP for none of the links" === END OF INSTRUCTIONS ===

1 1

Searchable metrics archive - Onionoo-like API available online for probing
by Kostas Jakeliunas 02 Sep '13

02 Sep '13

Hello, this accompanies my status report [1], and includes info how to query the searchable metrics archive for anyone curious. I also refer to the original (now semi-outdated) project proposal/document. [0] Only sending to tor-dev@for now. The Onionoo-like backend is listening on http://ts.mkj.lt:5555/ (backup URI where it's actually running (domain unrelated): ravinesmp.com:5555) This document details how it can be queried: https://github.com/wfn/torsearch/blob/master/docs/onionoo_api.md (It is, by design, an almost-subset (it does change some things as of now, though) of the Onionoo API [2].) Try not to DoS the thing, it is still fragile! :) (or, maybe do, but gently?) I will try to follow-up with a more detailed email, but this will have to do for now. Anyone curious should refer to the API doc. I'll soon add date range parameters, so it will become somewhat more useful. Examples: http://ts.mkj.lt:5555/details?search=mor http://ts.mkj.lt:5555/details?search=79. http://ts.mkj.lt:5555/statuses?lookup=9695DFC35FFEB861329B9F1AB04C46397020C… Please report any inconsistencies / errors / time-outs / anything that takes a few seconds or more to execute. I'm logging the queries (together with IP addresses for now - for shame!), so will be able to later correlate activity with database load, which will hopefully provide some realistic semi-benchmark-like data. [0]: http://kostas.mkj.lt/gsoc2013/gsoc2013.html [1]: https://lists.torproject.org/pipermail/tor-dev/2013-August/005310.html [2]: https://onionoo.torproject.org/ -- Kostas. 0x0e5dce45 @ pgp.mit.edu

2 3

Trac upgraded to 0.12.3
by Erinn Clark 01 Sep '13

01 Sep '13

Hello everyone! Weasel and I have upgraded https://trac.torproject.org to a newer version. Right now it has all of the functionality you were already accustomed to, but in the next few days it will get some more exciting features, like git integration. (I already verified that we are able to do it with our current git setup, but I will need to go through and poke at individual repos to make sure it works correctly.) Right now I'm just starting with a small list: arm, tor-browser-bundle, obfsproxy, stem, tor, torbrowser, and torbutton. (I tried bridgedb but trac freaks out because it doesn't like something about that repo; I will try to figure out what's wrong after I finish these others). If you want git integration for your repo(s), please file a ticket against the Trac component and I'll see what I can do! Once the git integration works, we'll demo a code review tool (http://trac-hacks.org/wiki/CodeReviewerPlugin) I'll email again once we're further along with both of these processes. In the meanwhile, let me know if you run into any issues with the new trac. Also, I would be remiss if I did not mention Mike's new favorite feature: a tag cloud. https://trac.torproject.org/projects/tor/tags Thanks, Erinn (& weasel)

2 2

Tor Control PHP library
by Kévin Dunglas 30 Aug '13

30 Aug '13

Hi, I've built a PHP library implementing the TOR control protocol: https://github.com/dunglas/php-torcontrol/ I've also built a Symfony framework bundle for this library: https://github.com/dunglas/DunglasTorControlBundle It allows to control a Tor server from a PHP script. Of course it's open source (MIT license). Can you add this library to the list ? -- Kévin Dunglas Freelancer http://dunglas.fr

2 2

Torsocks 2.0.0-rc1 release
by David Goulet 29 Aug '13

29 Aug '13

Greetings everyone! Two months ago I sent an email [1] to the tor-dev mailing list explaining some important issues with torsocks. I proposed myself to do a complete rewrite for the reasons detailed in the email. I've released today the first release candidate for a 2.0.0 version. https://github.com/dgoulet/torsocks Package: https://github.com/dgoulet/torsocks/archive/v2.0.0-rc1.tar.gz I'm asking the community for feedbacks, testing, code review, optimization design and/or anything useful to help improve the project (even correcting the mistakes or improving any comments). The TODO file in the repository contains some needed features but that are not critical to the project for a beta release. Hopefully, with your help, we will be able to replace the current version with a better one, more tested, more hardened and more portable!... and maybe with a more meaningful name :). What is really needed right now is OS X and BSD testers. I did my best to only use POSIX standard and making sure that libc calls are the same on all platforms but I was not able to test it on other OS other than Linux so they might some issues. There is *no* Windows support but again that would be amazing if someone could port it. Again, please contribute and help this very useful tool getting better! I encourage you all to use tor-dev mailing list for any development issues. I'm also on IRC in #tor-dev as "dgoulet". You can also find me on OFTC, freenode and indymedia servers. Big thanks to everyone! Using it is testing it! :) Cheers! David [1] https://lists.torproject.org/pipermail/tor-dev/2013-June/004959.html

4 7