- tor-dev - lists.torproject.org

Status report - Stream-RTT
by ra 14 Sep '13

14 Sep '13

tl;dr When building a circuit, measuring the RTT a single time could provide better latency and anonymity while not affecting throughput. Multiple measurements could be used for running real-time applications like VoIP or optimizing throughput. Despite the fact that the Tor network is currently in an unusual state so to say, I have been spending the last weeks looking into stream-RTT data of circuits. I gathered the data shortly before and at the beginning of the huge botnet usage. This is what I have found out: As assumed stream-RTT measurements of a single circuit are not at a fixed value but distributed since they are subject to multiple influences. After comparing stream-RTT distributions of multiple circuits, I found lots of different shapes and I realized that no single distribution fits them all. The Time-To-First-Byte (TTFB) for fetching a small website over HTTP is used to approximate the latency of a certain circuit. I used different methods to check the correlation between the RTT of a circuit and its TTFB - all indicating a very high correlation. Hence, stream-RTTs of a circuit make a good estimator for its TTFB and therefor its latency. In terms of latency, using a single stream-RTT measurement ("First-RTT") performs better than the currently used method CBT. So far I haven't done any testing/calculations on the other metrics: bandwidth and anonymity. I would assume the former to be unaffected by First-RTT. Latter could probably be slightly increased, if the percentage of discarded circuits would be reduced from 20% with CBT to 10% or 15% with First-RTT - while still achieving a minor improvement in latency. Nevertheless I would not recommend using First-RTT as method for providing low latency circuits to applications, because it only gives a small hint about the quality of a circuit and cannot make sure that some latency properties hold for a certain circuit. Nevertheless First-RTT works pretty well comparing to the minimum effort it takes. Additionally I played around a lot with methods to provide a better estimator for latency properties of a certain circuit. But they all need far more than a single measurement and are therefor out of scope for the common case. Besides they cannot protect against suddenly changing circuit conditions. But they could be used to fulfill a application specific maximum RTT for real-time applications like VoIP. With the use of similar techniques it should be possible to detect circuits that include a node that's within its bandwidth limit. This could be used for providing high bandwidth circuits for applications like BitTorrent. Best, Robert

1 0

Re: [tor-dev] Help with rebasing arma's n23-5 tor branch to current master
by Charlie Belmer 14 Sep '13

14 Sep '13

Hey Karsten & Dev team, I didn't mean to take this off channel, so my apologies for that. I ran a few more tests this week (and my apologies on the slowness of some of this work - life has been hectic this past month), but I am not finding much of use. I could not find any errors which are not also present in when N23 is disabled. If you want to browse the logs for yourself, I uploaded a partial log from a run to http://www.charliebelmer.com/scallion.n23.log(730MB). The most telling entries I found so far are below, but I see the same errors whether UseN23 is set to 1 or not, so it is likely not the root cause. I'll keep digging. Error 1 (Seems to occur when opening channel): 0:0:49:406270 [thread-2] 0:9:41:348157618 [scallion-info] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] bw_weights_error_t networkstatus_check_weights(int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int)(): Bw weight mismatch 1. G=214 M=12174 E=165 D=27654 T=40207 Wmd=4769 Wme=-724848 Wmg=0 Wed=461 Wee=734848 Wgd=4769 Wgg=10000 Wme=-724848 Wmg=0 0:0:49:407692 [thread-2] 0:9:41:348157618 [scallion-info] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] bw_weights_error_t networkstatus_check_weights(int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int64_t, int)(): Bw weight mismatch 1. G=214 M=12174 E=165 D=27654 T=40207 Wmd=4769 Wme=-724848 Wmg=0 Wed=461 Wee=734848 Wgd=4769 Wgg=10000 Wme=-724848 Wmg=0 Error 2: 0:0:54:227263 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void conn_read_callback(int, short, void *)(): socket 1000264 wants to read. 0:0:54:227281 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int connection_read_to_buf(connection_t *, ssize_t *, int *)(): 1000264: starting, inbuf_datalen 0 (0 pending in tls object). at_most 16448. 0:0:54:227302 [thread-2] 0:10:55:012000001 [shadow-debug] [4uthority1-73.1.0.0] [_tcp_endOfFileSignalled] 73.1.0.0:10021 (descriptor 1000264) <-> 91.1.0.0:9111: signaling close to user, socket no longer usable 0:0:54:227321 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] TLS error: unexpected close while reading (SSL negotiation finished successfully) 0:0:54:227338 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int tor_tls_read(tor_tls_t *, char *, size_t)(): read returned r=0, err=-8 0:0:54:227354 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int connection_read_to_buf(connection_t *, ssize_t *, int *)(): TLS connection closed on read. Closing. (Nickname nonexit10, address 91.1.0.0) 0:0:54:227373 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void channel_close_for_error(channel_t *)(): Closing channel 0x7f546d03ce40 due to lower-layer error 0:0:54:227389 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void channel_change_state(channel_t *, channel_state_t)(): Changing state of channel 0x7f546d03ce40 (global ID 21) from "open" to "closing" 0:0:54:227420 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void channel_remove_from_digest_map(channel_t *)(): Removed channel 0x7f546d03ce40 (global ID 21) from identity map in state closing (4) with digest BBF25A366EA5376B0026CE6EDE3DF536211429BE 0:0:54:227440 [thread-2] 0:10:55:012000001 [shadow-debug] [4uthority1-73.1.0.0] [tcp_close] 73.1.0.0:10021 (descriptor 1000264) <-> 91.1.0.0:9111: user closed connection 0:0:54:227452 [thread-2] 0:10:55:012000001 [shadow-debug] [4uthority1-73.1.0.0] [_tcp_setState] 73.1.0.0:10021 (descriptor 1000264) <-> 91.1.0.0:9111: moved from TCP state 'TCPS_CLOSEWAIT' to 'TCPS_LASTACK' 0:0:54:227473 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void connection_mark_for_close_internal_(connection_t *, int, const char *)(): Calling connection_mark_for_close_internal_() on an OR conn at /home/coffee/shadow/shadow/build/tor/src/or/connection.c:2932 0:0:54:227491 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int conn_close_if_marked(int)(): Cleaning up connection (fd -1). 0:0:54:227509 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void circuit_n_chan_done(channel_t *, int)(): chan to NULL/91.1.0.0:9111, status=0 0:0:54:227546 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] void channel_change_state(channel_t *, channel_state_t)(): Changing state of channel 0x7f546d03ce40 (global ID 21) from "closing" to "channel error" 0:0:54:227565 [thread-2] 0:10:55:012000001 [scallion-debug] [4uthority1-73.1.0.0] [scalliontor_logmsg_cb] int connection_remove(connection_t *)(): removing socket -1 (type OR), n_conns now 26 On Tue, Sep 3, 2013 at 10:47 AM, Karsten Loesing <karsten(a)torproject.org>wrote: > On 9/3/13 4:32 PM, Charlie Belmer wrote: > > Hi Karsten, > > > > I have run several tests, but I haven't been able to satisfy myself yet. > I > > attached the results from my latest round of tests for your review. I am > > comparing the n23 branch (UseN23 1 set in each torrc) with the initial > head > > I merged n23 with. It seems that for some reason, the n23 branch > completes > > far fewer downloads, though I haven't yet found the cause, or found > > anything interesting in the logs. > > > > $ grep "fg-download-complete" m1.large-tor/data/scallion.log | wc -l > > 8827 > > $ grep "fg-download-complete" m1.large-n23/scallion.log | wc -l > > 308 > > > > I would definitely appreciate any thoughts, insights, or advice! > > Interesting! Can you re-run the n23 simulation on info or even debug > log level? That might reveal some more information. You'll do that by > running "scallion --log-level=INFO". A tiny network should be > sufficient for that. Watch out for filling disks. > > If you want me to look at the scallion.log file, feel free to post it > somewhere, and I'll see if something looks unusual to me. > > Thanks, > Karsten > > > > On Tue, Sep 3, 2013 at 8:58 AM, Karsten Loesing <karsten(a)torproject.org > >wrote: > > > >> Hi Charlie, > >> > >> did you have the chance to simulate your rebased n23-5 branch using > Shadow? > >> > >> If not, maybe I can help by simulating it once and writing down more > >> specific instructions for you? > >> > >> Thanks! > >> Karsten > >> > >> > >> On 8/8/13 10:36 AM, Karsten Loesing wrote: > >>> (Moving this discussion back to tor-dev@ with Charlie's permission.) > >>> > >>> On 8/6/13 4:10 AM, Charlie Belmer wrote: > >>>> Hey Karsten, > >>>> > >>>> I've been reading up on Shadow & Scallion to do some performance > >> testing on > >>>> N23. Before I go too far down that road, I wanted to check in with > you. > >>>> > >>>> Are there existing blueprints or best practices I should be following? > >>> > >>> Thanks for looking into this! > >>> > >>> So, I wouldn't call the following notes blueprints or best practices, > >>> but they are a description of what I would do to run Shadow simulations > >>> of your branch. > >>> > >>> First of all, be sure to read the Shadow wiki which has contains lots > of > >>> interesting stuff: https://github.com/shadow/shadow/wiki > >>> > >>> The next step would be to get a vanilla Tor master (that your branch is > >>> based on) running in a minimal or tiny Shadow network. It's probably > >>> easiest to use a local Ubuntu VM for this. Or you can start an > m1.large > >>> EC2 instance for the tiny network, or an even smaller instance for the > >>> minimal network. > >>> > >>> Once that simulation succeeds, you'll want to run another simulation > >>> using your branch, extract performance metrics using Shadow's analyze > >>> script, produce a PDF with graphs also using the analyze script, and > >>> then wonder if things are faster than before. > >>> > >>> The minimal or tiny networks are mostly there to learn that things are > >>> working as expected. You'll probably want larger networks for real > >>> performance results, but starting small to get everything working is > >>> probably easier. > >>> > >>> Sounds doable, I guess. However, here's where things start to get > >>> tricky: in order to simulate Tor master (or a branch based on it) in > >>> Shadow, you'll have to perform quite a few extra steps like compiling > >>> your own CMake and Clang/LLVM. Not really hard, but you need to get > all > >>> pieces together before Shadow does what you want. > >>> > >>> I'm pasting detailed instructions that I used to simulate Steven's uTP > >>> branch in Shadow below as an example. Not everything applies to your > >>> branch, but it should give you the general idea. > >>> > >>> If something remains unclear, please feel free to ask! > >>> > >>>> Depending how it goes, I'd also look at re-trying adaptive 23 or > tuning > >> the > >>>> existing implementation. > >>> > >>> Neat! We should probably include Mashael in that discussion once you > >>> have a working and simulated n23-5 branch. Maybe she has ideas what > >>> parts need tweaking. I'm cc'ing her in this mail so she knows what > >>> you're working on. > >>> > >>> In any case, having a working rebased n23-5 branch is already a big > step > >>> forward. > >>> > >>> Thanks! > >>> Karsten > >>> > >>> > >>> === START OF INSTRUCTIONS === > >>> > >>> Launch new EC2 instance running Ubuntu Server 13.04 64 bit > >>> > >>> Set ulimit -n to 25000, see Shadow wiki for details > >>> > >> > https://github.com/shadow/shadow/wiki/Preparing-your-machine#system-configs… > >>> > >>> $ sudo apt-get update > >>> $ sudo apt-get upgrade > >>> > >>> Clone and test-compile modified libutp > >>> > >>> $ sudo apt-get install build-essential git automake libssl-dev > >> libevent-dev > >>> $ mkdir src > >>> $ cd src/ > >>> $ git clone https://github.com/sjmurdoch/libutp > >>> $ cd libutp/ > >>> $ make > >>> $ cd ../ > >>> > >>> Clone and test-compile Tor branch utp > >>> > >>> $ git clone https://git.torproject.org/tor.git > >>> $ cd tor/ > >>> $ git remote add sjm217 https://git.torproject.org/sjm217/tor.git > >>> $ git fetch sjm217 > >>> $ git checkout -b utp sjm217/utp > >>> $ ./autogen.sh > >>> $ LDFLAGS="-L/home/ubuntu/src/libutp" > CFLAGS="-I/home/ubuntu/src/libutp" > >>> LIBS="-lutp -lrt" ./configure --disable-asciidoc > >>> $ make > >>> $ make distclean > >>> $ cd ../ > >>> > >>> Modify Tor to use uTP for all links (or "0 &&" for not using uTP for > any > >>> link) > >>> > >>> diff --git a/src/or/channeltls.c b/src/or/channeltls.c > >>> index 0551b73..b7b36e1 100644 > >>> --- a/src/or/channeltls.c > >>> +++ b/src/or/channeltls.c > >>> @@ -418,7 +418,7 @@ channel_tls_connect(const tor_addr_t *addr, > uint16_t > >>> port, > >>> /* Create a uTP connection */ > >>> tor_addr_to_sockaddr(addr, port, (struct sockaddr*)&sin, > sizeof(sin)); > >>> tor_addr_to_str(addr_str, addr, sizeof(addr_str), 0); > >>> - if (!strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { > >>> + if (1 || !strncmp(addr_str, "128.232.10.129", sizeof(addr_str))) { > >>> log_info(LD_CHANNEL, > >>> "Trying uTP connection to %s", addr_str); > >>> tlschan->utp = UTP_Create(tor_UTPSendToProc, tlschan, (const > struct > >>> sockaddr*)&sin, > >>> > >>> Install CMake 2.8.10 > >>> > >>> $ wget http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz > >>> $ tar xf cmake-2.8.10.2.tar.gz > >>> $ cd cmake-2.8.10.2/ > >>> $ ./configure > >>> $ make > >>> $ export PATH=~/src/cmake-2.8.10.2/bin:$PATH > >>> $ cd ../ > >>> $ which cmake # result: /home/ubuntu/src/cmake-2.8.10.2/bin/cmake > >>> > >>> Build Clang and LLVM from source > >>> > >>> $ wget http://llvm.org/releases/3.2/llvm-3.2.src.tar.gz > >>> $ wget http://llvm.org/releases/3.2/clang-3.2.src.tar.gz > >>> $ tar xaf llvm-3.2.src.tar.gz > >>> $ tar xaf clang-3.2.src.tar.gz > >>> $ cp -R clang-3.2.src llvm-3.2.src/tools/clang > >>> $ cd llvm-3.2.src/ > >>> $ mkdir build > >>> $ cd build/ > >>> $ cmake -DCMAKE_INSTALL_PREFIX=/home/ubuntu/.local ../. > >>> $ make > >>> $ make install > >>> $ export PATH=~/.local/bin/:$PATH > >>> $ cd ../../ > >>> $ which clang # result: /home/ubuntu/.local/bin//clang > >>> > >>> Install Shadow > >>> > >>> $ sudo apt-get install gcc xz-utils make automake autoconf cmake tidy > >>> libtidy-dev libglib2.0 libglib2.0-dev dstat pdftk python2.7 > >>> python-matplotlib python-numpy python-scipy > >>> $ git clone https://github.com/shadow/shadow.git > >>> $ cd shadow > >>> $ ./setup dependencies > >>> $ ./setup build -g -i /home/ubuntu/src/libutp -i > >>> /usr/include/x86_64-linux-gnu/c++/4.7 -l /home/ubuntu/src/libutp > >>> --tor-prefix /home/ubuntu/src/tor --tor-lib utp > >>> $ ./setup install > >>> $ export PATH=~/.shadow/bin/:$PATH > >>> > >>> Simulate tiny network in Shadow > >>> > >>> $ cd resource/examples/scallion/ > >>> $ tar xf tiny-4GB-m1.large.tar.xz > >>> $ cd tiny-4GB-m1.large/ > >>> $ scallion > >>> > >>> Graph simulation results > >>> > >>> Add one line to beginning of analyze.py: > >>> > >>> import matplotlib; matplotlib.use('Agg') > >>> > >>> For additional traffic simulations: > >>> > >>> $ grep -v "\[traffic-" scallion.log > scallion-notraffic.log > >>> > >>> $ cd ~/src/ > >>> $ mkdir results > >>> $ cd results/ > >>> $ python ~/src/shadow/contrib/analyze.py parse --output uTP-1 > >>> > >> > ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-1/scallion.log > >>> $ python ~/src/shadow/contrib/analyze.py parse --output uTP-0 > >>> > >> > ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-utp-0/scallion.log > >>> $ python ~/src/shadow/contrib/analyze.py parse --output vanilla > >>> > >> > ~/src/shadow/resource/examples/scallion/tiny-4GB-m1.large/data-vanilla/scallion.log > >>> $ python ~/src/shadow/contrib/analyze.py plot --title "sjm217's utp > >>> branch (#9166)" --prefix "sjm217-utp" --data vanilla/ "vanilla > >>> 0.2.4.4-alpha" --data uTP-1/ "uTP for all links" --data uTP-0/ "uTP for > >>> none of the links" > >>> > >>> === END OF INSTRUCTIONS === > >>> > >>> > >> > >> > > > >

2 2

Hidden Service location referrals in DNS/web
by Jeroen Massar 13 Sep '13

13 Sep '13

Hi, I don't know what the current status/discussion for this is, hence this email, a reply to a ticket # or mail thread would be fine too ;) Is there currently a plan or way to state for a site: "you can find us as hidden service X" ? In DNS we could use something like: www NAPTR 10 50 "s" "TOR+HTTPS" "" _https._tcp _https._tor. SRV 0 0 443 <fingerprint>.onion. Of course, there is a latency hit there, DNSSEC should be used for authentication and if there is somebody watching DNS they know that you have something that looks at Tor records, thus it might not be ideal for all situations. Having a Torified DNS resolver would resolve that though. Using _https._tor. allows other services to also use this scheme. When 'datagram tor' comes out we can add a "_dtor" or something to support datagram variants of the connections. Another way, which does not show that you are looking for Tor would be to use the "alternate" link ability of HTML, eg for RSS there is: <link rel="alternate" type="application/rss+xml" title="RSS version" href="/rss/" /> Thus we could have something like: <link rel="alternate" type="application/tor+hiddenservice" title="Tor hidden service version" href="https://<fingerprint>.onion" /> on a webpage. If the webpage you get that on is https you have assurance that the hiddenservice is truly there. That would be webpage only though, but that is quite a large portion of the web... Greets, Jeroen

1 0

Thoughts on Proposal 203 [Avoiding censorship by impersonating a HTTPS server]
by Kevin Butler 13 Sep '13

13 Sep '13

(This email got way out of hand from a basic 'I'll bounce an idea here', here's to hoping I haven't made some huge oversight.) I've been thinking about the https frontend after reading the basic problem when I started looking into Tor dev but never took the time to read the actual proposal. When I got some basic idea around how to solve the core problem, I took the time to take a read and it turns out the proposal actually has 90% of what I could think of, but I'm glad I took some time to think from a (I hope) fresh perspective. So anyway, on point. I think Designs #2&3 are the best ideas for Proposal 203 (probably leaning toward #2 more). They're basically the same concept anyway. I came to the same conclusion that we definitely need a shared key to be distributed per bridge address for this to work in any fashion, ideally these keys could be rotated frequently. I also totally agree with the server being a key implementation detail, ideally we want something drop in that could go alongside an existing website. As for content I think mock corporate login pages are a neat idea, while mock private forums are not. Regarding authentication and distinguishability, I don't agree with trying to distinguish Tor clients from non-Tor based on anything the client initially sends, as any sort of computation that isn't webserver-y could be a timing attack or otherwise. I have some specific ideas around how we can implement this to address the issues/concerns outlined in the current proposal. I think the best course of action is to use a webserver's core functionalities to our advantage. I have not made much consideration for client implementation. But here are some thoughts on how we could potentially achieve our goals: - Shared secrets are shared with users whenever bridge IPs are exchanged, it is necessary for these to be large random values and not user-like passwords. (As one of the Authorize proposals also mentions) This exchange would ideally give a domain name for the bridge so as we're not trying to connect to an IP, but to reduce user error the domain and key should be concatenated and base64d so it's a single copy/paste for the user without them trying to navigate to a url thinking it's a Tor enabled link or something. - The users Tor client (assuming they added the bridge), connects to the server over https(tls) to the root domain. It should also downloads all the resources attached to the main page, emulating a web browser for the initial document. - The server should reply with it's normal root page. This page can be dynamically generated, no requirement for it to be static, the only requirement is that one of the linked documents (css, js, img) be served with a header that allows decent caching (>1hr). The far future file could be the document itself but it doesn't have to be. - (This part is probably way too trashy to the server performance, I'm winging it as I can think of it.) - For all files included in the main document, whichever has the furthest future cache header, we'll call that file F. - If we have precomputed the required values (see below) for F, then we are ok to move to the next step (see next bullet point), otherwise, serve all of the files with cache headers under one hour. - If F doesn't have the pre-computations ready, this is the time to spin off a subprocess to start calculating stuff (at lowish cpu priority probably). - The subprocess should calculate an intensive function (e.g. scrypt) of hash(contents of F xored with the shared key) for (X...Y) iterations, inclusive. X and Y should be chosen so that X is on the magnitude of seconds to compute while Y is a couple of thousand iterations above it. Store a 'map' of numberOfIterations => { result, hmac(result + tls cert identifier) }. The hmac should be keyed with the shared secret. The tls cert identifier should probably be its public key or signature? It should store these results in fast cache (hopefully in memory). - So we have our file F, and a precomputed value Z which was the function applied Y times and has a hmac H. We set a cookie on the client base64("Y || random padding || H") - The server should remember which IPs which were given this Y value. This cookie should pretty much look like any session cookie that comes out of rails, drupal, asp, anyone who's doing cookie sessions correctly. Once the cookie is added to the headers, just serve the document as usual. Essentially this should all be possible in an apache/nginx module as the page content shouldn't matter. - Here's a core idea: the server has a handler setup for each of the Z values, hex encoded is probably best (longer!) e.g. /FFFEF421516AB3B2E42... - The webserver should be setup to accept secure websocket upgrades to these urls and route the connection to the local Tor socket. - If the iteration value for the given url is not the same as the one given to the ip trying the path, or the iteration value doesn't match Y, the connection should be dropped/rejected. (This can be legitimate) - If the connection is accepted, the current Y value should be decremented. If Y < X for the current F then we should rotate our keys. (This is a bit of a question, we could manipulate one of the files, but that interferes with the website and could cause distinguishability). - Basically after Y-X Tor clients (not related to how many https users are served), we should be rotating our keys incase the keys leaked, or changing handlers to stop old handlers being used. - When rotating keys we should be sure to not accept requests on the old handlers, by either removing them(404) or by 403ing them, whatever. The decrementing of Y is to try to make replay attacks less feasible, although that would mean tls was broken if they were able to get the initial value, but fuck, who knows with breach & crime et cetera. - (Best read the rest before reading this part: To reduce key churn, or allow long term guard-like functionality, the old handlers could be saved and remain unique to a single ip, by sending cookies from client to server that are a unique id accepted from that ip, the server could know to use an old shared key or something so the client wouldn't blacklist them. Or the client could know not to blacklist previously successful bridges by remembering their tls cert or something. I haven't really thought much about this, but it's probably manageable.) - The idea here is that the webserver (apache/nginx) is working EXACTLY as a normal webserver should, unless someone hits these exact urls which they should have a negligable chance of doing unless they have the current shared secret. There might be a timing attack here, but in that case we can just add a million other handlers that all lead to a 403? (But either way, if someones spamming thousands of requests then you should be able to ip block, but rotating keys should help reduce the feasability of timing attacks or brute forcing?) - So, how does the client figure out the url to use for wss://? Using the cache headers, the client should be able to determine which file is F. If all files are served with a cache header under one hour, then we wait a time period T. Realistically, if the Tor client knows this is a bridge, the only reason this wait should happen is if precomputings happening, so it should just choose another bridge to use... or wait minutes and notify the user that it's for good reason. - Assuming we get a valid F, we look at our cookies. For all cookies, if they're base64, convert to binary, then, try treating the first K bytes (we should have an upper bound for Y, lets say it's probably an 8 byte unsigned long) as a number I. We replicate the computations that the server would have done to get our Zc(lient). - Using this Zc, and the cert provided by the server, we can compute our local Hc. If Hc doesn't match the last (Length of hmac used) bytes in the cookie then try the next cookie. - If no cookie matches, then we either have an old key or we're being MITMd (the computation was ok but the cert didn't match). In these cases, we should fake some user navigation for a couple of pages then close the connection and blacklist the bridge (run for the hills and don't blow the bridge!). - If we get a match, then we know Zc, so we upgrade the connection to wss://domain/Zc which should be a valid secure websocket connection (usable as tcp) unless another ip was already accepted on this iteration value then the server should reject us. If we get rejected at this stage, we know the server had good reason (trying to stop replays) so we just retry from the start and cross our digital fingers. (If bridges are sufficiently private then this should be a non-issue as it will likely only happen with 2 Tor clients connecting within the same second or so) - At this point there should be an encrypted tcp tunnel between the Tor client and the bridge's apache/nginx, and an unencrypted connection between the webserver and the bridge's Tor socket. Should be able to just talk Tor protocol now and get on with things. So to summarise, - Using general web tools to negotiate, secret paths, headers and cookies - Proof of work-ish system using the shared key to establish a unique url - Checking for a MITM and allowing key rotation by using our shared key with a hmac to determine: - If the provided certificate matches what the server thought it gave us - If our result is correct with our current key - Assuming it's sound, I think the serverside could be implemented as an apache module that could be a relatively easy drop in. Concerns: - Distinguishability of client https & websockets implementation. - Content for servers - Everything above as I'm sure theres obviously critical flaw I'm overlooking! - Amount of work it would take on client side :( Rym

2 3

RFC: Lightweight Obfuscated Datagram Protocol (LODP)
by Yawning Angel 13 Sep '13

13 Sep '13

Hello all, Inspired by a conversation with arma, I have been working on designing a UDP based protocol to serve as the bulk data transport for something along the lines of "obfs3, but over UDP". The spec document is approaching the point where I would like to have increased public review before I go and implement it. People who sit in #tor-dev probably have seen various revisions of this draft already. Some notes: * This is heavily inspired by SSU from the i2p project, and Dust by Brandon Wiley. * Ensuring that data gets delivered reliably and in order is outside the scope of this document. Such things are a well researched problem so my reference implementation will likely draw upon an existing protocol instead of reinventing the wheel here. * As far as existing work in this area, the only other protocols that attempt to do similar things are i2p's SSU and Dust. * LODP assumes that there is a shared secret between peers to attempt to thwart active probing attacks. * As part of my evil plans, "secure fingerprinting resistant UDP" is something useful to have. * Open questions/things that are subject to change are tagged with XXX. Questions, comments, feedback appreciated. -- Yawning Angel

1 1

Pluggable transport weekly meeting - Friday Sept 13
by vmonmoonshine＠gmail.com 13 Sep '13

13 Sep '13

Hello pluggable transport devs, As there was no objection to the proposed time, we are going to have our first one today/tomorrow: Friday - Sept 13 CEST: 18:00 BST (Summer GMT): 17:00 UTC: 16:00 EST: 12:00 MNT: 10:00 PST: 9:00 @ #tor-dev on irc.oftc.net And stick to it in future. The format will be scrum-esque. At the beginning we'll ask who likes to participate. Then we ask each participant in some order to give a short update on what they have done last week, what will be doing in the coming week and if they have any questions/problem they want to ask from/share with other developers. We hope to be able to keep it under 30min, so if you find your peer for deeper detailed discussion then you might want to keep the discussion for after the meeting. Thanks! Hopefully See you all, Vmon

1 0

Pluggable transport weekly meeting
by Vmon 13 Sep '13

13 Sep '13

I sent this email quite a while ago and I was surprised that nobody was interested/replied. Today I found out that I had sent it to a wrong address. But here we are, so I'm sending it again. So please reply so we can kick this off soon. Thanks, Vmon ---------- Forwarded message ---------- From: <vmonmoonshine(a)gmail.com> Date: Wed, Aug 21, 2013 at 6:46 AM Subject: Pluggable transport weekly meeting To: tor-dev-request(a)lists.torproject.org Hello Tor devs, Following up on what we came up with in the dev summit ( https://trac.torproject.org/projects/tor/wiki/org/meetings/2013SummerDevMee… ), we are going to have weekly 30-minute IRC meeting focusing on pluggable transports. The format (I think) will be scrum-esque that every developer who is working on a pluggable transport will update everybody else about the work they did/are doing on their transport during the week and ask questions if they have any, for example if they got stuck somewhere and they think somebody can help. Preliminary, we decided to have the meeting on Fridays, cause why not, but if you have serious problem with Fridays then we might be able to pick a better day. For the time of the meeting, considering the geographical positions of the current transport developers, we'll probably end up having a CEST evening and PST morning meeting. Having that in mind I suggest: CEST: 18:00 BST (Summer GMT): 17:00 EST: 12:00 MNT: 10:00 PST: 9:00 So if this doesn't work for you, please reply to this email with your alternative proposal. Thanks, Vmon

7 9

Pluggable Transports and rate limiting
by George Kadianakis 11 Sep '13

11 Sep '13

Hey Yawning (and tor-dev), a topic that we will soon need to consider seriously is rate limiting of pluggable transports. For example, Obfsproxy at the moment does not understand rate limiting and will happily read and write as many bytes as needed. After some discussions in IRC and #3587, we decided to add rate-limiting commands to the TransportControlPort -- that's the (unimplemented) real-time sister of ExtendedORPort. You can see the details in the relevant proposal: https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/196-transpor… (see RATE_LIMIT command etc.) The idea here was that Tor would monitor its bandwidth usage and order PTs to slow down appropriately. If an accounting limit was hit, Tor would order PTs to stop pushing traffic completely. Does this look like a reasonable way to enforce accounting/rate-limiting policies? How could it be improved? For example, I'm kind of sad about this approach, because it requires little-t-tor to do all the hard work of monitoring bandwidth usage and giving out intelligent rate-limiting orders. For example, how do we rate-limit N pluggable transports to 100 kb/s? Do we allow each pluggable transport to push 100/N kb/s? I don't like that this logic must be implemented in little-t-tor. Also, do the TransportControlPort rate-limiting commands look reasonable? Is the payload of RATE_LIMITED (two integers: bandwidth rate and bandwidth burst) properly designed, or does it need more information? Also also, will this work nicely with transport combos (#7167)?

3 3

Tor Cloud is broken - please help debug?
by Runa A. Sandvik 09 Sep '13

09 Sep '13

Hi everyone, It seems something is wrong with the Tor Cloud [1] instances [2]. My best guess is that the instances are failing to auto-update packages and are left in a state where tor is not running, new packages are not being installed, and the instances are not doing anything useful. I am not able to look into this issue until next week. Does anyone want to poke at it sooner? The best ways to debug this would be: * Set up a Tor Cloud instance (open all the right ports and make sure you have a set of SSH keys for it), connect to it and look through the logs. I'm guessing the package update process is waiting for user input or something similar. * Set up an Ubuntu cloud instance and go through the Tor Cloud setup [3] manually to see when/if something breaks. [1]: https://cloud.torproject.org/ [2]: https://metrics.torproject.org/network.html#cloudbridges [3]: https://gitweb.torproject.org/tor-cloud.git/blob_plain/HEAD:/ec2-prep.sh -- Runa A. Sandvik

1 0

dynamic changes to ORPort set (via controller)
by Zack Weinberg 09 Sep '13

09 Sep '13

Suppose I have a relay (not a client) which starts out with one open ORPort and a local controller. 1) Is there a better way (read "supported in STEM") than using SETCONF to add ORPort lines, to direct this relay to open additional ORPorts? 2) Will adding or removing ORPorts at runtime affect established connections to other ORPorts (those which are open before and after the change)? thanks zw

2 2