- tor-dev - lists.torproject.org

Request for comment: scheduler heuristics
by Andrea Shepard 09 Nov '13

09 Nov '13

As some of you may be aware, I've been working on new scheduler code for Tor recently. Currently, Tor calls channel_flush_some_cells() whenever a channel becomes able to accept more writes, or channel_write_*_cell() whenever a new cell arrives on a channel able to accept cells without other queued cells on circuits. Thus, we can only see one channel at a time when making scheduling decisions, and the existing circuitmux code is only concerned with providing an API for implementing prioritization of circuits attached to a particular channel. The new scheduler aims to remove this limitation. To do this, we have to get a global view of all possible choices we could make about which circuit to get a cell from next, so instead we keep lists of channels which can currently accept writes and which currently have cells available to send, and trigger the scheduler using event_active() whenever a channel meets both conditions. Then the scheduler runs from a libevent callback and loops over the list of channels in the intersection of both sets. At present it just invokes the existing circuitmux code, so the behavior differs little from the old code, but this provides a framework where we can potentially see multiple channels simultaneously to make scheduling decisions. Notice, though, that as described it will rarely actually see more than one pending channel - just looping over them all every time is too greedy. We want to put off making those choices long enough to see all our options, without delaying so long the channel send queues downstream of the scheduler loop start emptying and the delayed choice actually causes increased latency. At first, NickM and I had discussed this and he suggested a global high/low- water mark system parallel to the ones for each connection; I added code to the channel layer to allow lower layer implementations to provide queue size and overhead estimates in a generic way, so now the channel code has an up to date estimate of the total number of bytes queued to send on all channels in both lower-layer queues and channel_t queues. I believe this design should not be used, because making the flow rate on one channel affect the scheduling of others like that is a potential DoS. If an attacker controls one node, he can build a circuit through a target node to that node, and then send traffic down it, but force the node to drain it only very slowly, keeping the target's queue size always above a minimum level. If the low-water mark for the global queue is set higher than the high-water mark for each connection it can prevent this, but remains vulnerable to a similar attack using multiple channels to different nodes. The simplest proposed fixes involve modifying the queue size used to control the scheduler to age out old contributions from non-draining or slow-draining channels - replace the actual queue size with an effective queue size which drains linearly at a some fraction of the node's expect bandwidth, or drops off exponentially. These are straightforward to implement, but I'm unsure what the best option is, so comment is invited. -- Andrea Shepard <andrea(a)torproject.org> PGP fingerprint (ECC): BDF5 F867 8A52 4E4A BECF DE79 A4FF BC34 F01D D536 PGP fingerprint (RSA): 3611 95A4 0740 ED1B 7EA5 DF7E 4191 13D9 D0CF BDA5

1 0

Re: [tor-dev] [tor-assistants] Please help me plan and run meetings in ways that you'll find most helpful
by matt＠pagan.io 08 Nov '13

08 Nov '13

> We've been doing regular meetings for little over a month now, and I'd > like to get some feedback and solicit suggestions, complaints, and > ways to make the meetings suck the least they can. Please let me know > what's working for you, what isn't working, and what should be better. > Thanks for setting up the development calendar. I expect that is going to help me keep better track of scheduled meetings. I only found out about the calendar on IRC yesterday when you mentioned it, Was there an email I missed where it was announced? Matt Pagan

3 2

[Draft Proposal] Scalable Hidden Services
by Matthew Finkel 07 Nov '13

07 Nov '13

Hi everyone, This is a proposal I wrote to implement scalable hidden services. It's by no means finished (there are some slight inconsistencies which I will be correcting later today or tomorrow) but I want to make it public in the meantime. I'm also working on some additional security measures that can be used, but those haven't been written yet. Many thanks to George for his initial feedback. I pushed this version to my public repo, and will continue to push updates there. In reality this is really 3.2 proposals, so the end result, if accepted, will look significantly different, but I'm indecisive at this point about which of the designs is least dangerous. Thanks, Matt

5 8

Re: [tor-dev] bananaphone obfsproxy module
by David Stainton 07 Nov '13

07 Nov '13

>> It wouldn't surprise me if the obfsproxy API is suboptimal or broken >> in some ways. If you find a problem, please let me know. I need a way for the BananaphoneTransport to build the markov model before the initial client connect... but right now it looks like __init__ is called upon connect. validate_external_mode_cli runs during setup before the connect... should I use that to set a BananaphoneTransport class variable to the markov model?

2 2

Please help me plan and run meetings in ways that you'll find most helpful
by Tom Lowenthal 07 Nov '13

07 Nov '13

Hi, my name is Tom, and I run meetings. There, I've said it. I've admitted it, and that's the first step on the road to recovery. We've been doing regular meetings for little over a month now, and I'd like to get some feedback and solicit suggestions, complaints, and ways to make the meetings suck the least they can. Please let me know what's working for you, what isn't working, and what should be better. In a slight departure from the norm, I'm even interested in non-constructive criticism: if something isn't working for you, but you don't know how to fix it, please still tell me so that I can try to come up with a way to mitigate it. Thanks for your feedback, -Tom

1 0

torsocks-v2.0.0-rc3 NetBSD improvements
by Thomas Klausner 06 Nov '13

06 Nov '13

Hi! I've patched torsocks-v2.0.0-rc3 to compile on NetBSD. There is one warning left: syscall.c: In function 'handle_mmap': syscall.c:88:9: warning: cast from pointer to integer of different size and I haven't done any functional tests yet. What is the easiest way to do that? Attached are the patches I've used to get this far. Some additional comments: The package installs a few files where it shouldn't: share/DEBUG share/SOCKS5 share/socks-extensions.txt These belong in share/doc/torsocks instead, I guess. And what is the point of src.old inside the tarball? Cheers, Thomas

3 3

Bug 9665
by Fábio Bertinatto 05 Nov '13

05 Nov '13

Hi, I would like to start contributing to Tor development thus I picked up a bug and tried to fix it. It's the 9665: https://trac.torproject.org/projects/tor/ticket/9665 I don't know whether I understood the problem well or not but I came up with the attached (trivial) solution. Could someone please check if it's ok? Thank you. -- F. Bertinatto

2 1

Re: [tor-dev] bananaphone obfsproxy module
by David Stainton 04 Nov '13

04 Nov '13

Howdy, Thanks. Your obfsproxy is a nice piece of work. Bananaphone + Obfs2 sounds cool! Modular transport chains make a lot of sense... I like modular transports... recently for fun I wrote a VPN in Python Twisted [https://github.com/david415/hushVPN] using twisted consumers and producers. My idea was to have a modular system where transports could easily be swapped. Over UDP it's very fast and I was thinking that maybe I could reuse some of that code to write packet transports for obfsproxy. For instance Tor over ICMP, DNS etc... Obfs2 over Bananaphone over UDP? Right now the david-bananaphone branch is functional code. I've tested it with ssh. SSH over Markov chains! It's laggy but it works. Leif's design plans are interesting... He will be better at explaining it. Perhaps it does something similar to your idea of RHE + Obfs2. Briefly: RHE(CRYPTO(HAMMERTIME(TOR))) - RHE is Banaphone's markov or random weighted Reverse Hash Encoding - CRYPTO is NaCl crypto_stream - Hammertime: "adds chaff to a bytestream to impede passive timing analysis" The goal is to not have any distinguishable patterns in the byte stream. Is that what Obfs2 accomplishes? We will do the crypto_stream secret key exchange using NaCl crypto_box and Elligator; spewing noise after keys are sent Elligator etc... We were thinking to first implement the key exchange without Elligator... and do that part later. I'm working on the basic key exchange to work in branch david-bananaphone-keyexchange https://github.com/david415/obfsproxy/tree/david-bananaphone-keyexchange I'm currently troubleshooting a problem with the message passing... Maybe I do not understand your api well enough... I'm getting strange behavior. It would help if another pair of eyes would look at this. Perhaps later I will formulate a question regarding the obfsproxy api... And I was hoping Leif would help me fix this branch: david-bananaphone-nacl-hammertime https://github.com/david415/obfsproxy/tree/david-bananaphone-nacl-hammertime I wrote a naclStreamEncoder coroutine and made hammertime_rh_server return coroutine chains with the NaCl stream encoders. It doesn't work at all... really a rough draft scribbling of code that makes the BananaPhoneBuffer unit tests through an exception... OK. I will add ALL commandline options to obfsproxy-bananaphone! OK... bandwidth overhead calculations... I'll take a look and run some tests. Onward! David On Tue, Oct 29, 2013 at 7:19 PM, George Kadianakis <desnacked(a)riseup.net> wrote: > Greetings, > > I'm the guy who develops obfsproxy. I recently noticed that you are > writing a bananaphone module for it. The code looks good and you even > understood the pretty-much undocumented obfsproxy API! Nice! > > BTW, I thought I should let you know that we are currently writing > software that combines transports together in chains. This means that > you don't have to worry about data redundancy in bananaphone, since > you can combine it with an obfuscating transport like obfs2 to create > the following chain: > tor data -> |obfs2 encoding| ->|bananaphone encoding| -> tor bridge -> |bananaphone decoding| ->|obfs2 decoding| -> tor data > This would create a bananaphone-encoded channel which transfers > obfs2'ed Tor data (so the bananaphone traffic will look uniform, since > obfs2 data is uniform (whether uniform bananaphone traffic is useful for > censorship circumvention is a different story))). > > FWIW, we are still in the process of writing this transport > combiner. You can read more about it in: > https://trac.torproject.org/projects/tor/ticket/7167 > https://trac.torproject.org/projects/tor/ticket/9744 > https://gitweb.torproject.org/pluggable-transports/obfs-flash.git > We should also create a Wiki page about it soon, I guess. > > Anyway, this transport combiner business means that you only have to > worry about making the bananaphone transport work. It would also be > nice if the transport was configurable, so that we can tweak its > parameters easily. I would also be interested in some bandwidth > overhead calculations :) > > BTW, feel free to send me any questions you have about obfsproxy or > pluggable transports. > > Cheers! > > PS: Feel free to CC the public tor-dev mailing list in your reply if > you want.

2 2

[RELEASE] Torsocks 2.0.0-rc3
by David Goulet 03 Nov '13

03 Nov '13

Hi everyone! I just pushed the 2.0.0-rc3 tag for the torsocks rewrite. https://github.com/dgoulet/torsocks/archive/v2.0.0-rc3.tar.gz Important fixes got in especially with the testing done with firefox lately. The good news is that it works well with firefox now :). Of course, you should ALWAYS use the TBB :). Support for x86 is also fixed as well as GNU/kFreeBSD and s390/s390x. Some unit tests were contributed by Luke Gallagher. Also, thanks to Lunar, there is now a Debian experimental package! http://packages.debian.org/experimental/torsocks There are several improvements on the way also that will make it probably in the 2.1.x version. * Optimistic data * Add --only-onions option Only sockify .onion address. Asked by the Tails folks. * Support .i2p address https://lists.torproject.org/pipermail/tor-dev/2013-November/005714.html * Maybe (hopefully), rename the whole thing with a more meaninful name. (Debate to come on tor-dev ML at some point in time :) * Basic UDP DNS support. Finally, to make this project go forward, please consider helping by reviewing and testing this code base. Here is the open ticket ref.: https://trac.torproject.org/projects/tor/ticket/10007 Here is the ChangeLog for 2.0.0-rc3. 2013-11-03 torsocks 2.0.0-rc3 * Fix: add fixtures directory to EXTRA_DIST * Fix: add fixtures.h to makefile.am as EXTRA_DIST * Fix: remove DBG call in syscall() * Fix: don't lookup symbol if already found * Lookup symbols in libc in constructor * Fix: bad ret value check for default logging * Add compatibility with GNU/kFreeBSD * Fix compatibility with s390 and s390x architectures * Fix: missing registry unlock on error * Fix: undefined syscall values for x86 * Fix: check fileno returned value * Fix: torsocks.in libdir path with lib prefix * Tests: add config-file tests * Tests: add utils unit tests Thanks everyone! David

1 0

Re: [tor-dev] .i2p address support in torsocks
by str4d 02 Nov '13

02 Nov '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 11/03/2013 06:58 AM, David Goulet wrote: > Apparently, I failed to put the person in CC :). I subscribed to tor-dev after talking with you :) > On 02 Nov (13:47:56), David Goulet wrote: >> On 02 Nov (19:25:42), Maxim Kammerer wrote: >>> On Sat, Nov 2, 2013 at 5:58 PM, David Goulet >>> <dgoulet(a)ev0ke.net> wrote: >>>> For now, it would only be .i2p address support (like .onion). >>>> In torsocks, it's not that difficult to support both >>>> addressing. >>> >>> Does I2P's SOCKS proxy work in a way that's similar to Tor? >>> Other proxies in I2P are protocol-specific — e.g., ports >>> 4444/4445 for HTTP(S) and 6668 for IRC. I am quite sure that >>> protocol-specific local I2P proxies like HTTP and IRC strip >>> sensitive information, so providing the user with an easy >>> access to .i2p services via SOCKS might be the wrong thing to >>> do. The SOCKS information page [1] is rather scarce on details >>> of what actually goes on inside the proxy, however. >> >> For now, it's simply detecting an .i2p address, opening a >> connection to the i2p daemon and pushing the request there. The >> person at i2p I talked to told me that it's quite straight >> forward and no special SOCKS5 mangling would be needed. Specifically, the I2P tunnel runs a local SOCKS server socket that handles the standard SOCKS4/4a and SOCKS5 protocols, and any data to be sent to the .i2p address is forwarded to an I2P socket. The standard (unfiltered) SOCKS client tunnel only forwards data between the SOCKS and I2P sockets, this should be functionally similar to Tor. We also have a variant that filters IRC traffic for anonymity (using SOCKS to connect to I2P IRC networks is a common use case), but this occurs during the forwarding step, the SOCKS server socket is unchanged. If a non-.i2p address is detected, the SOCKS server opens a SOCKS client connection to a configured SOCKS outproxy. This is designed to enable chaining to Tor, but it has some bugs that need fixing (this not working reliably is why I looked into torsocks). The configured SOCKS outproxy also needs to be listening inside I2P, which adds latency. If torsocks supported separate SOCKS ports for each configured .suffix, a user could run both I2P and Tor locally and use them in parallel instead of in series. >> If there is some work to do on the protocol side like you >> mention, I would imagine that the i2p daemon does it or else... >> well there is a problem :). If there are problems with our SOCKS implementation, they can be fixed (or implemented - I2P supports datagrams, but SOCKS UDP is not yet implemented). For filtering, it is up to the user to decide if they want to use a filtered SOCKSIRC tunnel or an unfiltered SOCKS tunnel, but it should not affect torsocks usage. (I want to make it possible for different filters to be written and used, but the user still needs to be made aware of the potential dangers to anonymity of generically SOCKSifying applications [1].) str4d >> >> Putting someone from i2p in CC: >> >> Cheers! David >> >>> >>> [1] http://www.i2p2.de/socks.html >>> >>> -- Maxim Kammerer Liberté Linux: http://dee.su/liberte >>> _______________________________________________ tor-dev mailing >>> list tor-dev(a)lists.torproject.org >>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > > >> _______________________________________________ tor-dev mailing >> list tor-dev(a)lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) iQEcBAEBCgAGBQJSdXoiAAoJENXeOJaUpGWyBBMH/0c39pn97sBZvBj7HaMJAde2 ucn6Ug9vd41AmxNcOEhG65wpOrfB32sfnwI7DsRAYlOhjdxdquUl1yQHzTzs7NeG VLwRNe7e7Zv7VRj9uRp6t5+0YQU6++RkzhX5Jtn7l5W+3XYq0c4lRpSpwawP4NHN Zc9MveAU1tbsQqvhsFoYoJEsADpBj+97Xf0D05buUYMttjsmKv254HhGaLpuDCpv gEY3nBDHlHZqRziwxV7WmWPIsetdkATnmkUqcoCIPbZzJh2j6W3rZmHUieGU9OPC M3offV3ImZd/gSVcKW1rzDEEZh2mDcmg0XkwMbsEwc/LCn15y5jPk2yFlGpKKsE= =u2QY -----END PGP SIGNATURE-----

2 1