June 2014 - tor-dev - lists.torproject.org

lcov test coverage reports
by David Murray 20 Jun '14

20 Jun '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, Would there be interest in using lcov [0] for test coverage analysis? Obviously, as lcov is an external pacakge, the existing scripts that only use built in gcov commands must remain. However, if I were to create a script that generates a pretty, HTML coverage report using lcov & genhtml, would this be something others would use? And how would you like to see this implemented? As a custom make target (e.g., make htmlcoverage + autotools black magic to silently disable/hide the target if we can't find lcov), or as a standalone script (i.e., ./scripts/test/run_lcov.sh). For what it's worth, I've manually run lcov on tor after enabling coverage through ./configure, and the output seems sane, so I think from a technical standpoint this would work. [0]: http://ltp.sourceforge.net/coverage/lcov.php Cheers, daube -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTnYryAAoJEBfCjrGktO5q+FgP/26ZgVZOAKSOBvWr6z2HYEku 1Yo+xFqTXk/Fw6yilv6Z4Jwc0Ar2B/VjHfLpOL1Vm+hFZ+VYsObSt9JGG2baItbG IrcedfjCuyjfaP1PloZoDM24FEbXqjseQO2EZ8bXbtErsW52Z4uAR3nYYcyXCCYp 1gi6lEwN/Yn0xm+fGgoz6/Z5FXvETdzbWLfhZoT4jFsnQsp/fByld55JLyft/8gA OrH3QQYGgXZkwIsnTkbLO3xWpXbqC5kU/ngCOL9WXXU99cgnW+bec7DA+cHhTqh/ 7oZD54noQ9XYtOA7AhAEOiTWnJUca2vV1KFPVBKSlqfkOhF3BRANHo2cIpsvgeBK 9+tcTyuwcFGAJVmW9l/AN3Y89CepCxnk6B1SSLFxuUVupAz2ilBM7KVtgj2ilJnC 2GjfuQB0RG/FW0CSEaa+DDtHfzCMfZNwGb6Fs1L41P8NnlGVFOz11NLlMG3t5ZpT 0OrncYaeVz74H+63Ig1ZwOjlfgUr9BHMLeYdwn9FmqIggyFRt1Z5PnNCaMuExs/v WQbLMP0oZ9XK1f51vfFVFYk/n1nh1c4J40/3I5B525GANAZ3fxTDLp0oSp/gNr5X lXPaF32KexhOiup5n1d6x8oJPzeNmex5CfcT29bZMPavF8p8v0OdZrUtU40bG7nO PUlA3EFfva1YW0aTRKY5 =+v+h -----END PGP SIGNATURE-----

2 3

GSoc Status Report Muticore Daemon
by Белоус Михаил 20 Jun '14

20 Jun '14

Hello everyone, This is second Google summer of Code status report, I implemented threadpool for tasks and create simple benchmark for threadpool. It pass test for workqueue, but I will write new test for new threadpool architecture. After my return I will learn more about work of daemon and rewrite bottle necks using my thread and write new benchmarks for this places. Sincerely your, Mikhail

1 0

Pluggable transports meeting today (16:00UTC Friday 20th of June 2014)
by George Kadianakis 20 Jun '14

20 Jun '14

Hello friends, just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur today at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. Thanks!

1 0

getting clock skew on demand rather than by event?
by Patrick Schleizer 19 Jun '14

19 Jun '14

Hi, whonixcheck is a diagnostic utility that can be run on demand. Not a Tor controller, that is supposed to be run continuously. It checks things such as if Tor's pid is running, bootstrap status, DisableNetwork 1, tor --verify-config and more. There is one thing in the logs, that would be worth notify a user about it: [warn] Our clock is 4 hours, 47 minutes behind the time published in the consensus network status document [or clock too fast] Now, parsing the log sounds like an unclean solution. Tor control protocol can tell about CLOCK_SKEW, but it's "only" an event. And "event" sounds to me, only a controller that is continuously running would be capable to learn about it. Is it possible to use stem for something like:, "Tor, tell me right now about your latest info on current clock skew"? Similar to tor_bootstrap_check.py? Cheers, Patrick [1] https://github.com/Whonix/anon-shared-helper-scripts/blob/master/usr/lib/an…

2 1

[GSoC] Consensus diffs - Second report
by Daniel Martí 19 Jun '14

19 Jun '14

Hello everyone, This is the second status report of my Google Summer of Code project, which is to implement consensus diffs for Tor. We hold weekly meetings every Wednesday at 14h UTC with the project mentors Nick and Sebastian. When I sent my first report I was a bit behind schedule as per my own timeline, mainly due to finals. These past two weeks I've been catching up on that timeline. I am now finishing the first part of it, which is to implement the diff generation code. If you read the last report, you might remember that the biggest issue at the time was performance. The code I had worked, but it was very pricy due to the quadratic time it took to compute a consensus diff, translating in running times from 10 to 20 seconds. In that e-mail I explained an idea I had in mind to overcome that. It got implemented and tested, and it works as expected. Now a diff generation between two consensus files takes roughly 0.04 seconds, which is almost the same amount of time that 'diff -e' takes. No precision seems to have been lost, but we'll have to do some edge case tests to verify that. The code was also missing documentation and some tor_asserts to make it more robust and understandable. That is what I'm finishing up at the moment, and then the tests will be the only task left. On to the second timeline item, which is the diff application code - The code sample I wrote for my application was a very simple version of it, supporting only 'delete' ed commands and just writing to stdout line-by-line. Taking that code as a starting point, plus being able to re-use some chunks from the diff generation code, I was able to have a working version in a few hours. So, like the diff generation bit, now that it seems to work properly it's time to document it properly and do some tests. I have until July 5th to finish these two tasks as per my own timeline, so I should be well able to have them done by then. The upcoming timeline item will be the joining of these chunks of code with Tor, at which point I expect some issues to arise. So I would like to get started with that bit the sooner the better, to have more time to get everything working and tested before we run out of time. As usual, any comments or ideas are very welcome. -- Daniel Martí - mvdan(a)mvdan.cc - http://mvdan.cc/ PGP: A9DA 13CD F7A1 4ACD D3DE E530 F4CA FFDB 4348 041C

1 0

Tor Geolocating exit nodes.
by JP Wulf 19 Jun '14

19 Jun '14

So Griffin Boyce is canvasing for some input to improve Tor, specifially for Journalists. https://twitter.com/abditum/status/479052228138119168 1. It is known that various actors are trying to compromise Tor comms by establishing own exit nodes. With enough nodes, they can break Tor (see slides). 2. Idea: Is it possible to allow the end user to determine the geo-location (with various degrees of fine tuning from hemisphere, through continental, to top country domain to regional? (I have NFI about the inner workings of TOR protocol and new work on it) For example. Say a journalist in Russia is using Tor, s/he declares in their tor client, that they only want to use exit nodes in South America and Australia. Thus minimising the chance the nodes are owned. This geolocation could perhaps be used to validate the integrity of the nodes (how I dont know, maybe by establishing TOR honeypots that can only be compromised through traffic through a compromised (owned) exit node). Risk: This is a rats nest, because if implemented incorrectly it may allow hostile actors to direct exit nodes to those that are owned. Thanks for reading my fiction. Maybe its useful in the light of what Griffin is asking about. -- JP Wulf Problem Solution Engineering http://nomeonastiq.com/

4 3

Composing multiple pluggable transports
by Ximin Luo 18 Jun '14

18 Jun '14

Hi Steven, Nikita, I was told that you two are interested in the idea of composing multiple PTs together. Here are our ideas on it. We have a GSoC student, Quinn also at Illinois, working on turning this into reality. ## Concepts On the most abstract level, pt-spec.txt defines an "input interface" to some generic component. It consists of the following: - dest addr, of the Bridge - headers/metadata, such as fingerprint[1] or other PT-level settings - data, the actual application-layer stream, such as OR protocol The concrete form of this is the SOCKS protocol, which allows tor to make a request with the above interface. (Actually, SOCKS does not fully support metadata, which means we've had to extend it ourselves. HTTP might have worked better.) pt-spec.txt does not specify the "output interface". This means it's impossible to chain general PTs, because there's nothing defined to chain. To work around this, we observe that in practise, many PTs follow the below "output interface"; we'll call these "direct PTs": - data, sent directly to the (TCP) address given at the input Direct PTs include obfsproxy, scramblesuit, fteproxy. Indirect PTs are all other PTs, that do something different other than a straight TCP connection *to the endpoint Bridge address*. These include flashproxy and meek. ## Design Our combiner will chain up a sequence of direct PTs, then the last PT can be any PT (either direct or indirect). So for example it could potentially support obfs3|fte|fte|fte|flashproxy and obfs3|fte but not flashproxy|obfs3|obfs3|meek. Not every chain makes sense from a security viewpoint, of course. Because the output interface (TCP) does not exactly match the input interface (SOCKS), we have a component called a "shim", which has an input interface of TCP and an output interface of SOCKS. This is placed between each pair of PTs in the chain. In simple terms, it works like this: pt0 (out) -TCP-> (in) shim1 (out) -SOCKS-to-next-shim-> pt1 -> to shim2 The extra info present in SOCKS (dest addr and metadata) absent from TCP, will be supplied by the combiner, as described in the next section. Also, in practise, these shims are within the same process as the combiner, there is no need to start a new process for these. ## Algorithm Let the ith PT be listening on port pPT[i], done at program start. Later, when tor wants to connect to a PT-chain, we intercept this connection, extracting the following information (the SOCKS in-interface): - dest addr, of the Bridge - headers. generic headers, plus PT-specific headers for each PT in the chain[2] - data, OR protocol Then, the combiner starts a new shim, one for each component in the chain, each listening on pS[i]. Each shim[i] is set-up so that when it receives a connection on pS[i], it tells PT[i] (i.e. the SOCKS client listening on pPT[i]) to connect to pS[i+1], with the metadata set to the generic headers plus specific headers for PT[i], and the data set to whatever it receives from the connection. (In practise the shims are set-up in reverse order, because shim[i] needs to know what pS[i+1] is, and we want to take advantage of OS's feature to "listen on any port".) Special cases: - The last (i.e. (n-1)th) shim tells its SOCKS client to connect to the original dest addr, of the Bridge - there is no pS[last+1]. - The first shim does not need to exist, since the combiner is just sending data to itself so can do this in-process After all the shims are set up, the combiner starts forwarding data from tor over onto PT[0]. Then the magic is complete. ASCII diagram, minus annotations about metadata: +----------+ socks | PT | [ tor ] to >-------> | combiner | bridge +-< | | | +----------+ in-process | v +-------+ socks | direct| tcp to >-> | PT[0] | to >-+ pS[1] | | pS[1] | +-------+ | +-------------------<-------------------+ | +-------+ | socks | direct| tcp +->[ shim[1] ] to >-> | PT[1] | to >-+ pS[2] | | pS[2] | +-------+ | +-------------------<-------------------+ | | [etc] | | +-------------------<-------------------+ | +-------+ | socks | direct| tcp +->[ shim[y] ] to >-> | PT[y] | to >-+ pS[z] | | pS[z] | +-------+ | +-------------------<-------------------+ | +-------+ | socks | any | +->[ shim[z] ] to >-> | PT[z] | whatever it wants --> bridge | | +-------+ Note that, if for whatever reason your chain has the same PT in multiple positions, the chain will re-use the same PT process. Everything should still work, because we have separate *shims* for each *position* that the PT appears within the chain. X [1] fingerprint not actually currently given to PTs by tor; but it should be for reasons argued elsewhere [2] we haven't exactly defined a format for this, but perhaps something like k=v for generic headers to all children (as currently), and x-chain-0-k=v for PT-specific headers, and maybe even something like x-chain-0onwards-k=v. -- GPG: 4096R/1318EFAC5FBBDBCE git://github.com/infinity0/pubkeys.git

2 1

Reminder: tor dev meeting this (and every) Wednesday, 1900 UTC
by Nick Mathewson 17 Jun '14

17 Jun '14

Hi, all! Here's your regular reminder for the weekly IRC meeting for people working on the program "tor". (This won't cover all the other programs developed under the Tor umbrella.) The next meeting time will be: Wednesday June 18, 19:00 UTC. (That's 3pm EDT and 12:00 noon PDT.) As usual, we'll do it on the #tor-dev IRC channel. I've seen a drop-off in the number of people coming since I stopped sending these out weekly, so I'm sending them again for a while. Among other stuff, I expect we'll be looking at tickets for 0.2.6.x. cheers, -- Nick Mathewson

1 0

ANN: OnionPy 0.1.5
by Lukas Erlacher 17 Jun '14

17 Jun '14

Hello all, my OnionOO python wrapper has now matured significantly. While I was gone for around a month due to being otherwise occupied, I have now given OnionPy a lot of polish. The new features: - Python2 compatibility (All it took for that was to change the readme) - Encapsulated caching layer that includes a memcached interface as well as a simple python dict caching mechanism, or you can add glue code for your favourite caching backend yourself - It's on Pypi! [1] - Extended test / demonstration script Many thanks to meejah for his help. If you are planning to make something in python that uses the tor network status, accessing OnionOO [2] using OnionPy might be exactly what you need. If you are interested, check out the repo and readme! [3] Of course, criticism, suggestions, and pull requests are always welcome. Best regards, Luke [1] https://pypi.python.org/pypi/OnionPy [2] https://onionoo.torproject.org/ [3] https://github.com/duk3luk3/onion-py

2 3

Running Tor with static path
by mahdi 16 Jun '14

16 Jun '14

Hi all, I want to execute an experiment on Tor in which i need to fix the ip address of entry,relay and exit onion router. For that i need to determine the IPs and keys of ORs in OP permanently. Is there any idea of what function of Tor's code should be replace?! Thanks a lot

4 3