September 2014 - tor-dev - lists.torproject.org

Damian's Status Report - September 2014
by Damian Johnson 30 Sep '14

30 Sep '14

Code! Beautiful code! How I've missed thee! As mentioned before I'm hip deep in the middle of an arm rewrite. Not only to overhaul the arm codebase, but to ensure Stem will be up to snuff for a Tor GUI or web dashboard (projects I'd *love* to jump into someday). I'm delighted to announce a nice bit of progress on this front. September went toward rewriting its header panel (the top portion of arm's interface)... * Old zomg my eyes, they bleed! https://gitweb.torproject.org/arm.git/blob/e249dc8f5c4c282326324161cf8421f9… * New hotness https://gitweb.torproject.org/arm.git/blob/HEAD:/arm/header_panel.py As mentioned though this is really more about Stem than arm. Stem improvements made along the way this month include... * Far better handling for the 'private' keyword (the default prefix) and the default suffix in our ExitPolicy class. (#10107) * New BaseController.connection_time() method that provides when our control connection was established or detached. [1] * New Controller.get_accounting_stats() method which reports accounting information for our relay. [2] * Controller now fetches our own descriptor if no fingerprint is provided to get_server_descriptor(), get_network_status(), or get_microdescriptor(). * New crop() and join() methods for the str_tools module. [3][4] * Dropped a redundant get_* prefix from most of our util function names. Old names still work as aliases. * Our launch_tor_with_config() function raised an OSError if called too many times, caught thanks to RSenet. (#13141) DocTor, the monitor for directory authority health, also got some love this month... * Corrected suppression behavior when we have staggered alerts. This should greatly cut down on the amount of noise on the list. * We now check if too many relays lack a bandwidth authority measurement... or we would if this wouldn't perpetually be in alarm. (#12989, #13046) * Revised DocTor's schedule as requested by Sebastian. (#13210) * Expanded suppression for messages about turtles and kicked off a conversation to at long last sort this out. (#13296) One final note is that I uploaded my Tor Ecosystem presentation to YouTube so volunteer page visitors can see a streaming presentation rather than downloading a 53 MB video... https://www.youtube.com/watch?v=fb6iqZcQsSg Cheers! -Damian [1] https://stem.torproject.org/api/control.html#stem.control.BaseController.co… [2] https://stem.torproject.org/api/control.html#stem.control.Controller.get_ac… [3] https://stem.torproject.org/api/util/str_tools.html#stem.util.str_tools.crop [4] https://stem.torproject.org/api/util/str_tools.html#stem.util.str_tools.join

1 0

Getting the network consensus
by Michael Rogers 30 Sep '14

30 Sep '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi all, While working on peer-to-peer hidden services I've been wondering how long two clients need to wait before arriving at the same view of the Tor network. The answer, which I find surprising, is that this is never guaranteed to happen. Here's why. In what follows I'll assume that the interval between network consensuses being published is one hour, but the conclusions remain the same if the interval is longer or shorter - just change the units. I'll use "1:00 consensus" to mean a consensus that becomes valid at 1:00, is fresh until 2:00, and is valid until 4:00. According to dir-spec.txt, each directory cache downloads the 1:00 consensus at a randomly chosen time between 1:00 and 1:30. Each client replaces the 1:00 consensus with a newer consensus at a randomly chosen time between 2:45 and 3:51. (The precise endpoint is 3:50:37.5, due to a calculation described in section 5.1 of the spec. I'm curious to know the origin of that calculation.) Some observations: 1. The period when clients are replacing each consensus slightly overlaps the period when they're replacing the previous consensus. For about five and a half minutes of every hour there are twice as many clients replacing their consensuses, so directory caches will experience greater load during this period. How much greater? We can divide clients that are downloading the consensus into three groups: A. Clients that have never downloaded a consensus. B. Clients that are replacing a consensus that's no longer valid. C. Clients that are replacing a valid consensus with a newer consensus. Clients in group A bypass the directory caches and hit the authorities, so they don't contribute to load on the caches. Clients in group B have just started up and loaded an expired consensus from disk. These clients are equally likely to hit the caches at any point in the consensus interval (although the load will vary by time of day, day of week, etc). Clients in group C are twice as likely to hit the caches between 45 minutes and 51 minutes past the hour as at any other time, due to the overlap between replacement intervals. So unless group C is dwarfed by group B, we should see a noticeable increase in load on the caches between 45 and 51 minutes past the hour. (If group C is dwarfed by group B then the increase will be lost in the noise.) If you run a directory cache, do your logs show that pattern? 2. The time a client chooses for replacing a newly downloaded consensus may be earlier than the time it chose for replacing the previous consensus. Again, this is due to the overlap between replacement intervals: a client may replace the 1:00 consensus at 3:48 and then choose 3:47 as the random time to replace the newly downloaded 2:00 consensus. What happens in that case? I haven't looked at the code, but I'm going to speculate that it doesn't explode in flames or travel through time. A reasonable way to handle the situation would be to round up the replacement time to the present time - in other words, to replace the newly downloaded consensus immediately. If that guess is correct, the extra load during the overlap between replacement intervals will get squeezed towards the end of the overlap, as replacement times occasionally get rounded up but never get rounded down. Some clients will download two consensuses back-to-back during the overlap. Again, if you run a directory cache I'd be interested to know whether you're seeing this pattern. 3. Clients often skip consensuses. The interval for clients to replace the 1:00 consensus runs from 2:45 to 3:51, and the interval for caches to download the 3:00 consensus runs from 3:00 to 3:30. So there's roughly a 46% chance that a client replacing the 1:00 consensus will get the 2:00 consensus, and roughly a 54% chance that it will get the 3:00 consensus. This means it's possible for two clients to replace their consensuses regularly but never see the same consensus: one of them sees the consensuses published on odd hours, the other sees the consensuses published on even hours. This situation is unlikely to continue for long if each client makes a fresh random choice for each replacement time, but there's no time at which we can say the clients will definitely have seen the same consensus. I can't see any partitioning attacks arising from this, but I thought I'd point it out anyway because I find the result surprising. As I said before, I'm curious to know the origin of the calculation in section 5.1 that produces the overlap between replacement intervals. The replacement time is chosen "uniformly at random from the interval between the time 3/4 into the first interval after the consensus is no longer fresh, and 7/8 of the time remaining after that before the consensus is invalid". It's the 7/8 part that results in the odd figure of 50 minutes 37.5 seconds. If the replacement time were to be chosen uniformly at random from the interval between the time 3/4 into the first interval after the consensus is no longer fresh, and the time one consensus interval after that, there would be no gap or overlap between replacement intervals. This wouldn't affect the third observation, however - there would still be no guarantee of two clients ever seeing the same consensus. Cheers, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJUKr8YAAoJEBEET9GfxSfM8NQIALyS7LuGYrxPfktaSD4pdnNL TTW1t/C4wyBYawn6xmu9Tf32Lbyjsb2Jt+6l+kY6R+8m0meR+zglxCozcivPtH2J RTnaPLOOA6SW1ibNiDSLfw0Zbm4GMF4Z8GrG+9PGCMUIaLccpmAtigDVDBSiFL2D JDSk3ep3ln/PIFH6OljKV8gSMqNPlhHqHc9+gxNlPUXvjJdcsJZPpTCnMlMKPDiB X5uk4k3E27r3pGM26kSSi3N5BYL6QAN5dXkH5hlgB/+XseSXz/KVtnfgOXuoKou2 bK5lszTLbc9xHNBW+d4EYOkI6LwjQT4xcSYlHYW/a28s+U5rvFHuM/UqGF291SE= =gPDQ -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] [tor-relays] [warn] No unused circIDs found on channel without wide circID support
by ra 29 Sep '14

29 Sep '14

reply-to: tor-dev On Monday 29 September 2014 15:00:29 tor-admin wrote: > since upgrading torland to 0.2.5.8-rc I see the below warnings. Is this > something to worry about? IMHO this might be due to a programming error in the circuit ID selection code or possibly indicate a DoS attack. > -------------------------------------------------------------------------- > Sep 28 16:07:24.000 [warn] No unused circIDs found on channel without wide > circID support, with 0 inbound and 4 outbound circuits. Found 0 circuit IDs > in use by circuits, and 64 with pending destroy cells. (0 of those were > marked bogusly.) The ones with pending destroy cells have been marked > unusable for an average of 7594 seconds and a maximum of 16137 seconds. > This channel is 18469 seconds old. Failing a circuit. > Sep 28 16:07:24.000 [warn] Circuitmux on this channel has 4 circuits, of > which 4 are active. It says it has 29535 destroy cells queued. Those lines are important. I would suggest opening a ticket on trac. See #11553. Best, Robert

1 0

Re: [tor-dev] Scaling tor for a global population
by teor 28 Sep '14

28 Sep '14

On 27 Sep 2014, at 10:18 , tor-dev-request(a)lists.torproject.org wrote: > Date: Fri, 26 Sep 2014 17:18:07 -0700 > From: Mike Perry <mikeperry(a)torproject.org> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Scaling tor for a global population > > 2. Cut off relays below the median capacity, and turn them into bridges. > > Relays in the top 10% of the network are 164 times faster than > relays in the 50-60% range, 1400 times faster than relays in the > 70-80% range, and 35000 times faster than relays in the 90-100% range. > > In fact, many relays are so slow that they provide less bytes to the > network than it costs to tell all of our users about them. There > should be a sweet spot where we can set this cutoff such that the > overhead from directory activity balances the loss of capacity from > these relays, as a function of userbase size. > > Result: ~2X reduction in consensus and directory size. Do these extra relays function as spare (albeit slower) capacity in the network? If so, we'd have to be careful to set the cut-off low enough that any spikes in usage could be accommodated. Also, doesn't reducing the number of routers in the consensus risk harming network diversity? I understand that if these routers are a net burden on the network, then they're not actually contributing much to diversity overall. But what about users with obscure configurations like "New Zealand routers only", or "custom list of 20 trusted routers"? (Is a restricted set of routers a common enough use case?) Do we risk pulling the data they need from the consensus? Then again, users with these sorts of configurations are probably technically savvy enough to fix any issues that occur with any changeover. T teor pgp 0xABFED1AC hkp://pgp.mit.edu/ http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w…

1 0

Re: [tor-dev] Scaling tor for a global population
by Ryan Carboni 28 Sep '14

28 Sep '14

> > But, because this is fraction rises with both D and U, these research > papers rightly point out that you can't keep adding relays *and* users > and expect Tor to scale. Broadcast a fraction of all available directories? Use md5 as a random number generator, hash the ECC/RSA keys using md5. A user connecting to the network will generate an 8-bit random value, and based on that, will download one of 1/256 directories. Right now, any relay with more than ~100Mbit of capacity really > needs to run an additional tor relay instance on that link to make > use of it. If they have AES-NI, this might go up to 300Mbit. Any plans to use ChaCha8 instead of AES? It would an order of magnitude faster. It is also unlikely for ChaCha8 to become sufficiently insecure to effect web-size non-video traffic.

1 0

obfs4 test bundles
by Yawning Angel 26 Sep '14

26 Sep '14

Hello all, So after a brief pause in my productivity due to the dev meeting and some IRL things, I'm back in the swing of development and have been working on getting obfs4 ready for development. I've recently mostly finished a fairly large cleanup/refactor of the obfs4 codebase and dusted off the Tor Browser build integration patch. Test bundles signed with my PGP key are available at: https://people.torproject.org/~yawning/volatile/tor-browser-obfs4-20140820/ Some quick notes: * The bundles are alpha. DO NOT USE THEM IF YOU REQUIRE STRONG ANONYMITY OR SECURITY. * While it is possible to pull out the obfs4proxy binary from the bundle and use it to run a bridge, that will be a *terrible* idea because the bundles are missing some logging related fixes. * The Windows and OSX bundles are entirely untested, because I do not have access to either platform. The Linux bundle appears to work. * Using obfs2/obfs3 will also exercise new code (obfs4proxy instead of obfsproxy). * I'm not uploading a 32 bit linux bundle for now. * The one obfs4 test bridge is mine, and is the one I posted to tor-dev@ (port is different, bridge parameters are the same). If you wish to follow obfs4 deployment the bug associated with this task is #12130. Questions, comments, feedback welcome. -- Yawning Angel

1 2

Pluggable transports meeting today (16:00UTC Wednesday 24th of September 2014)
by George Kadianakis 24 Sep '14

24 Sep '14

Hello! just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur today at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. As you see, the meeting day was recently changed from Fridays to Wednesdays: https://lists.torproject.org/pipermail/tor-dev/2014-August/007317.html Thanks for your attention!

2 1

Making and distributing custom TBB with a new "home-page"
by Fabio Pietrosanti (naif) 23 Sep '14

23 Sep '14

Hi all, for a very interesting deployment of GlobaLeaks in the are of Human Rights defense, we will have the need to distribute a customized Tor Browser Bundle to the sources. The "customization" requirement is simple: Have as a default home-page the GlobaLeaks .onion site . We must go that way because: - the "target country" where the sources are cannot download TBB due to torproject.org being censored - the sources are absolutely non-technologically savy (average 60yo lawyers doing human rights defense) The website where there will be the leaking instructions and the download of such custom TBB will be "privately distributed" trough word of mouth and trusted connections, with no public solicitation. So we must do some piece of software that will: - Download TBB in specific languages (2-3 specific languages) for each platform - Unpack TBB (in all formats for Windows, OSX, Linux) - Apply the customization (set the home-page, with slightly different parameters depending on the language) - Check periodically if a new version is available and, in that case, re-execute the process described above to release updated version of TBB The questions are: a) Which is a simple/stable/resilient way to check which is the latest version of TBB b) Does someone have already done that kind of customization-process? c) Can everything be done from Linux, like a cron-job, in a fully automatic way? d) Which other customization / ideas / concern are there regarding this process? I'd personally love if the customization would enable me to completely disable the "URL Bar" and all of the Browser Button in order to make it useful only to use it as a console to send information being a source, without the possibility to go browse other sites. Waiting for comments before writing some quick specs -- Fabio Pietrosanti (naif) HERMES - Center for Transparency and Digital Human Rights http://logioshermes.org - http://globaleaks.org - http://tor2web.org

3 4

meek on Microsoft Azure
by David Fifield 22 Sep '14

22 Sep '14

I've got meek working on Microsoft Azure, a cloud services platform and CDN. If you'd like to try it, use a 4.0-alpha bundle and enter the bridge line: Bridge meek 0.0.2.0:3 url=https://az668014.vo.msecnd.net/ front=ajax.aspnetcdn.com As usual, the IP address is ignored; I'm just assigning incrementing numbers in the port. az668014.vo.msecnd.net is a CDN domain that points to meek-reflect.azurewebsites.net, where the service is really running. The front domain ajax.aspnetcdn.com is one that hosts a bunch of common web libraries: http://www.asp.net/ajax/cdn. I didn't look very hard to see if there are other good front domains. More information on Azure: https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure David Fifield

1 0

Attack linking Tor circuits
by Otto Huhta 19 Sep '14

19 Sep '14

Dear Tor developers, We’ve been working on an MSc Thesis project looking into an attack that links different Tor circuits back to the same user, using only information available to a Tor middle node. We’ve discovered that an adversary can quite accurately determine if two consecutive circuits are used by the same user just by looking at when these circuits are built and destroyed or when they begin and stop relaying traffic. This is mainly the result of the fixed 10 minute circuit lifetime and the fact that the transition to using a new circuit is quite sharp. According to our measurements, circuits belonging to the same user are built at very close to 10 minute intervals. Also, when one circuit stops relaying traffic another one suddenly becomes active, at almost the very same moment. Thanks to these properties we could train a classifier to quite efficiently identify matching pairs of circuits (9% EER). We looked into the threat posed by this attack to Tor users and concluded that because of entry guards the threat is real, given that an adversary can control a significant portion of middle node traffic. Finding matches among all possible Tor circuits would result in too many false positives but thanks to entry guards an adversary can focus only on circuits built through these specific nodes and quite efficiently determine if two circuits belong to the same user. Possible plans of moving to using only a single guard node make the situation much more severe: we concluded that for a user picking a slow guard node, the adversary can find almost all circuits belonging to a same user (again assuming control of a large portion of middle node bandwidth). We thus see that before moving on with any such plans, one should consider the effect of our findings on the threat posed by a middle node linking user circuits. As a final note, we considered the effects of randomizing the circuit lifetime (e.g. between 5-15 minutes) to mitigate this threat but concluded that it would not necessarily solve the problem. The most prominent feature linking two circuits together seems to be the time difference between traffic ending on one circuit and beginning on another circuit, and this link cannot be broken by varying the circuit lifetime. However, if someone were to devise such a modification, we would be happy to re-run our tests to see what kind of an effect this would have on the success rate of an adversary. For those interested in our findings, the full thesis Linking Tor Circuits can be found at: http://www0.cs.ucl.ac.uk/staff/G.Danezis/students/Huhta14-UCL-Msc.pdf Best regards, Otto Huhta, UCL

2 3