- tor-dev - lists.torproject.org

Log output cause for concern?
by Steve Snyder 03 Oct '14

03 Oct '14

Over 3 days I've gotten 9 instances of the sequence shown below. This on Tor v0.2.5.8-rc / x86_64. I am at 80% CPU utilization with BandwidthRate set to 90Mbps on this 100Mbps interface. Should I be concerned about this log output, or is it just verbose because v0.2.5.x is still in development? --------------------------------------- Sep 30 15:30:08.000 [warn] No unused circIDs found on channel without wide circID support, with 0 inbound and 0 outbound circuits. Found 0 circuit IDs in use by circuits, and 64 with pending destroy cells. (0 of those were marked bogusly.) The ones with pending destroy cells have been marked unusable for an average of 16204 seconds and a maximum of 32369 seconds. This channel is 61794 seconds old. Failing a circuit. [3827 similar message(s) suppressed in last 3600 seconds] Sep 30 15:30:08.000 [warn] Circuitmux on this channel has 0 circuits, of which 0 are active. It says it has 32669 destroy cells queued. Sep 30 15:30:08.000 [warn] Channel 2454544 (at 0x7f3ca1b12720) with transport TLS channel (connection 5592182) is in state open (2) Sep 30 15:30:08.000 [warn] * Channel 2454544 was created at 1412029214 (61794 seconds ago) and last active at 1412090927 (81 seconds ago) Sep 30 15:30:08.000 [warn] * Channel 2454544 says it is connected to an OR with digest 7CF62B2A0EA73C3621B79118A40F0A9673FA3146 and no known nickname Sep 30 15:30:08.000 [warn] * Channel 2454544 says its remote address is [scrubbed], and gives a canonical description of "[scrubbed]" and an actual description of "[scrubbed]" Sep 30 15:30:08.000 [warn] * Channel 2454544 has these marks: !bad_for_new_circs canonical is_canonical_is_reliable !client !local outgoing Sep 30 15:30:08.000 [warn] * Channel 2454544 has 0 queued incoming cells and 0 queued outgoing cells Sep 30 15:30:08.000 [warn] * Channel 2454544 has 0 active circuits out of 0 in total Sep 30 15:30:08.000 [warn] * Channel 2454544 was last used by a client at 0 (1412091008 seconds ago) Sep 30 15:30:08.000 [warn] * Channel 2454544 was last drained at 1412090927 (81 seconds ago) Sep 30 15:30:08.000 [warn] * Channel 2454544 last received a cell at 1412090006 (1002 seconds ago) Sep 30 15:30:08.000 [warn] * Channel 2454544 last transmitted a cell at 1412090927 (81 seconds ago) Sep 30 15:30:08.000 [warn] * Channel 2454544 has received 381 cells and transmitted 11081 Sep 30 15:30:08.000 [warn] * Channel 2454544 has averaged 162.188976 seconds between received cells Sep 30 15:30:08.000 [warn] * Channel 2454544 has averaged 5.576573 seconds between transmitted cells Sep 30 19:43:59.000 [notice] Heartbeat: Tor's uptime is 5 days 18:00 hours, with 10312 circuits open. I've sent 3545.65 GB and received 3417.71 GB. Sep 30 19:43:59.000 [notice] Average packaged cell fullness: 99.112% Sep 30 19:43:59.000 [notice] TLS write overhead: 3% Sep 30 19:43:59.000 [notice] Circuit handshake stats since last time: 352835/352843 TAP, 113291/113292 NTor.

1 0

Scaling tor for a global population
by Andrew Lewman 03 Oct '14

03 Oct '14

I had a conversation with a vendor yesterday. They are interested in including Tor as their "private browsing mode" and basically shipping a re-branded tor browser which lets people toggle the connectivity to the Tor network on and off. They very much like Tor Browser and would like to ship it to their customer base. Their product is 10-20% of the global market, this is of roughly 2.8 billion global Internet users. As Tor Browser is open source, they are already working on it. However ,their concern is scaling up to handling some percent of global users with "tor mode" enabled. They're willing to entertain offering their resources to help us solve the scalability challenges of handling hundreds of millions of users and relays on Tor. As this question keeps popping up by the business world looking at privacy as the next "must have" feature in their products, I'm trying to compile a list of tasks to solve to help us scale. The old 2008 three-year roadmap looks at performance, https://www.torproject.org/press/2008-12-19-roadmap-press-release.html.en I've been through the specs, https://gitweb.torproject.org/torspec.git/tree/HEAD:/proposals to see if there are proposals for scaling the network or directory authorities. I didn't see anything directly related. The last research paper I see directly addressing scalability is Torsk (http://www.freehaven.net/anonbib/bibtex.html#ccs09-torsk) or PIR-Tor (http://www.freehaven.net/anonbib/bibtex.html#usenix11-pirtor) Is there a better list available for someone new to Tor to read up on the scalability challenges? -- Andrew pgp 0x6B4D6475 https://www.torproject.org/ +1-781-948-1982

17 35

RFC: Trunnel -- a binary parser generator for Tor and more
by Nick Mathewson 02 Oct '14

02 Oct '14

Hi! Back when the Heartbleed vulnerability went public, I thought to myself: "If we keep generating writing C code to parse binary formats by hand, the same thing will happen to us sooner or later." So on and off, I've been working on a little tool to automatically generate binary encoding and parsing code based on C-like structure descriptions. It's a bit like nail[0], but more lightweight: it doesn't support backtracking, sub-byte alignment, or any of several other features that Tor doesn't actually want. You can see a pre-alpha version of the tool (called "Trunnel") in one of my git repositories [1]. The "doc" subdirectory has a simple manual [2]. To make sure that it would suit our needs, I wrote up trunnel specification files for the current Tor protocols, for SOCKS, and for the new protocols in proposals 220, 224, and 229. It seems to work okay. In case you want to see what the tool does without running it yourself, you can find the SOCKS+proposal229 specification here [3], and compare it with the generated header [4] and C implementation [5]. Trunnel is written in Python; my tests have 99% line coverage on the Python code. The C code which is generated by those tests itself has 99% line coverage from other unit tests, so I'm thinking that it's reasonably tested. (Branch coverage is lower, but still pretty high.) So, please check this out and have a look at it! My open questions are: * Is this a good idea? * Is it a good idea to use this in Tor? * Are there any tricky bugs left in the generated code? * What am I forgetting to think of? FWIW, my current plan is not to do a blanket replacement of our existing parse code with Trunnel (yet) but rather to use Trunnel to generate parsing code for new cell formats that are getting added in 0.2.6 and later. [0] https://github.com/jbangert/nail [1] https://gitweb.torproject.org/user/nickm/trunnel.git . [2] Also online at http://www.wangafu.net/~nickm/trunnel-manual.html [3] https://gitweb.torproject.org/user/nickm/trunnel.git/blob/HEAD:/examples/so… [4] http://www.wangafu.net/~nickm/volatile/socks5.h [5] http://www.wangafu.net/~nickm/volatile/socks5.c cheers, -- Nick Mathewson

1 1

Schedule change for ooni dev meeting at 19:00 CET on Thursday October 2 2014
by Arturo Filastò 01 Oct '14

01 Oct '14

Hello Oonitarians, This is a reminder to inform you that the weekly dev meeting will happen today at 19:00 CET Thursday October 2. Note: That the schedule has changed from Friday to Thursday. Adjust your calendaring software accordingly. See you! ~ Art.

1 0

ooniprobe 1.2.0 and oonibackend 1.1.4 are out!
by Arturo Filastò 01 Oct '14

01 Oct '14

Greetings! We are very pleased to announce the release of new versions of ooniprobe and oonibackend. One of the most interesting new features that are now part of ooniprobe is the ability to generate test decks for the country you are in a way that is much easier than before. As a matter of fact to start contributing useful measurements it's just a mater of 5 minutes of setup: https://pypi.python.org/pypi/ooniprobe#ooni-in-5-minutes Here are some more details of what this release includes: ooniprobe ========= * Introduce a new tool for generating ooniprobe test decks called oonideckgen. * Introduce a new tool for updating resources used for geoip lookup and deck generation. * Add support for policy aware bouncing in the client. https://trac.torproject.org/projects/tor/ticket/12579 * Various improvements to the bridge_reachability test (enable better tor logging and also log obfsproxy) * Fix backward compatibility with twisted 13.1 and add regression tests for this. https://trac.torproject.org/projects/tor/ticket/13139 oonibackend =========== * Add support for specifying the report archive path from config file * Write tor notice level logs to file * Fix bug that lead test helpers to not being started The new packages have been uploaded to pypy and can be found at the following links: https://pypi.python.org/pypi/ooniprobe https://pypi.python.org/pypi/oonibackend Soon ooniprobe will also be available inside of debian testing thanks to Lunar^. Have fun, ~ Art.

1 0

Summary of meek's costs, August 2014
by David Fifield 01 Oct '14

01 Oct '14

The "meek" pluggable transport has started getting some users since my blog post two weeks ago (https://blog.torproject.org/blog/how-use-%E2%80%9Cmeek%E2%80%9D-pluggable-t…) See the attached file userstats-bridge-transport-meek-2014-08-01-2014-08-29.png. Today I logged in to the Google and Amazon accounts to see what the usage looks like. Here is a summary. In short, App Engine cost $1.02 this month, and Amazon cost $2.54. meek-appspot-2014-08-29.png shows the Google App Engine bytes/second for the month of August. App Engine costs $0.12 per GB, with 1 GB for free each day. It's finally at the point where it's costing money every day: All of January 2014: $ -- All of February 2014: $0.10 All of March 2014: $ -- All of April 2014: $0.83 All of May 2014: $0.75 All of June 2014: $0.73 All of July 2014: $0.59 August 2014 to date: $1.02 Total to date $4.02 Here are daily totals in the past two weeks: 2014-08-14 $ -- 2014-08-15 $ -- 2014-08-16 $ -- 2014-08-17 $ -- 2014-08-18 $0.09 2014-08-19 $0.00 2014-08-20 $0.04 2014-08-21 $0.07 2014-08-22 $0.09 2014-08-23 $0.07 2014-08-24 $0.03 2014-08-25 $0.09 2014-08-26 $0.19 2014-08-27 $0.07 2014-08-28 $0.28 meek-aws-2014-08-29.png shows the AWS megabytes per day (note that the y-axis is different from the other graph so they are not directly comparable). The charges break down quite differently: there's no cost for bandwidth so far, and all the charges are for the sheer number of HTTPS requests. The Amazon bill is broken down by region: Asia Pacific (Singapore) Region $0.0090 per 10,000 HTTP Requests 3 Requests $0.01 Asia Pacific (Sydney) Region $ -- EU (Ireland) Region $ -- South America (Sao Paulo) Region $ -- Asia Pacific (Tokyo) Region $0.0090 per 10,000 HTTP Requests 38 Requests $0.01 US East (Northern Virginia) Region $0.0100 per 10,000 Proxy HTTPS Requests (US) 2,513,557 Requests $2.51 $0.0075 per 10,000 HTTP Requests 3 Requests $0.01 Total to date $2.54 David Fifield

1 1

Re: [tor-dev] Scaling tor for a global population
by teor 01 Oct '14

01 Oct '14

[thread reconstructed] >> On 09/30/2014 06:28 AM, AFO-Admin wrote: >>> if we would get multithread support, this would boost the bandwith >>> that is avaible, there i'm sure. >>> All my relays run in CPU limit because i don't think wasting even more >>> ipv4 addresses is great, today you get more and more cpu cores that is >>> not linier with the IPC increase. >>> >>> E.g. you have a Server with 2x E5-2683 v3 v3 and a 10 Gbit/s pipe you >>> would need atleast 14 IP's to use most of the CPU. And every IP's gets >>> blacklistet, with that much Tor nodes in the same /24 maybe the entire >>> /24. So most ISP's wouldn't be happy with that and with the IPv4 >>> shortage this days im also not very happy with that. >>> We really shouldn't waste more IPv4 IP's then needed and the only >>> solution is to change the max amount of Tor Processes from 2 to a >>> higher number or move to IPv6 or get multithreading working. [Moritz quoted] >>> E.g. you have a Server with 2x E5-2683 v3 v3 and a 10 Gbit/s pipe you >>> would need atleast 14 IP's to use most of the CPU. > > Moritz Bartl transcribed 0.5K bytes: >> Raising the limit from 2 relays per IP to x per IP has been discussed in the >> past and would be an easy change. > Date: Tue, 30 Sep 2014 20:35:21 +0000 From: isis <isis(a)torproject.org> > We *still* have that limit? I thought we killed it a long time ago. > > Can we kill it now? It's not going to do anything to prevent Sybils, it'll > only prevent good relay operators on larger machines from giving more > bandwidth. Allowing multiple (>2) tor instances would alleviate AFO's issue in the short term, although in this particular case, they might need 14(!) instances (or AES-NI). Given the shortage of IPv4s, and the availability of multi-processor, high-bandwidth servers, we could trial raising the Tor instance limit per IP. (As this is an authority parameter, the change could happen much sooner than predominantly-IPv6 tor or multithreaded tor.) 4 would allow 1 tor process per logical processor in many server machines (e.g. 4x1 and 2x2). At ~320Mbps per tor process (the maximum bandwidth in the current network), this could saturate a 1 Gbps link with 1 IP. 8 would allow 1 tor process per logical processor in almost all servers (e.g. 4x2 and 8x1), and could saturate a 2.5 Gbps link with 1 IP. In AFO's 10 Gbps case, they'd need 32 processes, or 4 IPs. (Which doesn't seem as unreasonable as 14 IPs.) The only drawback I can see is that IPs with slow connections/few CPUs could then launch 4 or 8 instances, and slow down the network. This could exacerbate the "wasted consensus entry" issue, where the consensus bytes used for a router outweigh its contribution. (But this seems unlikely.) In the short term, can we trial raising the Tor instance limit per IP to 4 or 8? (In the longer term, I'm happy to help with (network) performance, multithreading, or IPv6 - probably in that order.) T teor pgp 0xABFED1AC hkp://pgp.mit.edu/ https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w…

1 0

Damian's Status Report - September 2014
by Damian Johnson 30 Sep '14

30 Sep '14

Code! Beautiful code! How I've missed thee! As mentioned before I'm hip deep in the middle of an arm rewrite. Not only to overhaul the arm codebase, but to ensure Stem will be up to snuff for a Tor GUI or web dashboard (projects I'd *love* to jump into someday). I'm delighted to announce a nice bit of progress on this front. September went toward rewriting its header panel (the top portion of arm's interface)... * Old zomg my eyes, they bleed! https://gitweb.torproject.org/arm.git/blob/e249dc8f5c4c282326324161cf8421f9… * New hotness https://gitweb.torproject.org/arm.git/blob/HEAD:/arm/header_panel.py As mentioned though this is really more about Stem than arm. Stem improvements made along the way this month include... * Far better handling for the 'private' keyword (the default prefix) and the default suffix in our ExitPolicy class. (#10107) * New BaseController.connection_time() method that provides when our control connection was established or detached. [1] * New Controller.get_accounting_stats() method which reports accounting information for our relay. [2] * Controller now fetches our own descriptor if no fingerprint is provided to get_server_descriptor(), get_network_status(), or get_microdescriptor(). * New crop() and join() methods for the str_tools module. [3][4] * Dropped a redundant get_* prefix from most of our util function names. Old names still work as aliases. * Our launch_tor_with_config() function raised an OSError if called too many times, caught thanks to RSenet. (#13141) DocTor, the monitor for directory authority health, also got some love this month... * Corrected suppression behavior when we have staggered alerts. This should greatly cut down on the amount of noise on the list. * We now check if too many relays lack a bandwidth authority measurement... or we would if this wouldn't perpetually be in alarm. (#12989, #13046) * Revised DocTor's schedule as requested by Sebastian. (#13210) * Expanded suppression for messages about turtles and kicked off a conversation to at long last sort this out. (#13296) One final note is that I uploaded my Tor Ecosystem presentation to YouTube so volunteer page visitors can see a streaming presentation rather than downloading a 53 MB video... https://www.youtube.com/watch?v=fb6iqZcQsSg Cheers! -Damian [1] https://stem.torproject.org/api/control.html#stem.control.BaseController.co… [2] https://stem.torproject.org/api/control.html#stem.control.Controller.get_ac… [3] https://stem.torproject.org/api/util/str_tools.html#stem.util.str_tools.crop [4] https://stem.torproject.org/api/util/str_tools.html#stem.util.str_tools.join

1 0

Getting the network consensus
by Michael Rogers 30 Sep '14

30 Sep '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi all, While working on peer-to-peer hidden services I've been wondering how long two clients need to wait before arriving at the same view of the Tor network. The answer, which I find surprising, is that this is never guaranteed to happen. Here's why. In what follows I'll assume that the interval between network consensuses being published is one hour, but the conclusions remain the same if the interval is longer or shorter - just change the units. I'll use "1:00 consensus" to mean a consensus that becomes valid at 1:00, is fresh until 2:00, and is valid until 4:00. According to dir-spec.txt, each directory cache downloads the 1:00 consensus at a randomly chosen time between 1:00 and 1:30. Each client replaces the 1:00 consensus with a newer consensus at a randomly chosen time between 2:45 and 3:51. (The precise endpoint is 3:50:37.5, due to a calculation described in section 5.1 of the spec. I'm curious to know the origin of that calculation.) Some observations: 1. The period when clients are replacing each consensus slightly overlaps the period when they're replacing the previous consensus. For about five and a half minutes of every hour there are twice as many clients replacing their consensuses, so directory caches will experience greater load during this period. How much greater? We can divide clients that are downloading the consensus into three groups: A. Clients that have never downloaded a consensus. B. Clients that are replacing a consensus that's no longer valid. C. Clients that are replacing a valid consensus with a newer consensus. Clients in group A bypass the directory caches and hit the authorities, so they don't contribute to load on the caches. Clients in group B have just started up and loaded an expired consensus from disk. These clients are equally likely to hit the caches at any point in the consensus interval (although the load will vary by time of day, day of week, etc). Clients in group C are twice as likely to hit the caches between 45 minutes and 51 minutes past the hour as at any other time, due to the overlap between replacement intervals. So unless group C is dwarfed by group B, we should see a noticeable increase in load on the caches between 45 and 51 minutes past the hour. (If group C is dwarfed by group B then the increase will be lost in the noise.) If you run a directory cache, do your logs show that pattern? 2. The time a client chooses for replacing a newly downloaded consensus may be earlier than the time it chose for replacing the previous consensus. Again, this is due to the overlap between replacement intervals: a client may replace the 1:00 consensus at 3:48 and then choose 3:47 as the random time to replace the newly downloaded 2:00 consensus. What happens in that case? I haven't looked at the code, but I'm going to speculate that it doesn't explode in flames or travel through time. A reasonable way to handle the situation would be to round up the replacement time to the present time - in other words, to replace the newly downloaded consensus immediately. If that guess is correct, the extra load during the overlap between replacement intervals will get squeezed towards the end of the overlap, as replacement times occasionally get rounded up but never get rounded down. Some clients will download two consensuses back-to-back during the overlap. Again, if you run a directory cache I'd be interested to know whether you're seeing this pattern. 3. Clients often skip consensuses. The interval for clients to replace the 1:00 consensus runs from 2:45 to 3:51, and the interval for caches to download the 3:00 consensus runs from 3:00 to 3:30. So there's roughly a 46% chance that a client replacing the 1:00 consensus will get the 2:00 consensus, and roughly a 54% chance that it will get the 3:00 consensus. This means it's possible for two clients to replace their consensuses regularly but never see the same consensus: one of them sees the consensuses published on odd hours, the other sees the consensuses published on even hours. This situation is unlikely to continue for long if each client makes a fresh random choice for each replacement time, but there's no time at which we can say the clients will definitely have seen the same consensus. I can't see any partitioning attacks arising from this, but I thought I'd point it out anyway because I find the result surprising. As I said before, I'm curious to know the origin of the calculation in section 5.1 that produces the overlap between replacement intervals. The replacement time is chosen "uniformly at random from the interval between the time 3/4 into the first interval after the consensus is no longer fresh, and 7/8 of the time remaining after that before the consensus is invalid". It's the 7/8 part that results in the odd figure of 50 minutes 37.5 seconds. If the replacement time were to be chosen uniformly at random from the interval between the time 3/4 into the first interval after the consensus is no longer fresh, and the time one consensus interval after that, there would be no gap or overlap between replacement intervals. This wouldn't affect the third observation, however - there would still be no guarantee of two clients ever seeing the same consensus. Cheers, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJUKr8YAAoJEBEET9GfxSfM8NQIALyS7LuGYrxPfktaSD4pdnNL TTW1t/C4wyBYawn6xmu9Tf32Lbyjsb2Jt+6l+kY6R+8m0meR+zglxCozcivPtH2J RTnaPLOOA6SW1ibNiDSLfw0Zbm4GMF4Z8GrG+9PGCMUIaLccpmAtigDVDBSiFL2D JDSk3ep3ln/PIFH6OljKV8gSMqNPlhHqHc9+gxNlPUXvjJdcsJZPpTCnMlMKPDiB X5uk4k3E27r3pGM26kSSi3N5BYL6QAN5dXkH5hlgB/+XseSXz/KVtnfgOXuoKou2 bK5lszTLbc9xHNBW+d4EYOkI6LwjQT4xcSYlHYW/a28s+U5rvFHuM/UqGF291SE= =gPDQ -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] [tor-relays] [warn] No unused circIDs found on channel without wide circID support
by ra 29 Sep '14

29 Sep '14

reply-to: tor-dev On Monday 29 September 2014 15:00:29 tor-admin wrote: > since upgrading torland to 0.2.5.8-rc I see the below warnings. Is this > something to worry about? IMHO this might be due to a programming error in the circuit ID selection code or possibly indicate a DoS attack. > -------------------------------------------------------------------------- > Sep 28 16:07:24.000 [warn] No unused circIDs found on channel without wide > circID support, with 0 inbound and 4 outbound circuits. Found 0 circuit IDs > in use by circuits, and 64 with pending destroy cells. (0 of those were > marked bogusly.) The ones with pending destroy cells have been marked > unusable for an average of 7594 seconds and a maximum of 16137 seconds. > This channel is 18469 seconds old. Failing a circuit. > Sep 28 16:07:24.000 [warn] Circuitmux on this channel has 4 circuits, of > which 4 are active. It says it has 29535 destroy cells queued. Those lines are important. I would suggest opening a ticket on trac. See #11553. Best, Robert

1 0

Re: [tor-dev] Scaling tor for a global population
by teor 28 Sep '14

28 Sep '14

On 27 Sep 2014, at 10:18 , tor-dev-request(a)lists.torproject.org wrote: > Date: Fri, 26 Sep 2014 17:18:07 -0700 > From: Mike Perry <mikeperry(a)torproject.org> > To: tor-dev(a)lists.torproject.org > Subject: Re: [tor-dev] Scaling tor for a global population > > 2. Cut off relays below the median capacity, and turn them into bridges. > > Relays in the top 10% of the network are 164 times faster than > relays in the 50-60% range, 1400 times faster than relays in the > 70-80% range, and 35000 times faster than relays in the 90-100% range. > > In fact, many relays are so slow that they provide less bytes to the > network than it costs to tell all of our users about them. There > should be a sweet spot where we can set this cutoff such that the > overhead from directory activity balances the loss of capacity from > these relays, as a function of userbase size. > > Result: ~2X reduction in consensus and directory size. Do these extra relays function as spare (albeit slower) capacity in the network? If so, we'd have to be careful to set the cut-off low enough that any spikes in usage could be accommodated. Also, doesn't reducing the number of routers in the consensus risk harming network diversity? I understand that if these routers are a net burden on the network, then they're not actually contributing much to diversity overall. But what about users with obscure configurations like "New Zealand routers only", or "custom list of 20 trusted routers"? (Is a restricted set of routers a common enough use case?) Do we risk pulling the data they need from the consensus? Then again, users with these sorts of configurations are probably technically savvy enough to fix any issues that occur with any changeover. T teor pgp 0xABFED1AC hkp://pgp.mit.edu/ http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w…

1 0

Re: [tor-dev] Scaling tor for a global population
by Ryan Carboni 28 Sep '14

28 Sep '14

> > But, because this is fraction rises with both D and U, these research > papers rightly point out that you can't keep adding relays *and* users > and expect Tor to scale. Broadcast a fraction of all available directories? Use md5 as a random number generator, hash the ECC/RSA keys using md5. A user connecting to the network will generate an 8-bit random value, and based on that, will download one of 1/256 directories. Right now, any relay with more than ~100Mbit of capacity really > needs to run an additional tor relay instance on that link to make > use of it. If they have AES-NI, this might go up to 300Mbit. Any plans to use ChaCha8 instead of AES? It would an order of magnitude faster. It is also unlikely for ChaCha8 to become sufficiently insecure to effect web-size non-video traffic.

1 0

obfs4 test bundles
by Yawning Angel 26 Sep '14

26 Sep '14

Hello all, So after a brief pause in my productivity due to the dev meeting and some IRL things, I'm back in the swing of development and have been working on getting obfs4 ready for development. I've recently mostly finished a fairly large cleanup/refactor of the obfs4 codebase and dusted off the Tor Browser build integration patch. Test bundles signed with my PGP key are available at: https://people.torproject.org/~yawning/volatile/tor-browser-obfs4-20140820/ Some quick notes: * The bundles are alpha. DO NOT USE THEM IF YOU REQUIRE STRONG ANONYMITY OR SECURITY. * While it is possible to pull out the obfs4proxy binary from the bundle and use it to run a bridge, that will be a *terrible* idea because the bundles are missing some logging related fixes. * The Windows and OSX bundles are entirely untested, because I do not have access to either platform. The Linux bundle appears to work. * Using obfs2/obfs3 will also exercise new code (obfs4proxy instead of obfsproxy). * I'm not uploading a 32 bit linux bundle for now. * The one obfs4 test bridge is mine, and is the one I posted to tor-dev@ (port is different, bridge parameters are the same). If you wish to follow obfs4 deployment the bug associated with this task is #12130. Questions, comments, feedback welcome. -- Yawning Angel

1 2

Pluggable transports meeting today (16:00UTC Wednesday 24th of September 2014)
by George Kadianakis 24 Sep '14

24 Sep '14

Hello! just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur today at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. As you see, the meeting day was recently changed from Fridays to Wednesdays: https://lists.torproject.org/pipermail/tor-dev/2014-August/007317.html Thanks for your attention!

2 1

Making and distributing custom TBB with a new "home-page"
by Fabio Pietrosanti (naif) 23 Sep '14

23 Sep '14

Hi all, for a very interesting deployment of GlobaLeaks in the are of Human Rights defense, we will have the need to distribute a customized Tor Browser Bundle to the sources. The "customization" requirement is simple: Have as a default home-page the GlobaLeaks .onion site . We must go that way because: - the "target country" where the sources are cannot download TBB due to torproject.org being censored - the sources are absolutely non-technologically savy (average 60yo lawyers doing human rights defense) The website where there will be the leaking instructions and the download of such custom TBB will be "privately distributed" trough word of mouth and trusted connections, with no public solicitation. So we must do some piece of software that will: - Download TBB in specific languages (2-3 specific languages) for each platform - Unpack TBB (in all formats for Windows, OSX, Linux) - Apply the customization (set the home-page, with slightly different parameters depending on the language) - Check periodically if a new version is available and, in that case, re-execute the process described above to release updated version of TBB The questions are: a) Which is a simple/stable/resilient way to check which is the latest version of TBB b) Does someone have already done that kind of customization-process? c) Can everything be done from Linux, like a cron-job, in a fully automatic way? d) Which other customization / ideas / concern are there regarding this process? I'd personally love if the customization would enable me to completely disable the "URL Bar" and all of the Browser Button in order to make it useful only to use it as a console to send information being a source, without the possibility to go browse other sites. Waiting for comments before writing some quick specs -- Fabio Pietrosanti (naif) HERMES - Center for Transparency and Digital Human Rights http://logioshermes.org - http://globaleaks.org - http://tor2web.org

3 4

meek on Microsoft Azure
by David Fifield 22 Sep '14

22 Sep '14

I've got meek working on Microsoft Azure, a cloud services platform and CDN. If you'd like to try it, use a 4.0-alpha bundle and enter the bridge line: Bridge meek 0.0.2.0:3 url=https://az668014.vo.msecnd.net/ front=ajax.aspnetcdn.com As usual, the IP address is ignored; I'm just assigning incrementing numbers in the port. az668014.vo.msecnd.net is a CDN domain that points to meek-reflect.azurewebsites.net, where the service is really running. The front domain ajax.aspnetcdn.com is one that hosts a bunch of common web libraries: http://www.asp.net/ajax/cdn. I didn't look very hard to see if there are other good front domains. More information on Azure: https://trac.torproject.org/projects/tor/wiki/doc/meek#MicrosoftAzure David Fifield

1 0

Attack linking Tor circuits
by Otto Huhta 19 Sep '14

19 Sep '14

Dear Tor developers, We’ve been working on an MSc Thesis project looking into an attack that links different Tor circuits back to the same user, using only information available to a Tor middle node. We’ve discovered that an adversary can quite accurately determine if two consecutive circuits are used by the same user just by looking at when these circuits are built and destroyed or when they begin and stop relaying traffic. This is mainly the result of the fixed 10 minute circuit lifetime and the fact that the transition to using a new circuit is quite sharp. According to our measurements, circuits belonging to the same user are built at very close to 10 minute intervals. Also, when one circuit stops relaying traffic another one suddenly becomes active, at almost the very same moment. Thanks to these properties we could train a classifier to quite efficiently identify matching pairs of circuits (9% EER). We looked into the threat posed by this attack to Tor users and concluded that because of entry guards the threat is real, given that an adversary can control a significant portion of middle node traffic. Finding matches among all possible Tor circuits would result in too many false positives but thanks to entry guards an adversary can focus only on circuits built through these specific nodes and quite efficiently determine if two circuits belong to the same user. Possible plans of moving to using only a single guard node make the situation much more severe: we concluded that for a user picking a slow guard node, the adversary can find almost all circuits belonging to a same user (again assuming control of a large portion of middle node bandwidth). We thus see that before moving on with any such plans, one should consider the effect of our findings on the threat posed by a middle node linking user circuits. As a final note, we considered the effects of randomizing the circuit lifetime (e.g. between 5-15 minutes) to mitigate this threat but concluded that it would not necessarily solve the problem. The most prominent feature linking two circuits together seems to be the time difference between traffic ending on one circuit and beginning on another circuit, and this link cannot be broken by varying the circuit lifetime. However, if someone were to devise such a modification, we would be happy to re-run our tests to see what kind of an effect this would have on the success rate of an adversary. For those interested in our findings, the full thesis Linking Tor Circuits can be found at: http://www0.cs.ucl.ac.uk/staff/G.Danezis/students/Huhta14-UCL-Msc.pdf Best regards, Otto Huhta, UCL

2 3

Responsible disclosure
by Bram de Boer 18 Sep '14

18 Sep '14

Hi, How can I responsibly report a bug that might affect security (e.g. possibility to DoS Tor nodes)? I searched the torproject.org website, but couldn't find any pointers with respect to responsible disclosure. Do I just file a trac ticket and/or drop it in this mailinglist? Do I report it directly to some of the key players in this project (Roger, Nick, etc.)? Thanks, Bram

3 2

Reminder OONI dev meeting happening tomorrow
by Arturo Filastò 18 Sep '14

18 Sep '14

Hi Oonitarians, This is a reminder to inform you that tomorrow (Friday 19th of September) there will the weekly dev meeting. It happens on IRC (irc.oftc.net) on the channel #ooni at 17:00 CET (15:00 UTC). Everyone is invited! See you tomorrow! ~ Art.

1 0

Gitian Build Issue - Tor Browser
by SiNA Rabbani 17 Sep '14

17 Sep '14

Dear Team, I need some help getting gitian-build to work for the Tor Browser. Can anyone help with this error please? ./mkbundle-windows.sh adding: Docs/ (stored 0%) adding: Docs/ChangeLog.txt (deflated 68%) adding: Docs/Licenses/ (stored 0%) adding: Docs/Licenses/Firefox.txt (deflated 68%) adding: Docs/Licenses/HTTPS-Everywhere.txt (deflated 49%) adding: Docs/Licenses/NoScript.txt (deflated 62%) adding: Docs/Licenses/PluggableTransports/ (stored 0%) adding: Docs/Licenses/PluggableTransports/LICENSE (deflated 73%) adding: Docs/Licenses/PluggableTransports/LICENSE.CC0 (deflated 60%) adding: Docs/Licenses/PluggableTransports/LICENSE.GO (deflated 47%) adding: Docs/Licenses/PluggableTransports/LICENSE.PYTHON (deflated 78%) adding: Docs/Licenses/Tor-Launcher.txt (deflated 48%) adding: Docs/Licenses/Tor.txt (deflated 75%) adding: Docs/Licenses/Torbutton.txt (deflated 46%) adding: Data/ (stored 0%) adding: Data/Browser/ (stored 0%) adding: Data/Browser/profile.default/ (stored 0%) adding: Data/Browser/profile.default/bookmarks.html (deflated 70%) adding: Data/Browser/profile.default/preferences/ (stored 0%) adding: Data/Browser/profile.default/preferences/extension-overrides.js (deflated 68%) adding: Data/Browser/profiles.ini (deflated 21%) adding: Data/Tor/ (stored 0%) adding: Data/Tor/torrc (stored 0%) adding: Data/Tor/torrc-defaults (deflated 29%) ****** SKIPPING already built Utilities Component of Windows Bundle (1/5 for Windows) ****** ****** Starting Tor Component of Windows Bundle (2/5 for Windows) ****** --- Building tor-windows for precise i386 --- Stopping target if it is up Making a new image copy Formatting 'target-precise-i386.qcow2', fmt=qcow2 size=11811160064 backing_file='base-precise-i386.qcow2' encryption=off cluster_size=65536 Starting target Checking if target is up. Preparing build environment Updating apt-get repository (log in var/install.log) Installing additional packages (log in var/install.log) Grabbing package manifest Creating build script (var/build-script) Running build script (log in var/build.log) ./bin/gbuild:21:in `system!': failed to run on-target setarch i386 bash -x < var/build-script > var/build.log 2>&1 (RuntimeError) from ./bin/gbuild:122:in `build_one_configuration' from ./bin/gbuild:224 from ./bin/gbuild:219:in `each' from ./bin/gbuild:219 from ./bin/gbuild:217:in `each' from ./bin/gbuild:217 All the best, SiNA

3 3

Re: [tor-dev] Guardiness: Yet another external dirauth script
by Tim 17 Sep '14

17 Sep '14

On 17 Sep 2014, at 22:00 , George Kadianakis <desnacked(a)riseup.net> wrote: > Sebastian Hahn <sebastian(a)torproject.org> writes: > >> On 16 Sep 2014, at 16:15, George Kadianakis <desnacked(a)riseup.net> wrote: >> >>> ====How the guardiness script will be deployed==== >>> >>> The idea is that dirauths will add another script to their crontab >>> that is called every hour (before or after the bwauth scripts). >> >> Only 4/9 have such scripts, if at all - it is possible to run a >> bwauth on a different host, and scp the file over. I don't know >> if any of the dirauth ops actually do this currently. >> > > This should also be possible with the guardiness stuff, as long as you > can arrange fresh consensuses to arrive on the second host. tor supports consensus downloads without being a router, but it's a bit of a hack: A tor instance configured with a DirPort and no ORPort will act as a directory cache only. Directory caches regularly download the latest consensuses. It's worth binding the DirPort to localhost only, and marking it NoAdvertise. This hides the cache from the consensus, and disables remote requests. However, the cache will also download router descriptors unnecessarily. (Microdescriptors and extra info can be turned off, but I'm not sure if router descriptors can.) T

2 1

[RFC] Proposal draft: The move to a single guard node
by George Kadianakis 17 Sep '14

17 Sep '14

> On Mon, Apr 7, 2014 at 11:34 AM, George Kadianakis > So, based on your > response, IIUC, the idea is that because young > > guards are underutilized, we want to increase the probability of them > > being chosen in non-guard positions, so that they become more utilized > > till more people pick them as guards? > > Right. > > > Some questions on the terminology you used: > > > > a) What do you mean by 'last rotation period'? When you say "for > > fraction k of the last rotation period", you mean that if the > > authorities read consensuses for the past 12 months, and the relay R > > was up as a guard for 6 months, then k would be 6/12 == 0.5? > > The "rotation period" is the average length of time between choosing > guards. For example, with the current parameters (IIRC), clients > discard a guard and choose a new one at a time uniformly chosen > between 30 and 60 days. So the rotation period is 45 days = 45*24 > hours = 1080 consensus votes, and essentially in each of those hours, > 1/1080 of the clients choose a new guard. So the last rotation > period's worth of status documents should be the only ones that matter > for figuring out how heavily a node is used as a guard. For example, > if a relay has had the guard flag for 15 days (=360 consensus > documents) out of the last 45, then that relay has about k=360/1080 of > the number of clients using it as a guard as it would have, if it had > been a guard for the full 1080 hours. > Ah, I see. I originally misinterpreted what you meant by 'last rotation period'. > (N.B. I'm making an assumption that guard rotations will be uniformly > distributed over the period. This should be safe in equilibrium, but > might get kind of shaky when the period first increases to 9 months or > whatever the final decision is) > Yep. We also don't consider new releases of TBB/Tails (many users pick new entry guards), or sudden bulk arrival of users (botnets, real life events, etc.). > > b) By (weight with the guard flag) you mean the result of: > > <consensus BW> * <consensus weight> ? > > > > Is that right, or did I misunderstand you? > > Yes. > > > Assuming the above terminology assumptions, I began trying to > > understand your formula. First of all, I was wondering how you ended > > up with it? Is this some standard form? I'm not very familiar with > > these things. > > Well, the way the weights work is that they compute what fraction of > guard bandwidth should be used for middle and exit traffic, assuming > that the rest will be all used up by client-guard connections. That > is, the weights are telling us to use (1-(<consensus middle weight> + > <consensus exit weight>)) fraction of the guard's bandwidth for these > connections. > > But if a relay has only fraction k of its "guard bandwidth" used up > this way, then (1-k) of this bandwidth should go back into the pool > for other positions. I think that's what the formula does, but I > could have it wrong. Does that make sense? > I see. That makes sense, I think. I will ponder on this a bit more, and then edit the proposal. (I would like to think a bit more about how accurate the assumption "relay has been a guard for 0.25 of the rotation period => relay has 0.25 of its bandwidth occupied for guard purposes") Thanks!

3 6

Temporary decrease in FTE users in late August?
by David Fifield 17 Sep '14

17 Sep '14

Kevin, Do you know the reason why FTE usage dropped from 200 to about 50 at the end of August and then recovered again? https://metrics.torproject.org/users.html?graph=userstats-bridge-transport&… I had a guess that maybe it had to do with availability of new bridges. If a bunch of bridges got blocked and then new bridges were made available, that could explain it. The 4.0-alpha-2 release was on September 3, but as far as I know it didn't contain new FTE bridges. They could have been added to BridgeDB, I suppose. David Fifield

2 1

Preferred OpenSSL config for Tor?
by Steve Snyder 16 Sep '14

16 Sep '14

What are the recommended build options for OpenSSL 1.0.1x when building it for use with Tor v0.2.5.x? Put another way, what default OpenSSL features (ciphers, etc.) should be disabled and what optional features should be enabled? My goal here is to build a Tor-oriented OpenSSL, one that dispenses with features Tor doesn't need/use while enabling those that improve security and/or performance. Thanks.

2 1

Transparent Proxy: how does it work?
by CJ 16 Sep '14

16 Sep '14

Hello dear torrorists :) I'm wanting to do some weird things with iptables in order to force some connections through Tor's TransProxy, but before that I have an interrogation on its internals: How does it detect if we're wanting to use, let's say, SMTP over Tor? Is there any kind of sniffer that tells tor "ok, so this one wants to go to mail.google.com:25, this other one to www.ethack.org:443 with SSL" ? If so, how does it work? It seems it may cause some problems when using Tor on a gateway, like some android device with tethering: forcing tethered connections through tor [orbot] seems to be really hard. Thank you in advance for your enlightenment. Cheers, C.

1 0