November 2014 - tor-dev - lists.torproject.org

Re: [tor-dev] [HTTPS-Everywhere] "darkweb everywhere" extension
by teor 09 Nov '14

09 Nov '14

Hi All, A "privacy everywhere" solution could have two components: 1. the HSTS-like "always open this site in .onion" policy already discussed; and 2. an AppLinks or AppLinks-like[1] header that specifies a preferred app and/or URL for Tor, I2P, namecoin, … The first handles the situation where you're already in the Tor browser, I2P, namecoin, and simply wish to pin the .onion etc. URL for that site. The second handles the situation where you're browsing the web in your standard browser, but want to switch to a .onion site in the Tor browser if one is available (or the equivalent namecoin or I2P action). AppLinks is mainly designed for mobile platforms, but has Windows schemes as well. I could imagine schemes like: al:onion:url https://facebookcorewwwi.onion al:onion:attribute value al:namecoin:url https://... al:namecoin:attribute value al:i2p:url https://... al:i2p:attribute value What do you think? I wonder if this is helpful, or could just end up being out of scope, duplicating effort, or abusing the AppLinks protocol design (which is focused on an app per platform, not multiple options for alternate URLs). T [1]: http://applinks.org/documentation/#applinknavigationprotocol teor pgp 0xABFED1AC hkp://pgp.mit.edu/ https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w… > On 3 Nov 2014, at 23:00, tor-dev-request(a)lists.torproject.org wrote: > Date: Mon, 03 Nov 2014 00:12:53 -0600 > From: Jeremy Rand <biolizard89(a)gmail.com> > To: tor-dev(a)lists.torproject.org, https-everywhere > <https-everywhere(a)lists.eff.org>, namecoin(a)googlegroups.com > Subject: Re: [tor-dev] [HTTPS-Everywhere] "darkweb everywhere" > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Yan, > > Namecoin would definitely be interested in something similar (we were > actually discussing the possibility of exactly this yesterday). Maybe > we could produce a list of relevant projects that would benefit from > this? (The three that come to mind immediately are Tor, I2P, and > Namecoin, but there may be others.) If there are more than a few > projects that would benefit, then it might be interesting to find a > neutral format for the HTTP header, so that we wouldn't have to list > all the supported TLD's explicitly in the spec. > > (CCing to Namecoin dev list.) > > - -Jeremy Rand > Lead Application Engineer, Namecoin Project > >> On 11/02/2014 11:48 PM, yan wrote: >> +tor-dev. tl;dr: Would be nice if there were an HTTP response >> header that allows HTTPS servers to indicate their .onion domain >> names so that HTTPS Everywhere can automatically redirect to the >> .onion version in the future if the user chooses a "use THS when >> available" preference. >> >> I imagine the header semantics and processing would be similar to >> HSTS. It would only be noted when sent over TLS and have the >> max-age and include-subdomains fields. >> >> -yan >> >> yan wrote: >>> Hi all, >>> >>> Some people have requested for the "Darkweb Everywhere" extension >>> [1] to be integrated into HTTPS Everywhere. This is an extension >>> for Tor Browser that redirects users to the Tor Hidden Service >>> version of a website when possible. >>> >>> I'm supportive of the idea; however, I'm worried that since >>> .onion domain names are usually unrelated to a site's regular >>> domain name, a malicious ruleset would be hard to detect. AFAIK >>> Darkweb Everywhere only defends against this by publishing a doc >>> in their Github repo that cites evidence for each ruleset [2]. >>> >>> What if, instead, we asked website owners to send an HTTP header >>> that indicates the Tor Hidden Service version of their website? >>> Then HTTPS Everywhere could cache the result (like HSTS) and >>> redirect to the THS version automatically in the future if the >>> user opts-in. >>> >>> If this is something that EFF/Tor would be willing to advocate >>> for, I would be happy to draft a specification for the header >>> syntax and intended UA behavior. >>> >>> Thanks, Yan >>> >>> >>> [1] https://github.com/chris-barry/darkweb-everywhere/ [2] >>> https://github.com/chris-barry/darkweb-everywhere/blob/master/doc/EVIDENCE.… > _______________________________________________ >>> HTTPS-Everywhere mailing list HTTPS-Everywhere(a)lists.eff.org >>> https://lists.eff.org/mailman/listinfo/https-everywhere >> >> _______________________________________________ tor-dev mailing >> list tor-dev(a)lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEcBAEBAgAGBQJUVxzXAAoJEFgMI9bDV/9qY3UIAJrl5LI/1OHJngu1W9DsLAjr > nh+Csnm66z5tQTwiwva1Tb4b6trHv4KkHItaTm0cI44mQNsd+YEkh0oRBTSNNcRm > HY0BDn2pqTlQPN9bWvclGEtCacevCbaQiZgPpxPa+1crtavto4VAnv0/EI85QVAe > XHUNBeAHmB3qNATXsVJ61oksWlU/x8ao62fB13cUd2fVyaasWz4PPsAJ9n3TkdYG > /el7mAuM6XdA1fFaGFd1ta0jRuER2VgKQvJQctu/6a/9jiNlib3YmMOOxvF0WR+/ > foUdhFkNCmRWwxqnxFDiKM0ilRLjTQ47CYRkgkqD4azPlkNvUULbO3KhaWPB9/4= > =rUsH > -----END PGP SIGNATURE----- >

2 1

Re: [tor-dev] [tor-internal] HS attack blog post (was Re: Hidden services and drug markets takedown)
by A. Johnson 09 Nov '14

09 Nov '14

I think the option to rate-limit guard selection is a great idea to defend against guard DoS. The downside is possible connection loss even if you’re not under attack and you just happen to pick flaky guards. In case you’re interested, I examined this defense and how often such benign service loss would occur in section 6.B of "The Sniper Attack: Anonymously Deanonymizing and Disabling the Tor Network” <http://ohmygodel.com/publications/sniper-ndss14.pdf>. Table 6 shows the probability that this happens for a Tor client that operates continuously for two months (after 2-3 months all guards will have expired and the process repeats). If, for example, you are willing to require only one active guard (a_g=1) and you limit yourself to no more than 4 new guards (r=4) chosen in the last 28 days (t=28), then you had a 0.0008 chance of having any down time (whether or not it happens depends on which guards you chose). If you increase the number of allowed new guards to 5 (r=5), then the probability of downtime was zero. Cheers, Aaron > Got it. Though on the client side, could we have a warning or have an option to hibernate a HS if they have been forced to switch guards N times in the last N minutes or hours or such? This would allow a DNS on the HS of course, but that may be preferable to discovery. > > David Chasteen > > PGP 0x48458ecd78833c0d > > On Nov 9, 2014 12:49 PM, "Matthew Finkel" <matthew.finkel(a)gmail.com> wrote: > On Sun, Nov 09, 2014 at 12:34:01PM -0500, David Chasteen wrote: > > Would it be possible to create some kind of tor weather-like detection > > and alert system to warn if such a massive DoS attack were underway such > > that users could know that perhaps now might not be the best possible > > time to use Tor? A hidden threat is worse than a known one. We're not > > going to be able to mitigate every known threat, but making users aware > > that the threat profile is heightened can allow them to make informed > > risk decisions. > > > > Unfortunately we don't receive any real-time statistics from relays, > and our metrics calculations are run daily, so there's a significant > delay in any visualizations we generate. Perhaps we can still look for > and see long-term attacks (> 24-36 hours) but nothing that will > significantly benefit users shortly after the attack begins. > _______________________________________________ > tor-internal mailing list > tor-internal(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-internal > _______________________________________________ > tor-internal mailing list > tor-internal(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-internal

1 0

Proposals 220 and 228 updated
by Nick Mathewson 07 Nov '14

07 Nov '14

Hi, all! I've checked in some small changes to proposals 220 and 228; please see the torspec git repository for information. These reflect changes that I have realized those proposals needed while I was working on implementing them in ticket 12498. best wishes, -- Nick

1 0

tor meetings for the week of November 10: including triage party!
by Nick Mathewson 07 Nov '14

07 Nov '14

Good morning/afternoon/evening! On Tuesday at 1700 UTC, there is the regular patch workshop. On Wednesday at 1330 UTC, there is our regularly scheduled development meeting. On Thursday at 1330 UTC, there is a bug triage party, where we will try to figure out what we need to solve in 0.2.6, and what we can mark as nice-to-have-but-deferrable. All meetings are on the #tor-dev channel on OFTC; if we get too noisy or have a conflict, we'll move somewhere else. Everybody is welcome! yrs, -- Nick

1 0

Why the seeming anticorrelation between obfs3 and vanilla bridges in metrics graphs?
by David Fifield 07 Nov '14

07 Nov '14

In the past few months of bridge user graphs, there is an apparent negative correlation between obfs3 users and vanilla users: when one goes up, the other goes down. If you draw a horizontal line at about 5500, they are almost mirror images of each other. I don't see it with any other transport pairs. Any idea why it might be? I can see what could cause a simultaneous decrease in vanilla and increase in obfs3: Tor gets blocked somewhere and users switch to obfs3. But I wouldn't expect blocking events to look so smooth or happen so frequently, and it doesn't explain why the reverse change happens later (obfs3 being blocked while Tor is unblocked is less plausible). I can also understand the overall long-term trend of obfs3 increasing and vanilla decreasing. But I don't see why they should mirror each other so closely over short time periods. Some hypotheses: 1. There are lots of users who have a mix of vanilla and obfs3 bridges configured. Their tor (randomly?) chooses one of them, which usually works. The number of such users is constant over the short term; i.e. the sum of obfs3+vanilla is constant, but the proportion of obfs3 and vanilla fluctuates randomly. 2. Maybe vanilla-down/obfs3-up is caused by blocking events, and vanilla-up/obfs3-down is caused by natural new-user churn and/or coincidence. 3. There is something about the way BridgeDB hands out bridges, or the way in which users use it, that causes it to give out obfs3 bridges at the expense of vanilla and vice versa. 4. Some kind of feedback loop: obfs3 bridges get used and get congested, so users switch to vanilla, which then get used and congested, etc. David Fifield

3 7

Help regarding contribution
by harsh sangwan 06 Nov '14

06 Nov '14

Sir / Madam, I am an undergraduate student. Recently, I've developed huge interest in Tor & thus would like to contribute to it by adding code to its projects. Problem is, I don't know much about contributing to open source & that's where I would like you to help me. I have expert skills in C/C++ & shell scripting. Can you please help me with these things ? 1. Finding me appropriate mentor & project given my technical skills. 2. Guiding me about the prerequisites I should fulfill before I can start contributing to Tor. Hope to see a good response.

2 1

OpenBSD in doc/TUNING
by Libertas 05 Nov '14

05 Nov '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I think it would be a good idea to add OpenBSD to doc/TUNING because its file descriptor limits are a little confusing, and because promoting OpenBSD relays benefits the Tor network's security. By default, OpenBSD limits the total number of file descriptors to 7030. This can be changed only by recompiling the kernel: http://unix.stackexchange.com/questions/104929/does-openbsd-have-a-limit-to… However, stricter limits are set in /etc/login.conf so that a single user cannot open all available file descriptors, preventing others from opening files. In this config file, openfiles-cur is the max upon login, and openfiles-max is the highest limit that can be set (even using ulimit -n) without changing /etc/login.conf and restarting. The _tor user is in the daemon class, at least on my install. You can check using 'userinfo _tor'. This means that the daemon class rules in /etc/login.conf also apply to Tor. See 'man login.conf' and Absolute OpenBSD (2nd Ed.) for more info. If I recall correctly, my OpenBSD relay maxed out at 1024 before I changed /etc/login.conf. This agrees with what I'm seeing in contrib/dist/tor.sh.in, and (again, IIRC) also happened to be the openfiles-max value. It's worth noting that contrib/dist/tor.sh.in only checks for the maximum file descriptor count in /proc/sys/fs/file-max (lines 53-65 on the master branch), which doesn't exist in OpenBSD, so Tor will probably default to 1024 regardless of what openfiles-max is. I'm still a beginner, so let me know if I made any mistakes. Also, let me know if I should send this to the relays list as well. I can write up a short draft paragraph if people think this is accurate and worthwhile. Help is welcome. Libertas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJUWQv+AAoJELxHvGCsI27NXvAP/i5Q0wWuSY7d+DUqJfdddju1 FoEf27qtLV6ze8WscWQgCHciTTdXDkW43OVptcTbuwJ9TOYMPPhtF/6ENgizxoin 37I1lVlm5sZ926UKRwkOapc4keJ910gpcePZJ9vgbCSNJhRO7tg4BKdhLJfeRY0f CaEFTB6bHigqfGAWeQjzDfGZ3LifawrdoN1tTDXpqpRH9tNhcAD+pvepP4Pze3xm lVUcQmD/Dn4rpQTS7IrNkR66WtR/NJ2xc8JSXrX/81hKT+SbNGmCjTxIc7uWrI0S aTrI9oAeQtJ4LRrlaZBzc1C1JbkW6xB5oNjtR6JEtK536/9nmzrRmZNGeKOtpGrk rIEwJjwrv5sNYRhul8Hl6ngrjIzwQLmZR+aNda0USEaRLpfUBYiFg9rqxTPLMU3Q VOo3Hf08zQTXCddcizs19Q1km67scJ5PkgSdkm8qS06H19uQEXXKcHOtUstNxGrY GxWAD5RIm/53QIYurNJW4UctwvzCuanviHTkWCDn0dwV42HvUUfJCgOnmkuf9n1+ ZMxxT1F49WV4pBEPEVp4cRN9NPqkfMMMtKLCtM6WxoJNXdkIyQz46D9ZNahaHFOh TuehsBW29GT/w9FW1APKeMleq+eV9lpZb2GbWXrjnFr7MRuCJZVCz5wzr11hnept trInFflWvbIWtp4yv6Vf =bm5p -----END PGP SIGNATURE-----

2 1

Damian's Status Report - October 2014
by Damian Johnson 05 Nov '14

05 Nov '14

I love the beginning of winter. Landscape a chilly tundra, no sun, yet still no snow to slip on. What I *don't* love, however, is all the sicknesses that float around. This month has been uncommonly unlucky for me with three separate bugs making the rounds. No single one was a big whoop, but still can't say I'm a fan of things making themselves at home in my warm, cozy respiratory track. Oh well. Enough of that. This month was mostly focused on arm, specifically rewriting the graph panel. Work here is ongoing, but I hope to have this section done in November. Other noteworthy things this month include... * Stem now has a set of methods to more easily work with hidden services thanks to federico3 and Patrick [3]. New methods include... * get_hidden_service_conf * set_hidden_service_conf * create_hidden_service * remove_hidden_service * Addressed OOM issues with DocTor. DocTor uses quite a bit of memory, but the real trouble was that sending a notification at the end forked, doubling allocated memory. I addressed this by spawning the mail subprocess earlier, before we read in descriptor data. * Couple DocTor patches from Tom Ritter - thanks Tom! [1][2] * Subteam discussions with Karsten and OONI folks accumulating in the VegasTeam3 wiki [4]. I'm dubious that this will mean much (or anything) in practice, but we'll see. * Attended a couple local tech events including TA3M Seattle and Seagl [5]. * Got promoted at work! This was five years in the making, so I'm pretty delighted. :P Cheers! -Damian [1] https://trac.torproject.org/projects/tor/ticket/13389 [2] https://trac.torproject.org/projects/tor/ticket/13390 [3] https://trac.torproject.org/projects/tor/ticket/12533 [4] https://trac.torproject.org/projects/tor/wiki/doc/VegasTeam3 [5] http://seagl.org/

1 0

Summary of meek's costs, October 2014
by David Fifield 04 Nov '14

04 Nov '14

Here's the summary of meek's CDN fees for October 2014. Earlier reports: https://lists.torproject.org/pipermail/tor-dev/2014-August/007429.html https://lists.torproject.org/pipermail/tor-dev/2014-October/007576.html App Engine + Amazon + Azure = total by month February 2014 $0.09 + -- + -- = $0.09 March 2014 $0.00 + -- + -- = $0.00 April 2014 $0.73 + -- + -- = $0.73 May 2014 $0.69 + -- + -- = $0.69 June 2014 $0.65 + -- + -- = $0.65 July 2014 $0.56 + $0.00 + -- = $0.56 August 2014 $1.56 + $3.10 + -- = $4.66 September 2014 $4.02 + $4.59 + $0.00 = $8.61 October 2014 $40.85 + $130.29 + $0.00 = $171.14 -- total by CDN $49.15 + $137.98 + $0.00 = $187.13 grand total Golly! I'm as shocked as you are. There was an explosion in users after the release of Tor Browser 4.0 on October 15 (first attached image): https://metrics.torproject.org/users.html?graph=userstats-bridge-transport&… Before that, we were mostly under the free thresholds on each service. Now we're consistently exceeding those thresholds and getting charged at the normal rate. For example, Amazon gives us 50 GB free each month; last month we only used about 32 GB but this month it was about 480 GB. There was really only half a month of heavy use, so next month I expect the charges to be at least double. But I'm going to try to apply some free credits; see below. At the moment, the most cost-effective backend is meek-azure, because it's running on a free research grant for a year. obfs3 is currently the most-used transport, with about 8000 simultaneous users, compared to meek's 500. https://metrics.torproject.org/users.html?graph=userstats-bridge-transport&… Just looking at the time period from October 15 to October 31, the mean value of the obfs3 graph is 8075, and that of the meek graph is 450, a ratio of 18:1. About 94% of October's meek users appeared after October 15. So a rough calculation of meek's monthly cost, if it were to become as used as obfs3 is today, is $171.14 * 0.94 * (8075/450) * 3/2 ≈ $4300 The 3/2 comes from assuming that Azure will cost as much as the other two eventually. The other big assumption in this calculation is that costs scale linearly with users, which isn't quite true because of the free thresholds and because of regional pricing differences. == App Engine a.k.a. meek-google == This is the first month we had charges for "instance hours"; previously we paid only for bandwidth. App Engine by default runs just one copy of your app, but spins up new instances of it during increased load. You get 28 instance hours for free each day, so as long as there's just one instance you don't pay anything. The total cost of $40.85 broke down as follows: 289 GB $34.72 123 instance hours $6.13 The attached meek-google-costs-2014-11-01.png shows the breakdown of costs between bandwidth and instance hours for the last few months. On October 25 I experimented with upgrading the App Engine instance class from "F1" to "F2". https://cloud.google.com/appengine/docs/adminconsole/performancesettings#Se… F2 has more CPU and I thought it might speed up TLS. F2 uses up instance hours at twice the rate of F1, so even if you only have one instance you pay for 20 hours. I downgraded the class back to F1 on October 27 when, for the first time, we hit the daily spending limit of $5.00 I had set, about 22 hours into the day. (I happened to be using meek-google at the time, so I noticed right away when it stopped working.) I bumped back up to F2 and increased the daily spending limit on October 31. This month I'm going to try to activate a credit I have for App Engine, which is $500 for three months. We'll probably run out of the $500 before the three months are up, because we're currently pretty close to the $5 per day that implies. The attached meek-google-2014-11-01.png shows B/s for the month. == Amazon a.k.a. meek-amazon == The Amazon bill of $130.29 breaks down in a complicated way by region. Each region has different free thresholds and different per-unit rates. I've summarized it here but the actual bill has 31 line items. This month is the first month with bandwidth charges; previously we had only paid for the number of HTTPS requests. Asia Pacific (Singapore) 23M requests $27.86 169 GB $19.49 Asia Pacific (Sydney) 150K requests $0.00 1 GB $0.15 Asia Pacific (Tokyo) 16M requests $18.93 108 GB $18.28 EU (Ireland) 15M requests $16.22 108 GB $11.75 South America (Sao Paulo) 180K requests $0.00 1 GB $0.03 US East (Northern Virginia) 8M requests $7.77 92 GB $9.81 The attached meek-amazon-2014-11-01.png shows GB/day for the month. == Azure a.k.a. meek-azure == Azure is still on a free grant for a year. Unfortunately I don't see a way to find out what it would be costing or even get a report of monthly bandwidth usage. The attached meek-azure-2014-11-01.png shows GB/hour for the last seven days of the month. David Fifield

1 0

Patch meeting today at 1700 UTC; one-time day change for tor dev meeting
by Nick Mathewson 04 Nov '14

04 Nov '14

Hi! The weekly tor patch workshop is still 1700 UTC on Tuesdays. That is now _noon_ Eastern and 9:00 am Pacific in the US, thanks to our bizarre timekeeping system. For this week only, I need to move the day of the tor dev meeting to **Thursday** at 1330 UTC (8:30 am Eastern) -- my kid has a doctor's checkup. yrs, -- Nick

1 0