April 2015 - tor-dev - lists.torproject.org

Should popularity-hiding be a security property of hidden services?
by George Kadianakis 16 Apr '15

16 Apr '15

Hello, Tor hidden services are meant to primarily provide server anonymity but they also provide various other properties. For example, their addresses are self-authenticated and their connections punch NAT. This post is about another property, which is that Tor does not reveal the popularity of a hidden service by default. That is, you can't easily get the user count of a specific hidden service. This is not that surprising to hidden service operators, since that's also how the normal Internet works. In the normal Internet, someone cannot learn the user count of an IP or website, except if they are the operator or they control DNS or the site publishes analytics. Over the past years, people have suggested various features that would provide us with interesting information and optimizations but would have the side effect of revealing the user count of hidden services (or only of popular hidden services) to the public. Some examples of such popularity-leaking features: - Hidden Services dynamically set their number of Introduction Points depending on their popularity. Basically, they self evaluate their popularity, and use a formula to decide the number of Introduction Points between 3 and 10. For a while, we thought that this formula does not work properly (#4862, #8950), but we recently discovered that it seems to be working in some manner. While an interesting and useful feature on its own, it has the side effect that it leaks how popular your hidden service is. Since a hidden service publishes a descriptor every hour, you can monitor hourly usage patterns of hidden services. Of course, you can't get the exact user count, but you might be able to get rough approximate numbers (we still haven't analyzed the formula enough to know *exactly* how much). - During our recent work on hidden service statistics [0] people have suggested to gather statistics that would get us closer to learn the number of hidden service users [1]. The suggested way to do so is to have HSDirs or introduction points count the total number of introductions or descriptor fetches and publish that number in their extra-info descriptor. Since given a hidden service address you can easily learn its HSDirs or IPs, it should be possible to map those statistics to specific hidden services, which would leak their popularity (more on this later). There might be more examples that I'm missing, but this should be enough to demonstrate the leaks. For the rest of this post, I will be presenting various arguments for and against leaking popularity. Disclaimer: I still am not 100% decided here, but I lean heavily towards the "popularity is private information and we should not reveal it if we can help it" camp, or maybe in the "there needs to be very concrete positive outcomes before even considering leaking popularity". Hence, my arguments will be obviously biased towards the negatives of leaking popularity. I invite someone from the opposite camp to articulate better arguments for why popularity-hiding is something worth sacrificing. == Arguments for leaking popularity and reaping its benefits == Here are a few arguments that people use to shrug off popularity-hiding. I can relate to some of them, but I find the reasoning of some others funny or even dangerous. - "If we don't care about leaking popularity we can get useful statistics" Indeed, we've dismissed various statistics that we could collect because we were afraid that they would leak the popularity of hidden services. If we didn't have this fear, we would have a better idea on how much usage hidden services see, or whether people are conducting DoS attacks on hidden services. - "If we don't care about leaking popularity we can get nice optimizations" As an example, the dynamic IP calculation is one of those optimizations. I'm not aware of other optimizations, but I bet that we can think of a few more if we completely remove popularity-hiding from our threat model. Also, people have claimed that more statistics would reveal more optimizations that we could do. - "Popularity-hiding is just a side-effect of the Tor protocol, and not a stated security goal" People have claimed that popularity-hiding is not a stated security goal of hidden services, or that the name "hidden services" does not imply popularity-hiding in any way. - "There are no realistic attacks that could happen from leaking popularity" People have claimed that popularity is just a curiosity, and nothing bad can come from leaking it. They say that protecting popularity does not offer security against realistic or dangerous attacks. Other people claim that popularity-revealing attack vectors are too noisy and contain too much random data, hence it's hard to get targetted popularity values out of them. They say that it might only be possible for very popular hidden services, or for unlikely edge cases. - "There are probably other ways to reveal popularity. You can't fix them all" That's actually a big fear of mine. That we are nitpicking about 2-3 popularity revealing vectors, while there are hundreds more currently open. See #8742 for example, but I bet there are more vectors that we need to think about. == Arguments for protecting popularity == And here are arguments that make me believe that popularity is something that should be protected. - Popularity attracts attention Anonymity likes uniformity, but popularity attracts attention. There are literally infinite possible use cases where a hidden service wants to be public and still not attract attention. However, since the above argument has not been particularly successful and only attack demonstrations will persuade a true skeptic's mind, here is an attack scenario: Try hard to imagine a dystopian future where authorities are tracking down and hacking activist websites. They just received a big list of hidden services, the result of a messy interrogation, but they are all locked. Their hackers can hack some of them but not all. Not much time before revolution, end of dystopian future and happiness for all humanity. The dictator needs to decide which hidden services to hack to stop the revolution. Which?? With popularity being public , they can get the popularity of the biggest ones and target those first. - Popularity can be used to find patterns in group movements now and in the past. Even though you can't track specific users using popularity, you can still track group of users. Also, these statistics are forever: even if you didn't care about a group of users in the past, but you start caring about them now, you can still look back and see their development over time. Here is an attack scenario: Imagine a community that practices very dangerous urban climbing [2]. Imagine thousands of friends climbing away in happiness from all over the world, Imagine now that this community splinters in other smaller communities, if you monitor their popularity, it will be possible for you to observe the movement of that subculture. As a further point, imagine now that dystopian future comes and very dangerous urban climbing gets outlawed. The police catches an urban climber in New London and gets a list of hidden services from her. They can then check _historically_ how many users those hidden services had. They can basically notice all the trends of the urban climbing scene in the past years. Creepy, no? As you probably well know, anonymity is not a binary option. It's not like you are not either super anonymous, or not. It's more of a fuzzy variable that depends on many things. OPSEC is a big part of anonymity, and it seems to me that popularity has OPSEC consequences. - Statistics noise will get reduced. Attacks only get better. In the statistics we were talking about, each HSDir would reveal the number of descriptor fetches it received over the past day. We know that each HSDir serves about 150 hidden services, which means that the final value in the end will contain the popularity of 150 hidden services in one number. This is expected to be extremely noisy, and I think that's one of the main hopes of people who don't care about popularity hiding. That allows them to claim that popularity will only be leaked for very popular hidden services. While this indeed seems reasonable, my main intuition is that attacks can only get better. Here are some ways that noise can be reduced. I will focus on the HSDir case, but same arguments apply to other suggested statistics like number of introductions per IP. -- It's still early in the hidden services scene, so not many services get lots of traffic. I imagine that many of those 150 hidden services are going to be very inactive, and not provide much noise. -- Hidden services publish hidden service descriptors to 6 HSDirs. This means that every day you will learn 6 noisy values for your target hidden service, not just 1. It's easier to remove noise that way. -- Also, those 5 irrelevant hidden services that provide the noise will publish themselves to 6 HSDirs. Applying the same logic as above, you might be able to learn information about the noise, which makes it easier to remove. In a way, you can put all the statistics measurements in a big system of equations, and start solving it to reduce noise in the equation you are interested in. -- Think of crazy edge cases. Maybe an introduction point is very weak and unlikely to be picked and only got 10 HSes for a day. If one of them is the hidden service you are interested in, there is going to be much fewer noise than usual. -- There might be other techniques for reducing noise, by combining other statistics (like the number of hidden services per HSDir which is already a stat), or by influencing the statistics yourself (like Aaron's attack on the stats aggregation protocol [3]). What I'm trying to say here, is that if you thought that the urban climbing example was ridiculous because such a community cannot be big enough to be visible in noisy statistics, maybe by reducing noise you can actually make it distinguishable. - There are not that amazing benefits from ditching popularity-hiding. To be honest, I have not heard convincing enough arguments that would make me ditch popularity hiding. Some extra statistics or some small optimizations do not seem exciting enough to me. Please try harder. This could be a nice thread to demonstrate all the positive things that could happen if we ditch popularity-hiding. Also, there is a small difference here between the stats and the introduction point formula. The dynamic introduction point formula is something that we could disable by default, but also leave it as a configurable option for people who want to use it. That is, it will then be *the choice of the hidden service operator* whether he cares about popularity being hidden or not. With the statistics that have been proposed, you don't give any choice. You just do it for all hidden services forever. - Principle of least surprise Hidden service operators except that hidden services are at least as secure as the normal Internet plus more. On the normal Internet, popularity is private by default. Having this assumption violated on hidden services, might not be polite. - Popularity-hiding is crucial to maintain the deep sea security model of hidden services As I have mentioned in the past, some people think of the onion land as a very deep ocean. In some places of the ocean, you might be able to see some buoys (some more visible than others). To visit them, you need to wear your goggles and your snokrel, dive in and enter from underwater. This might not seem like a very concrete security model, but in any case popularity is not revealed at any point. The sea is opaque and you can't see the divers entering the hidden services. Anyway this post has grown to immense size, and I was really hoping it would be shorter. On a more practical note, over the next few weeks, we should decide what we want to do with the dynamic introduction point formula and whether we should keep it or not (#4862). My current intuition is that it should be disabled but also kept there as an option for people who want to enable it. In any case, I hope that this thread can stimulate discussion. Also, if you are a hidden service operator I'm curious to hear about whether you believe that popularity hiding is a security property that should be preserved if that's even possible. Cheers! [0]: https://blog.torproject.org/blog/some-statistics-about-onions [1]: https://lists.torproject.org/pipermail/tor-dev/2015-February/008247.html [2]: https://www.youtube.com/watch?v=kpS7vhvkIQM [3]: https://lists.torproject.org/pipermail/tor-dev/2015-March/008404.html

4 3

TOR SoP proposal: extending and improving TOR network anomaly detection
by Kibo Schaffer 15 Apr '15

15 Apr '15

Hi all, I'm applying for TOR Summer of Privacy. I'm curious if there are any developers here who are able and willing to become a mentor for this idea. I want to improve TOR's ability to detect anomalies such as sybil attacks, and make it easy to include other heuristics for other potential attacks. When a potential attack is detected, users and maintainers are notified (as necessary). There has been research and development with this field with TorDoctor, exitmap, and HoneyConnector. However, as far as I am aware, these projects could use some help being solidified and integrated into TOR. Please let me know what you think, if you want to suggest any improvements, and / or if you are a mentor interested in this idea. Cheers, Kibo [1] TorDoctor -- https://gitweb.torproject.org/doctor.git [2] exitmap / HoneyConnector -- http://www.cs.kau.se/philwint/spoiled_onions/

2 2

Tor Summer of Privacy deadline is soon!
by Damian Johnson 15 Apr '15

15 Apr '15

Hi all, just a quick friendly heads up that the deadline is coming up soon. If you've been thinking of making an application then now is the time! Application deadline for students is April 17th (end-of-day everywhere [1]). NO LATE APPLICATIONS WILL BE CONSIDERED. We're sorry about needing to be strict on this, but we have a really tight deadline for making selection decisions. Late applications make the process harder, so the only fair thing is to not allow any exceptions. Cheers! -Damian [1] https://en.wikipedia.org/wiki/Anywhere_on_Earth

1 0

Re: [tor-dev] TSOP proposal
by Alden Page 15 Apr '15

15 Apr '15

Hey Remi, It's great to see someone else interested in adversarial stylometry! I have been working on a project quite similar to your proposal since January as my undergraduate thesis. I am considering applying for TSoP in order to fund the continued development of the project, but I think the mentor issue is a strong indication that we should focus our efforts on something more related to Tor. Since I have made considerable progress implementing the tool you described, perhaps we should instead combine our efforts? Rachel Greenstadt, whose work I am sure you are familiar with, also expressed interest in collaboration when I announced this project in December. You will find that I have implemented many of your proposed features in my project on GitHub. Near-term work involves making the project easier to install and implementing Writeprint feature set extractors. If you manage to get it running (for now, you will have to install dependencies manually), I would love to hear some feedback from you! Until then, see my thesis. Contribute at: www.github.com/pagea/unstyle A preliminary version of my thesis can be found below. I will provide a more permanent URL on GitHub in the future. https://www.scribd.com/doc/261957061/Unstyle-A-Tool-for-Evading-Authorship-… Full disclosure: I am applying to TSoP this year, and I am still strongly considering proposing the continued development of Unstyle, in addition to a few other potential applications. Cheers, Alden Page

1 0

Summer of Privacy applicant, Multi-threading
by Kim Reece 15 Apr '15

15 Apr '15

Hello, I see several folks doing brief intros here so I'll do so as well. Most of my application is in already, just finishing up the patch. I'm a Mathematics undergrad, 3rd year come Fall. I've used multi-threading in coursework and in an internship I did at JPL/NASA. I want to expand the multi-threading in Tor this summer. For my patch I'm working on #3569 (Refactor socks parsing). It won't be a complete implementation of that bug this week, but I aim to complete the first part of it, ie. the refactoring internal to the function to expose the state model clearly in code structure. To that end I have one quick code style question: Is clarity preferred over the (very short) body of a conditional only occurring in one place? I'm encountering at least once instance where the choice is between repeating a (very short) code block, thereby introducing the potential that one section of it may in the future be edited without the other receiving the same edit (even though they're right next to each other), or retaining a gnarly conditional that took a truth table for me to be sure I had the right end of. If code occurring exactly once is a priority, the gnarliness can be repaired with an explanatory comment, but I'd like to know which is preferred. Thanks, -- Kim

1 0

Using consensus package versions for third party software
by John Brooks 15 Apr '15

15 Apr '15

Proposal 227 added a method for putting non-little-t-tor package versions and digests in the consensus, intended to authenticate Tor Browser updates. This is done in tor 0.2.6, although it’s not yet in use by Tor Browser or the consensus. I propose using this feature to notify Ricochet[1] users of software updates. My reasoning is: - It’s vital to the security of Ricochet’s users that they’re quickly notified of version updates - Tor is the only infrastructure Ricochet uses; adding an “update server” would harm its model - The consensus-based method is better than anything we can independently do - I think Ricochet is an ideal case of where this work can be useful The benefits from my side are obvious, but it’s less clear if this is appropriate for Tor. Ricochet users[2] are an extremely small portion of Tor’s users, but this would be in every consensus. Some directory authorities would have to volunteer to keep those votes up to date. I’m interested in feedback from the community in general, and especially from directory authorities on whether this is something they’d be interested in supporting. Of course, even if people think this is a good idea, the burden is on me to convince DAs to lend their time. Thanks, - John [1] https://ricochet.im/ [2] It’s part of the design that I can’t accurately count users. My best guess is “low hundreds, and growing”.

1 0

Fwd: Orbot v15 RC3... now with x86/ATOM-power!
by Nathan Freitas 14 Apr '15

14 Apr '15

----- Original message ----- From: Nathan of Guardian <nathan(a)guardianproject.info> To: guardian-dev(a)lists.mayfirst.org Subject: Orbot v15 RC3... now with x86/ATOM-power! Date: Mon, 13 Apr 2015 09:55:44 -0400 Making some tiny changes to external/Makefile and jni/Application.mk means we can now build the tor, polipo and xtables binaries, along with the tun2socks.so for any necessary chip architecture. I've added support for x86 in this release, which only added a few MB to the overall APK. Somehow, devices with Intel ATOM chips were able to run Orbot v14 releases, even though we didn't target x86. I think ATOM has some sort of light ARM compatibility mode, but we broke that in the update for v15... anyhow, this is better, as it is officially supported and tested. One interesting issue is that GoLang 1.4.1, which we are using to cross-compile the Meek and Obfs4 pluggable transports to Android, only supports targeting Android ARM for right now... I assume that will change soon, but if not, we may have to add Obfsclient back into Orbot for supporting x86 devices. The most popular x86/ATOM devices we have heard people complaining about seem to be Galaxy Tab 3 and the Asus Zenphone and Padphone devices. Anyhow... details below... **** APK: https://guardianproject.info/releases/Orbot-v15.0.0-RC-3.apk Sig: https://guardianproject.info/releases/Orbot-v15.0.0-RC-3.apk.asc Tag: https://gitweb.torproject.org/orbot.git/tag/?id=15.0.0-RC-3 /** 15.0.0 RC 3 / 13-Apr-2015 / b941a1c7d5588db7efd9fab672ec518a69f76f84 **/ b941a1c show warning about bridge limits on Intel x86/ATOM devices 325ca1f only ARM chips can support the new Obfs4, Meek bridges so hide the UI options that promote them, and just request standard br i 3c6f173 make buttons not resize weirdly with long strings 421764b make socksbypass local port random 4ab1854 update resource installer to handle different architecture e7a7d8c support variable arch builds to support x86 move asset builds to /assets folder 74deb39 support building tun2socks for x86 933b2e9 Small VPN and socket monitoring related fixes -- Nathan of Guardian nathan(a)guardianproject.info

3 2

Re: [tor-dev] Draft of proposal "Direct Onion Services: Fast-but-not-hidden services"
by teor 13 Apr '15

13 Apr '15

> Date: Fri, 10 Apr 2015 15:47:23 +0300 > From: George Kadianakis <desnacked(a)riseup.net> > > David Goulet <dgoulet(a)ev0ke.net> writes: > >>> On 09 Apr (21:58:49), George Kadianakis wrote: >>> Hello, >>> >>> I inline a proposal for Direct Onion Services. It's heavily based on >>> the "encrypted services" proposal of Roger, but I refined it, made it >>> more proposaley and enabled a few more optimizations. You can find the >>> original piece here: >>> https://gitweb.torproject.org/torspec.git/tree/proposals/ideas/xxx-encrypte… >>> >>> Feedback would be greatly appreciated. >>> [snip] > >>> >>> - We really really need a better name for this feature. I decided to >>> go with "Direct Onion Services" which is the one that the NRL people >>> suggested here: >>> https://lists.torproject.org/pipermail/tor-dev/2015-February/008256.html >>> I'm not super happy with it, but I can't think of better ones myself. >>> Here are some candidates: >>> 1 Way Anonymous Hidden Services >>> Fast Onion Service >>> Short-circuited Onion Services >>> Fast-but-not-hidden Services >> >> "Fast Onion-Space Service" was proposed by pfmbot on #tor-dev. I like it >> because it describes pretty much what it is and the acronym is FOSS. :) > > It's not clear to me why acronym collisions are a plus :) > >>> >>> - We also need a better torrc option. If the name of this feature does >>> not portray its dangers, then the torrc option should be a bit >>> terrifying. That's why I went 'DangerousDirectOnionService' which I >>> actually don't like at all. BTW, it's unclear whether this mailing >>> list is the best place to brainstorm for names. >> >> This one depends on the name quite heavily but I don't think we should >> have a scary name. Scary name seems to me they will simply bring more >> questions without context of the actual feature. What is "Dangerous" >> about this, why is it in Tor at all?... This could simply be resolved by >> clear and thorough documentation of the risks/benefits of the options. >> >> For instance, we could *not* put it in the default torrc file and thus >> will only be in the man page with *good* documentation. Like you said >> also in the proposal, a big warning is very important at boot. >> >> If it's off by default and not present in the torrc file, there are no >> reasons why someone would enable this just because it's says >> "DirectOnionServiceEnable". (Ok I might be an optimist but still, we are >> not that responsible for reckless HS operator ;) > > Yes, I agree that the 'Dangerous' prefix is stupid. I think I'm OK > with changing the proposal to use 'DirectOnionServiceEnable' but I would > prefer if it's a bit more alarming. >> >> [snip] I've been hesitant to weigh in on the naming conversation for hidden services. But I am concerned about the issues raised in the previous few emails on the topic. Matt @ Speak Freely has a strong point about acronym collisions - they only serve to confuse users and dilute the search space. And usability becomes a security issue as soon as names are confusing to users. Others have also made important points about how we name things in general. So I'd like to propose some general naming principles, mostly distilled from recent conversations about names: 1. Avoid acronyms that collide with popular acronyms, particularly those in the security / software / technology spaces. 2. Names should be kept consistent across the entire Tor code base and ecosystem. 3. When selecting option, feature, and concept names: A) Names should describe the most important user-visible impact of an option, feature, or concept. B) If the feature has a serious impact on anonymity (or other typical user assumptions), the name should indicate that. C) Names should avoid specific implementation details, as these may change. D) Names should avoid giving relative opinions on performance, security, or privacy impacts, as these are often dependent on context. E) Project names are an exception to principle 3, and should have abstract names, or descriptive acronyms. My evaluation of the suggestions around hidden services / onion services based on these principles is: > Onion Service Collides with OS, or Operating System, violating principle 1: no collisions. If we decide to disregard this acronym collision, and accept the usability impact, every option and the entire manual page should be changed in the same release, to satisfy principle 2. This would involve a transition period in which both HS and OS options would work. (We should also consider the usability of options with OS in the name before making the switch.) > Direct Onion Service > Dangerous Direct Onion Service As it's already been noted, these collide with DOS and DDOS (1), and don't describe which end is direct (3A & 3B). > Fast Onion Service > Short-circuited Onion Services I'm concerned about the collisions with OS implied by these acronyms (1). Also, the first provides a performance goal (3D), and the second, an implementation detail (3C). Neither describes the most important user-visible behaviour - lack of server anonymity (3B). > Fast-but-not-hidden Services This seems to combine a performance goal (3D), with a description of the impact of the service. Aren't we better to address the speed / anonymity trade off in the manual and other documentation? > 1 Way Anonymous Hidden Services This is descriptive (3A & 3B), but which end is anonymous? Why not state that it's the client: Client Anonymous Hidden Services Or, even better, state that the server is non-anonymous: Exposed Server Hidden Services Public Server Hidden Services Yielding option names like the following: ExposedServerHiddenServiceEnable PublicServerHiddenServiceEnable But I'd be happy with any feature and option names that make it obvious that the hidden service server has reduced anonymity compared to standard hidden services. teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR C3C57B23 349825DE 929A1DEF C3531C25 A32287ED

2 1

Re: [tor-dev] Fwd: Orbot v15 RC3... now with x86/ATOM-power!
by Nathan Freitas 13 Apr '15

13 Apr '15

On Mon, Apr 13, 2015, at 02:05 PM, Daniel Martí wrote: > Possibly stupid question, but wouldn't a static linux/x86 binary work > just fine as long as you're executing it directly? As far as I know the > Android port is just for all the bindings involved in e.g. writing a > game in Go that does OpenGL calls back to Android. There a few more changes in there, as well, such as how DNS works on Android, and few other small underlying differences. > > For example, I found this bit in a blog post online: > > > Copy myexectuable under your Android project’s ./libs/armeabi/ folder. > > If you’re targeting one of the rare x86 Android devices, build to > > x86/Linux and copy the resulting binary to ./libs/x86 (and yes, you > > can include both in a single APK, and it will deploy whichever one is > > appropriate). > > But then this would mean that we wouldn't need android/arm either, since > we could just use linux/arm instead :) We were previously using linux/arm for the PT compiling, and it mostly worked, but it turns out there are enough small differences in the Android API from Linux, that it does matter to target Android specifically. This is similar to why we need the Android NDK for tor cross-compile, and not just a generix linux ARM build. +n

2 1

Re: [tor-dev] Fwd: Orbot v15 RC3... now with x86/ATOM-power!
by Nathan Freitas 13 Apr '15

13 Apr '15

On Mon, Apr 13, 2015, at 12:57 PM, Yawning Angel wrote: > On Mon, 13 Apr 2015 10:14:43 -0400 > Nathan Freitas <nathan(a)freitas.net> wrote: > > One interesting issue is that GoLang 1.4.1, which we are using to > > cross-compile the Meek and Obfs4 pluggable transports to Android, only > > supports targeting Android ARM for right now... I assume that will > > change soon, but if not, we may have to add Obfsclient back into Orbot > > for supporting x86 devices. > > Hmm, maybe I should add obfs4 support to obfsclient. I have code for > all of the crypto I would need to add. Let me reach out to the GoLang Android community and see what the blocker is... I was thinking the same thing though, it wouldn't be hard to still support Obfsclient, along with the Go-based PT's. Thanks!

2 1