- tor-dev - lists.torproject.org

Re: [tor-dev] Should we disable the collection of some stats published in extra-infos?
by Karsten Loesing 28 Mar '16

28 Mar '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 15/01/16 23:00, Rob Jansen wrote: > Hello, Hi Rob, I'm moving this discussion from metrics-team@ to tor-dev@, because I think it's relevant for little-t-tor devs who are not subscribed to metrics-team@. Hope you don't mind. > I was recently reviewing the statistics that Tor allows relays to > collect and report to the dir servers [1], which then get published > in extra-info documents [2]. Most of this can be enabled by simply > setting a torrc option. There are quite a few statistics that I > feel should not be collected. I'm wondering if the original purpose > for collecting many of these statistics still exists, and if we > still feel that the privacy compromises that were made when the > collection was implemented are still valid in most cases. > > Here are the stats I am most worried about, and why: > > [unique ips per country code] *-ips (there are many of these, e.g. > "entry-ips") Usually this involves storing individual user IP > addresses in memory (in order to track uniqueness) over some period > of time (usually 24 hours), sometimes for longer than the user > would have otherwise been known to Tor (if a user's session is 1 > hour, Tor could remember the IP for at most 23 additional hours). > This is reported, e.g., per entry; there are many cases in the data > where it is very likely that only one user is connecting to a guard > from a given country (because it is rounded up to 8). Users in > small countries have the greatest risk (intersection attacks become > really easy). I agree that might just lose these statistics. We used them in the past as first approximation to counting users, but obviously that only works as long as clients only connect to a single relay. The only place where we're still using them is in a workaround for estimating bridge users. See #15469 for more details and #8786 for something we'd have to implement before taking these statistics out. > [exit statistics by port number] exit-kibibytes-written > exit-kibibytes-read exit-streams-opened Tor is classifying its > traffic into ports, which could uniquely identify the application > being used by the client. They also track bandwidth usage per port > (and per exit); again, this is bad for those using a random or > unique looking ports (that a given exit does not see very often) > because it could be used to create a fingerprint. Intersection > attacks become easier with this information. Agreed, I can see us dropping these statistics, too. We're currently not using them. But also see my suggestion below. > The less problematic stats: > > [circuit-based cell statistics] cell-processed-cells > cell-queued-cells cell-time-in-queue cell-circuits-per-decile This > provides queue timings and number of cells being processed at a > relay. The number of cells can be used to compute bandwidth of > circuits. It may be possible to launch some attacks that create > several circuits with the intent of moving which decile buckets > some legitimate circuits get placed into, but this is less > worrisome of an attack than the others. I'm less worried about this one. But, suggestion below. > Should Tor still be collecting these things? Should Tor disable the > collection of these statistics until we have a more > privacy-preserving way to collect and aggregate them? > > The good news is that privacy-preserving techniques exist that can > reduce information leakage. I'm developing a tool based on the > secret-sharing variant of PrivEx [3] to collect some of these types > of statistics while providing privacy guarantees. We are currently > using it to collect only those stats that are useful for producing > Tor traffic models. A great advantage of this tool is that the > various counters that we store during the collection phase get > noise added and are randomized during initialization; only the > aggregates are ever known and revealed by the aggregation server, > limiting the information that is lost if a relay is compromised. > This is a large improvement over the current collection method, > which only adds noise before publication and reveals statistics on > a per-relay basis. Suggestion: How about we evaluate these statistics published by relays in the past years to see if there are other benefits or risks we didn't think of, and then we decide whether to leave them in, modify them, or take them out? The reason is that I'd want to avoid removing this code only to realize shortly after that we overlooked a good reason for keeping it. These statistics are being collected for years now, and it might take another year or so for relays to upgrade to stop collecting them. So what's another month. Thanks for (re-)starting this discussion! > All the best, Rob All the best, Karsten > [1] https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt [2] > https://collector.torproject.org/recent/relay-descriptors/extra-infos/ > > [3] www.cypherpunks.ca/~iang/pubs/privex-ccs14.pdf > > > > _______________________________________________ metrics-team > mailing list metrics-team(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team > -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWnffFAAoJEJD5dJfVqbCrSREH/1G3nZDwC7a8FGFEySgQX3MZ n1dlMcylDU6kypgGMEYs2KmMRwhNYmCVFn6fRJgrFN/KgJ0oZJG1pRcglnIFwNwH 5VUWDZp3+frhY5jBbJ3JXqA4Exhi/0mUYCTRwfcFQ/JOyzjlcFbLRW/rkqFHn62H wLidaVeagiJ+TI/T8zgwtqQjTDSZrZPpmlxvlO57D1bGhW1ZPJVbOeUzuudhNpRS WVu//MH51juCzcML32MeiMV6wWYYOm1irKZ8lZHVCPbsL98qoh3ewtO6fb62THIa sWcQZlwMQdcC0+509PmMeYwkRn+40MUwkk84/glyP57dfpgtOwzZCohL3yQRQ1s= =F6F+ -----END PGP SIGNATURE-----

2 4

guide to start developing tor
by RAJAT MODI 28 Mar '16

28 Mar '16

hi i am new to tor and want to contribute to tis development .i have already browsed the volunterr page on your website .However i am unable to decide from where should i start ,Kindly guide me for the same . -- RAJAT MODI

1 0

Remember: Network Team/tor core dev meeting Monday at new time!
by Nick Mathewson 28 Mar '16

28 Mar '16

Hello! Tomorrow on Monday, we will be having our meeting on the new time: 1700 UTC (1pm eastern, 10am Pacific). The patch party will be Tuesday at 2300 UTC. If you like hacking Tor, set an alarm and stop on by! Peace, -- Nick

1 0

Sibyls: introducing AS-level sign-up rate limits, per relay caps, family-level caps
by nusenu 27 Mar '16

27 Mar '16

Hi, there is a strict limit on the number of relays that one can run on a single IP address to make Sybil attacks harder, but there are no limits on sign-up rates. Such sign-up rate limits would prevent many of the known and simple Sibyls of the past months and probably reduce the manual actions required by dir auth ops to mitigate them (but I've little inside on that part). In the past 8 months the proposed AS sign-up rate limit thresholds (see below) triggered on ~27 events [some-examples]. I consider all of them to be true-positives, but 15 (vs total: >750) likely unrelated relays signed up on the same ASes during these events. AS-level sign-up rate limit thresholds --------------------------------------- max new relays per 24 hours per AS per OS family: 9 (default); value for OVH/ONLINE/Digital Ocean ASes: 14 I deduced to these threshold values by watching the past 8 months of OrNetRadar emails while keeping false-positives at a minimum (at the price of more false-negatives). How does a well behaving operator (aka Brian) add more than 9 relays per AS per day? All relays in an effective family should be treated as "1 new relay". (In my opinion a reason to keep the family [0] functionality and implement prop242). denial of service risk To prevent trivial dos attacks where an attacker with a single IP generates several new relay fingerprints until the entire AS is blocked from adding new relays for a few hours these relays should come from distinct IP addresses. Pros: + automatically enforced + less manual action required by dir auth ops (maybe some dir auths can commend on that.) + address more (and smaller) Sibyls (I assume that many small Sibyls are currently ignored due to risk/effort trade-offs) Cons: - maxmind AS-IP mapping db becomes crucial and needs to be in sync across dir auths - false-positives - non-dir auths will not notice if an AS run into this limit unless they publish some kind of transparency reports - dir auths must keep more state (not sure if they know all existing FPs of the past already) Feasibility Initially the idea was to provide immediate feedback to the relay that tries to publish its descriptor to the dir auth via the HTTP response (an important point), but since descriptors are not uploaded to a single central dir auth that does not work I guess. So before thinking on how one would actually achieve that or something similar I wanted to gather the general opinion about something like this. single relay caps ------------------ 250000 consensus weight (to prevent [1]) family-level caps ------------------ max cw fraction: 2.5% [2] max. exit probability: 10% [3] guard probability: 5% [4] AS-level caps: --------------- max cw fraction/guard/exit probability: 15% [5] AS12876 (ONLINE SAS) and AS16276 (OVH) are already past that value, so if you do not want to change the game for them you would have to set it to ~20%. [0] https://trac.torproject.org/projects/tor/ticket/6676 [1] https://lists.torproject.org/pipermail/tor-talk/2015-December/039696.html https://trac.torproject.org/projects/tor/ticket/17810#comment:8 [2] https://raw.githubusercontent.com/ornetstats/stats/master/o/main_operators_… [3] https://raw.githubusercontent.com/ornetstats/stats/master/o/main_exit_opera… [4] https://raw.githubusercontent.com/ornetstats/stats/master/o/main_guard_oper… [5] https://compass.torproject.org [some-examples] http://article.gmane.org/gmane.network.onion-routing.ornetradar/1135 http://article.gmane.org/gmane.network.onion-routing.ornetradar/1132 http://article.gmane.org/gmane.network.onion-routing.ornetradar/1100 http://article.gmane.org/gmane.network.onion-routing.ornetradar/1030 http://article.gmane.org/gmane.network.onion-routing.ornetradar/939 http://article.gmane.org/gmane.network.onion-routing.ornetradar/943 http://article.gmane.org/gmane.network.onion-routing.ornetradar/898 http://article.gmane.org/gmane.network.onion-routing.ornetradar/856 http://article.gmane.org/gmane.network.onion-routing.ornetradar/813

1 1

[GSoC 2016] Roster - Gamification for Tor Operators (Sean Saito)
by Sean Saito 26 Mar '16

26 Mar '16

Hi! I have finished and submitted my final proposal for GSoC yesterday. This summer I would like to complete Roster, a Tor Project initiative I’ve been working on since last year. It is basically a social incentivization scheme to gamify operating a Tor relay. All the details are included in the Google doc, and, while the deadline for edits has passed, I would appreciate any comments and feedback. https://docs.google.com/document/d/1wOBUhOPAAWTLfQIt4ChKixLMgIrZlO3YBQ-eixh… Thanks a lot! Best, Sean

1 0

Tor 0.2.8 and Fallback Directory Mirrors
by Tim Wilson-Brown - teor 26 Mar '16

26 Mar '16

Dear Tor App Developers, Just a heads' up that tor 0.2.8 includes a fallback directory mirrors feature, where tor clients bootstrap from a set of hard-coded long-lived directory mirrors. This makes bootstrap more reliable, and makes it harder to block tor. The wiki has an entry with more details. [0] We're currently trying to finalise a list of around 100 fallback directory mirrors by the time tor 0.2.8-stable is released. [1] If we have this many mirrors, I am confident we won't overwhelm any of the mirrors when an app like Orbot, Tor Browser, or Tails switches to tor 0.2.8. (Are there any other tor apps with millions of users that I've missed?) If this feature does cause problems, or if your app needs to bootstrap only from the authorities (Tails time syncing? [2]), the fallback directory mirrors feature can be disabled using the tor config option "UseDefaultFallbackDirs 0". Tim [0]: https://trac.torproject.org/projects/tor/wiki/doc/FallbackDirectoryMirrors <https://trac.torproject.org/projects/tor/wiki/doc/FallbackDirectoryMirrors> [1]: https://trac.torproject.org/projects/tor/ticket/17158 <https://trac.torproject.org/projects/tor/ticket/17158> [2]: https://tails.boum.org/contribute/design/Time_syncing/ <https://tails.boum.org/contribute/design/Time_syncing/> Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

2 2

Much-revised draft, RFC: removing current obsolete clients from the network
by Nick Mathewson 26 Mar '16

26 Mar '16

[This is a significantly revised version of the last version of this proposal draft, sent here for comment. The last version of this draft was https://lists.torproject.org/pipermail/tor-dev/2015-September/009587.html . This is more or less a complete rewrite, after I convinced myself that the original ideas wouldn't work. Part of the functionality of that old draft has been subsumed in proposal 264.] 1. Introduction Frequently, we find that very old versions of Tor should no longer be supported on the network. To remove relays is easy enough: we simply update the directory authorities to stop listing relays that advertise versions that are too old. But to disable clients is harder. In another proposal I describe a system for letting future clients go gracefully obsolete. This proposal explains how we can safely disable the obsolete clients we have today (and all other client versions of Tor to date, assuming that they will someday become obsolete). 1.1. Why disable clients? * Security. Anybody who hasn't updated their Tor client in 5 years is probably vulnerable to who-knows-what attacks. They aren't likely to get much anonymity either. * Withstand zombie installations. Some Tors out there were once configured to start-on-boot systems that are now unmaintained. (See 1.4 below.) They put needless load on the network, and help nobody. * Be able to remove backward-compatibility code. Currently, Tor supports some truly ancient protocols in order to avoid breaking ancient versions or Tor. This code needs to be maintained and tested. Some of it depends on undocumented or deprecated or non-portable OpenSSL features, and makes it hard to produce a conforming Tor server implementation. * Make it easier to write a conforming Tor relay. If a Tor relay needs to support every Tor client back through the beginning of time, that makes it harder to develop and test compatible implementations. 1.2. Is this dangerous? I don't think so. This proposal describes a way to make older clients gracefully disconnect from the network only when a majority of authorities agree that they should. A majority of authorities already have the ability to inflict arbitrary degrees of sabotage on the consensus document. 1.3. History The earliest versions of Tor checked the recommended-versions field in the directory to see whether they should keep running. If they saw that their version wasn't recommended, they'd shut down. There was an "IgnoreVersion" option that let you keep running anyway. Later, around 2004, the rule changed to "shut down if the version is _obsolete_", where obsolete was defined as "not recommended, and older than a version that is recommended." In 0.1.1.7-alpha, we made obsolete versions only produce a warning, and removed IgnoreVersion. (See 3ac34ae3293ceb0f2b8c49.) We have still disabled old tor versions. With Tor 0.2.0.5-alpha, we disabled Tor versions before 0.1.1.6-alpha by having the v1 authorities begin publishing empty directories only. In version 0.2.5.2-alpha, we completely removed support for the v2 directory protocol used before Tor 0.2.0; there are no longer any v2 authorities on the network. Tor versions before 0.2.1 will currently not progress past fetching an initial directory, because they believe in a number of directory authority identity keys that no longer sign the directory. Tor versions before 0.2.4 are (lightly) throttled in multihop circuit creation, because we prioritize ntor CREATE cells over TAP ones when under load. 1.4. The big problem: slow zombies and fast zombies It would be easy enough to 'disable' old clients by simply removing server support for the obsolete protocols that they use. But there's a problem with that approach: what will the clients do when they fail to make connections, or to extend circuits, or whatever else they are no longer able to do? * Ideally, I'd like such clients to stop functioning _quietly_. If they stop contacting the network, that would be best. * Next best would be if these clients contacted the network only occasionally and at different times. I'll call these clients "slow zombies". * Worse would be if the clients contact the network frequently, over and over. I'll call these clients "fast zombies". They would be at their worst when they focus on authorities, or when they act in synchrony to all strike at once. One goal of this proposal is to ensure that future clients to not become zombies at all; and that ancient clients become slow zombies at worst. 2. Some ideas that don't work. 2.1. Dropping connections based on link protocols. Tor versions before before 0.2.3.6-alpha use a renegotiation-based handshake instead of our current handshake. We could detect these handshakes and close the connection at the relay side if the client attempts to renegotiate. I've tested these changes on versions maint-0.2.0 through maint-0.2.2. They result in zombies with the following behavior: The client contact each authority it knows about, attempting to make a one-hop directory connection. It fails, detects a failure, then reconnects more and more slowly ... but one hour later, it resets its connection schedule and starts again. In the steady state this appears to result in about two connections per client per authority per hour. That is probably too many. (Most authorities would be affected: of the authorities that existed in 0.2.2, gabelmoo has moved and turtles has shut down. The authorities Faravahar and longclaw are new. The authorities moria1, tor26, dizum, dannenberg, urras, maatuska and maatuska would all get hit here.) (We could simply remove the renegotiation-detection code entirely, and reply to all connections with an immediate VERSIONS cell. The behavior would probably be the same, though.) If we throttled connections rather than closing them, we'd only get one connnection per authority per hour, but authorities would have to keep open a potentially huge number of sockets. 2.2. Blocking circuit creation under certain circumstances In tor 0.2.5.1-alpha, we began ignoring the UseNTorHandshake option, and always preferring the ntor handshake where available. Unfortunately, we can't simply drop all TAP handshakes, since clients and relays can still use them in the hidden service protocol. But we could detect these versions by: Looking for use of a TAP handshake from an IP not associated with with any known relay, or on a connection where the client did not authenticate. (This could be from a bridge, but clients don't build circuits that go to an IntroPoint or RendPoint directly after a bridge.) This would still result in clients not having directories, however, and retrying once an hours. 3. Ideas that might work 3.1. Move all authorities to new ports We could have each authority known to older clients start listening for connections at a new port P. We'd forward the old port to the new port. Once sufficiently many clients were using the new ports, we could disable the forwarding. This would result in the old clients turning into zombies as above, but they would only be scrabbling at nonexistent ports, causing less load on the authorities. [This proposal would probably be easiest to implement.] 3.2. Start disabling old link protocols on relays We could have new relays start dropping support for the old link protocols, while maintaining support on the authorities and older relays. The result here would be a degradation of older client performance over time. They'd still behave zombieishly if the authorities dropped support, however. 3.3. Changing the consensus format. We could allow 'f' (short for "flag") as a synonym for 's' in consensus documents. Later, if we want to disable all Tor versions before today, we can change the consensus algorithm so that the consensus (or perhaps only the microdesc consensus) is spelled with 'f' lines instead of 'f' lines. This will create a consensus which older clients and relays parse as having all nodes down, which will make them not connect to the network at all. We could similarly replace "r" with "n", or replace Running with Online, or so on. In doing this, we could also rename fresh-until and valid-until, so that new clients would have the real expiration date, and old clients would see "this consensus never expires". This would prevent them from downloading new consensuses. [This proposal would result in the quietest shutdown.]

7 8

[GSoC '16] Exitmap project - Final Proposal
by Mridul Malpotra 25 Mar '16

25 Mar '16

Hey everyone! I have completed my proposal on the project Exitmap improvements. Sorry for submitting it late. Link for suggestions: https://docs.google.com/document/d/1AqjH3AsBbfesJg36SZg1frPQsdpAnJmzH8k1-G6… Some points that have changed from the previous discussion: 1. Telnet/SNMP sniffing attacks would not be appropriate for Exitmap as it is an active attack scanner compared to say HoneyConnector etc. hence it has not been included. 2. I have added most of the points that I had suggested above with tweaks as advised by Dr. Winter. I will be submitting the proposal in an hour. Thank you for your time. Sincerely, -- Mridul Malpotra Undergrad @ IIIT-Delhi <http://www.iiitd.ac.in/> mridulmalpotra.github.io ============================= PGP keyID: 0xb716e33ab6d0a653 <http://pgp.mit.edu/pks/lookup?op=get&search=0xB716E33AB6D0A653>

1 1

Re: [tor-dev] GSoC proposal: Panopticlick
by Akito Ono 25 Mar '16

25 Mar '16

Hi Georg, >Thanks. What does "check fingerprint's uniqueness" entail? Is it just a >"Unique? - Yes/No"-thing? If not, what are your ideas in this regard? I think it is not a "Yes/No"-thing. My idea is that the uniqueness of fingerprint is evaluated in each element relatively in recorded data and total uniqueness of a fingerprint is calculated by those ratios. I added my ideas to my proposal. Best regards, Akito

1 0

Notes from the prop259 proposal reading group
by George Kadianakis 25 Mar '16

25 Mar '16

Hello, we had a meeting about proposal 259 "New Guard Selection Behaviour". You can see the logs here: http://meetbot.debian.net/tor-dev/2016/tor-dev.2016-03-23-15.01.log.txt Some notes: - The latest version of the proposal can be found here: https://lists.torproject.org/pipermail/tor-dev/2016-March/010625.html There is also some discussion with Tim that has been happening in that thread since yesterday. It's likely that the proposal will be slightly changed (read: improved) as implementation and testing proceeds. The implementation of the proposal is currently happening here: https://github.com/twstrike/tor_for_patching/tree/prop259 - We discussed whether the guard algorithm should care about circuit restrictions like requiring that the guard of a circuit needs to be Stable (needs_uptime) or Fast (needs_capacity). We decided that for now the algorithm should be able to handle these restrictions, by skipping to the next guard that satisfies the conditions in case the top guard does not. This is also how the current guard algorithm works. In the future we should make it so that all Guards are both Stable and Fast, so that this stupid check does not need to happen [TODO: I should open a ticket for this if it doesn't already exist]. - We discussed how ReachableAddresses should work with regards to guardlists. A suggestion by Tim came from here: https://lists.torproject.org/pipermail/tor-dev/2016-March/010630.html who says: "I suggest that we compose the set of UTOPIC guards based on addresses that are reachable and preferred (or, if there are no guards with preferred addresses, those guards that are reachable). I suggest that we use the same mechanism with DYSTOPIC guards, but add a port restriction to 80 & 443 to all the other restrictions. (This may result in the empty set.)" I think this suggstion makes sense for now. - We also talked about directory guards and how they should work but we didn't come to a conclusion. My current intuition is that if a directory circuit appears that requires a directory guard, we treat it as a constraint the same way we treat needs_capacity and needs_uptime. So if our top guard cannot be a directory guard (is not a V2Dir), we skip it and go down our list till we find a guard that does. In the future (almost) all guards will be directory guards so this should become less of an issue (see #12538). - Finally, a topic that came up after the meeting in: https://lists.torproject.org/pipermail/tor-dev/2016-March/010635.html had to do with how we treat bad guards in SAMPLED_UTOPIC_GUARDS and SAMPLED_DYSTOPIC_GUARDS. So if suddenly 90% of the guards in SAMPLED_UTOPIC_GUARDS drop out of the consensus, what do we do? We mark them as bad, and then what? Do we remove them from the sampled list? Or do we keep them in there in case they come back? (This latter behavior is what tor currently does). But then if we keep them, don't we also need to add some more guards in there to make up for the bad ones? How exactly should this work? Some thinking needs to be done here. - We also discussed the need for good debug logging in the prop259 tor branch, so that we can test it ourselves. Ideally, the logging should be nice and intuitive, so that we can monitor the logs every now and then and check if the guard picking works properly. For example, if the algorithm keeps on switching guards all the time, there is a problem. If the algo is using the 5th guard while the 1st guard is up, it's a problem. Good logging should be able to reveal all these problems without having to spend 20 minutes investigating every new log message.

2 1