November 2014 - tor-dev - lists.torproject.org

New documentation for Tor Metrics website
by Karsten Loesing 02 Dec '14

02 Dec '14

Dear list, writing documentation is hard work. I'm sure you already knew that, but let me re-assure you that I now believe it, too. At least until I forget it again and carelessly start documenting something else that looks tiny and friendly and trivial to document. Anyway, I spent the last day (felt like half a week) on rewriting the relevant text for the Tor Metrics website, and I wrote a short glossary of terms I didn't want to explain over and over. Would people on this list mind reviewing my text and suggesting improvements? For reference, the old text and real graphs, tables, etc. are still available here: https://metrics.torproject.org/ The new text with sample graphs and the glossary are here, conveniently packaged into a single HTML page: https://people.torproject.org/~karsten/volatile/metrics.html I'd be happy for any suggestions here or off-list, either as comments, patches, or edited HTML files. Whatever works best for you works for me. I'll update the .html file whenever I include suggestions. I hope to put the new text on the real Tor Metrics next week. Thanks in advance, much appreciated! In fact, I'm pretty sure that all future visitors of Tor Metrics will appreciate your efforts. All the best, Karsten

4 8

Help regarding Development
by Tom Mody 02 Dec '14

02 Dec '14

I would like to contribute to Tor ,, I'm not getting where to start .... Please , Can anyone help me how to start ??

2 2

Specification for 'How to Safely Sign a statement with a .onion key'
by Tom Ritter 01 Dec '14

01 Dec '14

Attached is a document written in the specification format for one aspect of CA-signed .onion addresses - specifically a "What is a safe way to sign (or not sign) a statement using the .onion key" It presents a couple options - I'd love to get feedback from folks on which they prefer. I recognize that no consensus or decision has been reached on whether to encourage or guide CAs on issuing .onions. (Although I'm in favor[0].) Although this is obviously written skewed towards CAs, it addresses a generic problem, and could be rewritten in that form. Excerpting from the Introduction/Motivation: Several organizations, such as Facebook, (collectively, 'applicants') have expressed a desire for an end-entity SSL certificate valid for their .onion Hidden Service address signed by a Certificate Authority (CA) present in browser trust stores. The existing Facebook .onion URL was issued by Digicert under a loophole in certificate issuance requirements. The Basic Requirements [0] is a document created by the Certificate Authority/Browser Forum that governs acceptable certificate issuing policies. Adherence to its policies is required for inclusion in many browsers' trust stores. .onion counts as an 'internal hostname' as it is not a recognized TLD by IANA. November 1, 2015 sets the deadline for when all internal server certificates must be revoked and no new certificates may be issued. Resolving the requirements and preferences for issuing .onion certificates before this date is extremely desirable, as it will determine if organizations such as Facebook will invest in the time and engineering effort needed to deploy Hidden Services. The full requirements for issuing .onion certificates is TBD, although recognition of .onion by IANA as a reserved domain is likely required. During discussions about the requirements for issuing for .onion one question that has arisen is what is a safe way to assert ownership of a .onion address and format an x509 certificate with a .onion Subject Alternate Name (SAN). This document is designed to address some of those questions. -tom [0] https://lists.torproject.org/pipermail/tor-dev/2014-November/007786.html

5 7

Of CA-signed certs and .onion URIs
by Tom Ritter 30 Nov '14

30 Nov '14

There's been a spirited debate on irc, so I thought I would try and capture my thoughts in long form. I think it's important to look at the long-term goals rather than how to get there, so that's where I'm going to start, and then at each item maybe talk a little bit about how to get there. So I think the Tor Project and Tor Browser should: a) Eliminate self-signed certificate errors when browsing https:// on an onion site b) Consider how Mixed Content should interact with .onion browsing c) Get .onion IANA reserved d) Address the problems that Facebook is/was concerned about when deploying a .onion e) Consider how EV treatment could be used to improve poor .onion readability (If you're not familiar with DV [Domain Validated] and EV [Extended Validation] certificates and their UI differences, you should take a peek. For example [0]. There are other subtleties and requirements on EV certs like OCSP checking that removes the indicator, and the forthcoming CT effort in Chrome, but that's mostly orthogonal.) -------------------- a) Self Signed Errors on .onion A .onion specifies the key needed. As far as little-t tor is concerned, it got you to the correct endpoint safely, so whatever SSL certificate is presented could be considered valid. However, if the Hidden Service is on box A and the webserver on box B - you'd need to do some out-of-application tricks (like stunnel) to prevent a MITM from attacking that connection. So as Roger suggested, perhaps requiring the SSL certificate to be signed by the .onion key would be a reasonable choice. But if you make that requirement, it also implies that HTTP .onions are less secure than HTTPS .onions. Which may or not be the case - you don't know. I'm not religious about anything other than getting rid of the error: I don't like that users are trained to click through security errors. This is a weakly held opinion right now - but I think it's fair to give DV treatment to http://.onion because it is, from little-t tor's point of view, secure. Following that conclusion, it is therefore fair to accept self-signed certificates and _not_ require a certificate for a https://.onion be signed by the .onion key. (Because otherwise, we're saying that SSL on .onion requires more hoops to achieve security than HTTP on .onion, which isn't the case.) -------------------- b) Mixed Content on .onion This is a can of worms I'm not going to open in this mail. But it's there, and I think it's worth thinking about whether a .onion requesting resources from http://.com or https://.com is acceptable. -------------------- c) Get .onion IANA reserved I think this is fairly apparent in itself, and is in the works [1]. Not sure its status but I would be happy to lend time in whatever IETF group/work is needed if it will help. -------------------- d) Address the problems that Facebook is/was concerned about when deploying a .onion There are reasons, technical and political, why Facebook went and got a HTTPS cert for their .onion. I've copied Alec so hopefully he'll agree, refute, or add. But from my perspective, if I were Facebook or another large company like that: i) I don't want to train my users to click through HTTPS warnings. (Conversely, I like training my users to type https://mysite.com) ii) I don't want to have to do the development and QA work to cut my site over to be sometimes-HTTP if it's normally always HTTPS iii) It would be convenient if I didn't have to do stunnel tricks to encrypt the connection between my Hidden Service server and (e.g.) load balancer, which is on another box iv) I'd really like to get a green box showing my org name, and it's even better that it'd be very difficult a phisher to get that (iii) can contradict with (A) above of course. Because I came to the conclusion of allowing invalid certificates, a MITM could attack Facebook between the HS server and load balancer. I'm not sure there is an elegant solution there. One would probably have to tunnel the connection over a mutually authenticated stunnel connection to prevent a MITM. But frankly, if we assume users are used to clicking through self-signed certs and we want to start the process of training them _not_ to, Facebook would have to do this now _anyway_. So... =/ I guess documenting the crap out of this concern and providing examples may be the best solution based off my mindset right now. It's awesome that Facebook set up a Hidden Service. I'd love to get a lot more large orgs doing that. We should reach out and figure out what the blockers are, what's painful about it, and what we can do to help. I would love doing that, it would be awesome. (And I'm not afraid to NDA myself up if necessary, seeing as I'm under NDA with half of the Bay Area anyway.) -------------------- e) Consider how EV treatment could be used to improve poor .onion readability This is the trickiest one, and it overlaps the most with the question of "Should we encourage CAs to issue certificates?" EV treatment in Tor Browser is a tool in the toolbox. I think it would be wasteful of written code and users who are accustomed to seeing it to not make use of it. I also think it dovetails nicely with how unreadable HS addresses are and how much more unreadable they're going to get soon when they get longer. I don't want a system that _requires_ participating in the DNS or CA model. Free or Paid, you still have to provide identifying information - and for an anonymity project I think we can all agree that's unacceptable. But as we hopefully expand hidden services to more and more corporate services - these organizations are legitimately concerned about (e.g.) phishing, and it's unreasonable to expect users to meticulously validate a .onion address. (Let alone how you find what the address should be validated against.) But a problem is that if we allow a .onion to certify anything it wants, it can certify any fraudulent information it wants. Bootstrapping off the other axis of Zooko's Triangle (Secure and Human Meaningful, but Centralized) is a way to combat that fraudulent information. (Not the only way, but a way.) Syrup-tan had an idea on irc: Have a DV certificate sign a certificate that is valid for the .onion URL, and display the URL of the DV certificate. This doesn't eliminate phishing - I can register facebok.com and then get that displayed. But doing bootstapping off DNS and DV certificates is a fairly low bar in terms of the cost to a .onion operator. (There are other concerns here, I'm not completely comfortable with repurposing the EV indicator in this way. Asa on irc had the good point that if we did this, maybe we'd want to change the EV green to another color just to be a little bit different. Not that I really expect users to notice that though...) Allowing an organization to purchase an EV certificate from a CA, and display the organization's name in the address bar, is another way - albeit a very high bar in terms of cost to an onion operator. A petname system based off who-knows-what (for example the namecoin/sovereign-keys like system of a land-grab, first-to-the-name approach) is a third, and would meet the goal of not requiring participating in the DNS and CA systems. but a high bar in terms of engineering effort for Tor. I think Tor Browser should do several of them. I think the EV certificates + partnering with CAs is dead simple and requires no engineering effort on behalf of Tor Browser. So that's a win, and I think worth doing. But there should be at least one more solution in the short to long term (e.g. a petname approach). Unfortunately, if the time between now and the 'long term' solution is too long, it locks out everyone who can't get an EV cert - which is a legitimate concern. Perhaps after there's a spec Tor likes, some large organization concerned about preventing phishing could throw some engineering time at the problem. Anyway, if it's not clear, I am volunteering to work on these things as I'm able. -tom [0] https://ftt-uploads.s3.amazonaws.com/browser-ssl-ui-comparison.png [1] https://datatracker.ietf.org/doc/draft-grothoff-iesg-special-use-p2p-names/

10 18

sending command from Exit node to client
by Mohiuddin Ebna Kawsar 30 Nov '14

30 Nov '14

Hi, 1* I want to know is it possible to send command from Exit node to Client? if yes then How? 2* I also want to know is it possible to communicate with tor with other software like snort? Regards kawsar

2 1

obfs4 questions
by Michael Rogers 29 Nov '14

29 Nov '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, In the obfs4 spec I couldn't find a description of how the secretbox nonces for the frames are constructed. A 16-byte nonce prefix comes from the KDF, but what about the remaining 8 (presumably frame-specific) bytes? If an attacker changes the order of the secretboxes so that the recipient tries to open a secretbox with the wrong nonce, is that guaranteed to fail, as it would if the secretbox had been modified? I can make a hand-wavy argument for why I think it will fail, but I don't know whether the secretbox construct is designed to ensure this. Any particular reason for using two different MACs (HMAC-SHA256-128 for the handshake, Poly1305 for the frames) and two different hashes (SHA-256 for the handshake, SipHash-2-4 for obfuscation)? Cheers, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJUeHO0AAoJEBEET9GfxSfMcPEH/iEYxlXtceeG3/wzp97oW/He lPNqqowyXczlyJO0SDG8L96hG6RYQZb7M0t8KJsYJapAznioZi2/qRQEC2/VFXg1 1EN//Bd9iO7QUXaIo1djC97Qoq3qmR/GY50xKYIjxr/gZSLk2dAAtleFUuerBrl9 nLrTr7kSKk3xzY0GFYtYKbj3bvuGusGrFioAIgfnKtF8iAlSjEIo8uE2Y1RFVu2d Q9GOake1VjC5V7Ue/MDCpWagwebPhnDHXSCWSXhvrYT5rmkjrkR2nhl2hAIq/0pA sfjsGquMrT5fdXcRrQaxHsamCt1228/ZlEAkCep/PRpS0NgLDJlRtPe49RD44Gk= =OHtY -----END PGP SIGNATURE-----

2 7

Stormy - request for feedback
by Griffin Boyce 28 Nov '14

28 Nov '14

Hello all, So as some of you know, I've been working on installers for hidden services, to ideally make very common services (such as blogs and plain websites) easy to deploy and automatically update. This is a very rough version of the one-click hidden service installer, but I'd love to get feedback on places where it breaks and where it could use a major structural change. Script is here, please feel free to flag bugs or tell me how I'm doing it wrong: https://github.com/glamrock/Stormy/blob/master/one-click-blog.sh Q: Can I use this right now to set up a hidden service? A: Please don't use this in production until firewall settings are in place. Q: Are there firewall settings in place? A: Not yet - the current setup is entirely for development and should not be used as-is. best, Griffin -- "I believe that usability is a security concern; systems that do not pay close attention to the human interaction factors involved risk failing to provide security by failing to attract users." ~Len Sassaman

5 5

The Sniper Attack: Anonymously Deanonymizing and Disabling the Tor Network
by Liste 27 Nov '14

27 Nov '14

http://www.nrl.navy.mil/itd/chacs/biblio/sniper-attack-anonymously-deanonym…

2 1

Potential projects for SponsorR (Hidden Services)
by George Kadianakis 27 Nov '14

27 Nov '14

Hello, this is an attempt to collect tasks that should be done for SponsorR. You can find the SponsorR page here: https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorR I'm going to focus only on the subset of those categories that Roger/David told me are the most important for the sponsor. These are: - Safe statistics collection - Tor controller API improvements - Performance improvements - Opt-in HS indexing service I haven't yet split projects into deliverables; this is a middle step to getting there. Next step is to filter and then ticketify what we have. After that we need to prioritize and pick the projects that will become deliverables. In each category, I have slightly ordered the items (so, more important items will usually be on the top, but that's not always true). I have also tried to include all the tickets that are marked as SponsorR in trac. So, let's go: == Safe statistics collection == We've discussed this quite a bit over the past year and I think we all pretty much agree on which stats are safe to collect and which not. I think we all agree that collecting the number of HS circuits and traffic volume from RPs (#13192) is harmless [0] and useful information to have. We need to clean up Roger's patch to add that information in extra-info descriptors, and then do some visualisations. That would give us a good idea of how much HSes are used. OTOH, other statistics like "# of HS descriptors" are not that harmless and the upcoming HS redesign will block us from getting this information anyway. For now, I think we should focus on #13192 for this project. == Tor controller API improvements == To better refine this project, we should think about what we want to get out of it. Here are some outcomes: a) A better control API allows us to perform better performance measurements for HSes. Karsten in #1944 worked on performance measurements of HS circuit establishment. You can find his very useful results here: http://ec2-54-92-231-52.compute-1.amazonaws.com/ We should understand exactly how Karsten is gathering those events, and see whether we can improve the timing accuracy or if we are missing any events. We need to also figure out how to do useful measurements in causal events like the race between the INTRODUCE_ACK cell and the RENDEZVOUS2. We also need to find a way to match rendezvous circuits with introduction circuits: https://trac.torproject.org/projects/tor/ticket/1944#comment:35 All in all, this seems like a project worth doing right because it will be useful in the future. It can even act as an automated regression test. b) This might also be a good time to start working on automated integration tests for HSes. It should be possible to spin up private Chutney networks and test that particular HSes are reachable. Or perform regression tests; for example, Roger recently suggested writing a regression test to make sure that clocks don't need to be synchronized to build HS circuits (#13494). We should also make testing networks better for HS testing: - #13401 TestingTorNetwork should crank down RendPostPeriod too? c) Tor should better expose error messages of failed operations. For example, this could allow TBB to inform users whether they mistyped the onion address or the HS is actually down, and it would also let us do #13208. Proposal 229 and ticket #13212 are related to this. We should see whether the PT team is planning to implement proposal 229 and how we can synchronise. d) There are various projects that are using HSes these days (TorChat, Pond, GlobaLeaks, Ricochet, etc.). We should think whether we want to support these use cases and how we can make their life easier. For example, Fabio has been asking for a way to spin up HSes using the control port (#5976). What other features do people want from the control port? And here are some more tickets marked as SponsorR from this category: - #8993 Better hidden service support on Tor control interface - #13206 Write up walkthrough of control port events when accessing a hidden service - #2554 extend torperf to record hidden service time components == Performance Improvements == This is the most juicy section. How can we make HS performance better? IIUC, we are mainly interested in client-side performance, but if a change makes both sides faster that's even better. Some projects: a) Looking at Karsten's #1944 results http://ec2-54-92-231-52.compute-1.amazonaws.com/ we see that fetching HS descriptors takes much more time than it should. I wonder why this is the case. Is there another ntohl bug there? We should perform measurements and get a good understanding of what's going on in this step. Here are some tickets that Roger opened to do exactly that: - #13208 What's the average number of hsdir fetches before we get the hsdesc? - #13209 Write a hidden service hsdir health measurer And here is a ticket with a potential issue: - #13207 Is rend_cache_clean_v2_descs_as_dir cutoff crazy high? b) Improving the other parts of the circuit establishment process is also important: - #8239 Hidden services should try harder to reuse their old intro points - #3733 Tor should abandon rendezvous circuits that cause a client request to time out - #13222 Clients accessing a hidden service can establish their rend point in parallel to fetching the hsdesc Furthermore, an area of Tor that might give us better performance but we haven't really explored yet is preemptive circuits. #13239 is about building more internal circuits for HSes. And here is a ticket suggesting more measurements: - #13194 Track time between ESTABLISH_RENDEZVOUS and RENDEZVOUS1 cell c) Another important project in this area is parallelizing HS crypto. I haven't looked at what this would actually entail, but it will probably involve implementing the undone parts of proposal 220/224. d) This might be the time to implement Encrypted Services? Many people have been asking for this feature and this might be the right time to do it: https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/ideas/xxx-en… e) Following the trail of #13207, we should look at all the magic numbers currently used by HSes, document them and see if they make sense. This includes the number of IPs (#8950), the number of HSDirs/replicas, the intro point expiration date, etc. Also, we should revisit the flags used when doing path selection for RPs, IPs, etc. f) On a more researchy tone, this might also be a good point to start poking at the HS scalability project since it will really affect HS performance. We should look at Christopher Baines' ideas and write a Tor proposal out of them: https://lists.torproject.org/pipermail/tor-dev/2014-April/006788.html https://lists.torproject.org/pipermail/tor-dev/2014-May/006812.html Last time I looked, Christopher's ideas required implementing proposal225 and #8239. g) All the projects above are aiming at improving circuit establishment performance, but none of them are dealing with performance improvements after the HS circuit has been established. On an even more researchy tone, Qingping Hou et al wrote a proposal to reduce the length of HS circuits to 5 hops (down from 6). You can find their proposal here: https://lists.torproject.org/pipermail/tor-dev/2014-February/006198.html The project is crazy and dangerous and needs lots of analysis, but it's something worth considering. Maybe this is a good time to do this analysis? h) Back to the community again. There have recently appeared a few messaging protocols that are inherently using HSes to provide link layer confidentiality and anonymity [1]. Examples include Pond, Ricochet and TorChat. Some of these applications are creating one or more HSes per user, with the assumption that HSes are something easy to make and there is no problem in having lots of them. People are wondering how well these applications scale and whether they are using the Tor network the right way. See John Brooks' mail for a small analysis: https://moderncrypto.org/mail-archive/messaging/2014/000434.html It might be worth researching these use cases to see how well Tor supports them and how they can be supported better (or whether they are a bad idea entirely). == Opt-in HS indexing service == This seems like a fun project that can be used in various ways in the future. Of course, the feature must remain opt-in so that only services that want to be public will surface. For this project, we could make some sort of 'HS authority' which collects HS information (the HS descriptor?) from volunteering HSes. It's unclear who will run an HS authority; maybe we can work with ahmia so that they integrate it in their infrastructure? If we are more experimental, we can even build a basic petname system using the HS authority [2]. Maybe just a "simple" NAME <-> PUBKEY database where HSes can register themselves in a FIFO fashion. This might cause tons of domain camping and attempts for dirty sybil attacks, but it might develop into something useful. Worst case we can shut it down and call the experiment done? AFAIK, I2P has been doing something similar at https://geti2p.net/en/docs/naming == Security / Miscellaneous == I also noticed that some tickets on trac were assigned to SponsorR but I couldn't fit them in the above categories. They are mainly security enhancements or code improvements. Here is a dump of the tickets: Security: - #13214 HS clients don't validate descriptor-id returned by HSDir - #7803 Clients shouldn't send timestamps in INTRODUCE1 cells - #8243 Getting the HSDir flag should require more effort - #2715 Is rephist-calculated uptime the right metric for HSDir assignment? Miscellaneous: - #13223 Refactor rend_client_refetch_v2_renddesc() - #13287 Investigate mysterious 24-hour lump in hsdir desc fetches - #8902 Rumors that hidden services have trouble scaling to 100 concurrent connections == Epilogue == What useful projects/tickets did I forget here? Which tasks from the above we should not do? I just went ahead and wrote down all the projects I could think of, with the idea that we will filter stuff later. Thanks! Footnotes: [0]: since RPs are picked at random by the client and not by the HS. [1]: see https://moderncrypto.org/mail-archive/messaging/2014/000434.html [2]: or if someone is more crazy, try to integrate GNUnet's GNS: https://gnunet.org/gns

16 25

Malicious relays and honeypots
by Gareth Owen 26 Nov '14

26 Nov '14

Hi all I wonder if it might be worth having a discussion on how to detect malicious and/or suspicious relays. To my knowledge, the project currently only scans for MITM and tries to detect larger Sybil attacks (but doesn't always act when detected). We have a lot of knowledge now about types of attacks and the unusual behaviours that might present if they are being deployed. For example: 1) Relays changing their ident every 24 hours are likely trying to be in a position to be the directory node for a HS (there are several of these relays active today). 2) Many relays being launched on sequential IP addresses, and/or with two nodes per IP - again - likely intercepting DHT publications or Sybil. 3) I have spotted several times a large number of relays in the same subnet, or adjacent /24 subnets. You could extend this to ASes. 4) Honeypot relays that try to spot unusual cells or traffic patterns traversing the tor network. For example, this could have detected the RELAY_EARLY attack if it was based on a different code base. One could define very robust and tight rules for what is permitted - flagging nodes sending unusual traffic. Additionally, PADDING cells are not currently used by the official client but are used very widely in traffic confirmation attacks - whilst intermediate relays wont be able to detect this, clients can and could flag it with an authority (via a 3-hop circuit). 5) One could scan for unusual descriptors being returned too - e.g., the descriptor is currently with in tight size bounds - but one could pad with bytes to support traffic confirmation if PADDING cells are put on the red flag list. 6) We also have the wider question of traffic tampering by exits, like the recent binary patching exit which I believe was not detected by the project. 7) And finally, Exits that only exit ports which permit tampering - e.g. the exits that only exit Bitcoin traffic for example. The question of course is where is the threshold and what does one do in the event of one of these.. Personally I am of the view that suspicious relays are not worth keeping in favour of diversity - but that view does contradict the project's I think. Best Gareth -- Dr Gareth Owen Senior Lecturer Forensic Computing Course Leader School of Computing, University of Portsmouth *Office:* BK1.25 *Tel:* +44 (0)2392 84 (6423) *Web*: ghowen.me

2 1