- tor-dev - lists.torproject.org

Is PublishServerDescriptor needed to collect metrics?
by David Fifield 02 Dec '14

02 Dec '14

If a bridge has PublishServerDescriptor 0 does that prevent it from counting in metrics? If so, what's the right setting to enable metrics (0|1|v3|bridge,...)? Is there a way to send metrics data without also ending up in BridgeDB? David Fifield

4 7

New documentation for Tor Metrics website
by Karsten Loesing 02 Dec '14

02 Dec '14

Dear list, writing documentation is hard work. I'm sure you already knew that, but let me re-assure you that I now believe it, too. At least until I forget it again and carelessly start documenting something else that looks tiny and friendly and trivial to document. Anyway, I spent the last day (felt like half a week) on rewriting the relevant text for the Tor Metrics website, and I wrote a short glossary of terms I didn't want to explain over and over. Would people on this list mind reviewing my text and suggesting improvements? For reference, the old text and real graphs, tables, etc. are still available here: https://metrics.torproject.org/ The new text with sample graphs and the glossary are here, conveniently packaged into a single HTML page: https://people.torproject.org/~karsten/volatile/metrics.html I'd be happy for any suggestions here or off-list, either as comments, patches, or edited HTML files. Whatever works best for you works for me. I'll update the .html file whenever I include suggestions. I hope to put the new text on the real Tor Metrics next week. Thanks in advance, much appreciated! In fact, I'm pretty sure that all future visitors of Tor Metrics will appreciate your efforts. All the best, Karsten

4 8

Re: [tor-dev] tor-dev Digest, Vol 47, Issue 3
by teor 02 Dec '14

02 Dec '14

Hi Tom, Can you please describe the bug? Is it in the Tor Trac with a bug number? In general, if you start in the tor directory of a source distribution: (I'll use gitweb links, but you can use git and clone a local copy) tor/src/or https://gitweb.torproject.org/tor.git/tree/src/or Contains the tor onion routing-specific source code Your bug is likely to be here tor/src/common https://gitweb.torproject.org/tor.git/tree/src/common Contains the common facilities used by the tor/src/or code Your bug is also likely to be here tor/src/ext tor/src/trunnel tor/src/win32 https://gitweb.torproject.org/tor.git/tree/src Contain external libraries and platform-specific source Your bug is probably not here, or in any of the other directories teor pgp 0xABFED1AC hkp://pgp.mit.edu/ https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7w… > Date: Tue, 2 Dec 2014 07:05:45 +0530 > From: Tom Mody <bug29195(a)gmail.com> > Subject: Re: [tor-dev] Help regarding Development > > hi list , > I am having troubles on bug solving , im new to it and i want to know where > to search in code for a particular bug ,, > please help ?? > >> On Tue, Nov 18, 2014 at 10:22 PM, rl1987 <rl1987(a)sdf.lonestar.org> wrote: >> >> 1. Read doc/HACKING. >> 2. Browse "Easy tickets" page [1] and see if you can solve any of them. >> >> [1] https://trac.torproject.org/projects/tor/query?status= >> accepted&status=assigned&status=needs_review&status= >> new&status=reopened&order=priority&keywords=~easy&col= >> id&col=summary&col=component&col=status&col=type&col= >> priority&col=milestone&report=30 >> >> 2014-11-14 03:53, Tom Mody ra??: >> >>> I would like to contribute to Tor ,, >>> >>> I'm not getting where to start .... >>> >>> Please , Can anyone help me how to start ??

1 0

Help regarding Development
by Tom Mody 02 Dec '14

02 Dec '14

I would like to contribute to Tor ,, I'm not getting where to start .... Please , Can anyone help me how to start ??

2 2

[PATCH] Tor Weather: Switched preferences and unsubscribes URLs.
by Ondrej Novy 01 Dec '14

01 Dec '14

--- weather/weatherapp/emails.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/weather/weatherapp/emails.py b/weather/weatherapp/emails.py index 2b5a0f5..cc298c9 100644 --- a/weather/weatherapp/emails.py +++ b/weather/weatherapp/emails.py @@ -179,7 +179,7 @@ def _add_generic_footer(msg, unsubs_auth, pref_auth): unsubURL = url_helper.get_unsubscribe_url(unsubs_auth) prefURL = url_helper.get_preferences_url(pref_auth) - footer = _GENERIC_FOOTER % (unsubURL, prefURL) + footer = _GENERIC_FOOTER % (prefURL, unsubURL) return msg + footer -- 1.9.3 (Apple Git-50)

1 0

Specification for 'How to Safely Sign a statement with a .onion key'
by Tom Ritter 01 Dec '14

01 Dec '14

Attached is a document written in the specification format for one aspect of CA-signed .onion addresses - specifically a "What is a safe way to sign (or not sign) a statement using the .onion key" It presents a couple options - I'd love to get feedback from folks on which they prefer. I recognize that no consensus or decision has been reached on whether to encourage or guide CAs on issuing .onions. (Although I'm in favor[0].) Although this is obviously written skewed towards CAs, it addresses a generic problem, and could be rewritten in that form. Excerpting from the Introduction/Motivation: Several organizations, such as Facebook, (collectively, 'applicants') have expressed a desire for an end-entity SSL certificate valid for their .onion Hidden Service address signed by a Certificate Authority (CA) present in browser trust stores. The existing Facebook .onion URL was issued by Digicert under a loophole in certificate issuance requirements. The Basic Requirements [0] is a document created by the Certificate Authority/Browser Forum that governs acceptable certificate issuing policies. Adherence to its policies is required for inclusion in many browsers' trust stores. .onion counts as an 'internal hostname' as it is not a recognized TLD by IANA. November 1, 2015 sets the deadline for when all internal server certificates must be revoked and no new certificates may be issued. Resolving the requirements and preferences for issuing .onion certificates before this date is extremely desirable, as it will determine if organizations such as Facebook will invest in the time and engineering effort needed to deploy Hidden Services. The full requirements for issuing .onion certificates is TBD, although recognition of .onion by IANA as a reserved domain is likely required. During discussions about the requirements for issuing for .onion one question that has arisen is what is a safe way to assert ownership of a .onion address and format an x509 certificate with a .onion Subject Alternate Name (SAN). This document is designed to address some of those questions. -tom [0] https://lists.torproject.org/pipermail/tor-dev/2014-November/007786.html

5 7

Reminder for weekly OONI dev meeting Monday 18:00 UTC (19:00 CET)
by Arturo Filastò 01 Dec '14

01 Dec '14

Hi, This is a reminder that today there will be the weekly OONI meeting. As I am writing this I am on a plane heading towards London and will be working on a GlobaLeaks workshop in some office there. The schedule for the workshop is not very well defined so I am not 100% sure I will manage to be online at the above defined time. If I am not there please start without me and I will be sure to read the backlog and write feedback asynchronously. ~ Arturo

1 0

Re: [tor-dev] [tor-reports] Tom's November 2014
by Karsten Loesing 01 Dec '14

01 Dec '14

On 01/12/14 02:34, Tom Ritter wrote: > With my shiny new ldap I got access to a user and official repo for > Depictor [0][1]. Not sure how I'll manage the difference. The code > for depictor is in both of them. Hi Tom, here's how I use official and user repos for my code: - The official repo contains what's deployed on the server, in its master branch. There are usually no other branches than master in that repo, though there would be for different version branches, or in the case of DocTor the old Java branch. I never rebase the master branch in the official repo. - The user repo has feature branches that I'm working on. For example, the feature branch for #23456 would be called task-23456, or task-23456-2, -3, etc. when I rebase that branch or mess with history in some other way. The user repo has no master branch, because that would be out of date most of the time anyway. I sometimes delete branches after they're merged into the official master branch, if I'm sure that nobody would base their work on those branches. There may be better approaches to using official vs. user repos, but maybe this is useful as a first data point for you. The Githax repo may contain more ideas: https://gitweb.torproject.org/githax.git/tree/doc/Howto.txt All the best, Karsten

1 0

Integration testing plans, draft, v1
by Nick Mathewson 01 Dec '14

01 Dec '14

Hi! This is the outcome of some discussion I've had with dgoulet, and some work I've done at identifying current problem-points in our use of integration testing tools. I'm posting it here for initial feedback, and so I have a URL to link to in my monthly report. :) INTEGRATION TEST PLANS FOR TOR November 2014 1. Goals and nongoals and scope This is not a list of all the tests we need; this is just a list of the kind of tests we can and should run with Chutney. These tests need to be the kind that a random developer can run on their own and reasonably hope to do. Longer/more expensive tests may be okay too, but if it needs anything spiffer than a linux desktop, consider using Shadow instead. Setting up an environment to run the tests needs to be so easy that nobody who writes C for Tor is likely to be dissuaded from running them. Writing new tests needs to be pretty simple too. Most tests need to be runnable on all the platforms we support. We should support load tests. Though doing so is not likely to give an accurate picture of how the network behaves under load, it's probably good enough to identify bottlenecks in the code. (David Goulet has had some success here already for identifying HS performance issues.) We should specify our design and interfaces to keep components loosely coupled and easily replaceable. With that done, we should avoid over-designing components at first: experience teaches that only experience can teach what facilities need which features. 2. Architecture Here are the components. I'm treating them as conceptually separate, though in practice several of them may get folded into Chutney. A. Usage simulator One or more programs that emulate users and servers on the internet. It reports what succeeded and what failed and how long everything took. Right now we're using curl and nc for this. B. Network manager This is what Chutney does today. It launches a set of Tor nodes according to a provided configuration. C. Testcase scripts We do this in shell today: launch the network, wait for it to bootstrap, send some traffic through it, and report success or failure. D. Test driver This part is responsible for determining which testcases to run, in what order, on what network. There is no current analogue to this step; we've only got the one test-network.sh script, and it assumes a single type of network. One thing to notice here is that testcase scripts need to work with multiple kinds of network manager configurations. For example, we'd like to be able to run HTTP-style connectivity tests on small networks, large networks, heterogenous networks, dispersed networks, and so on. We therefore need to make sure that each kind of network can work with as many tests as possible, so that the work needed to write the tests doesn't grow quadratically. The coupling for the components will go as possible: A. Usage simulations will need to expose their status to test scripts. B. The network manager will need to expose information about available networks and network configurations to the test scripts, so that the test scripts know how to configure usage simulations to use them. It will need to expose commands like "wait until bootstrapped", "check server logs for trouble", etc. C. Each testcase needs to be able to identify which features it needs from a network, invoke the network commands it needs, and invoke usage simulations. It needs to export information about its running status, and whether it's making progress. D. The test driver needs to be able to enumerate networks and testcases and figure out which are compatible with each other, and which can run locally, and which ones meet the user's requirements. 2.1. Minimal versions of the above: A. The minimal user tools are an http server and client. Use appropriate tooling to support generating and receiving hundreds to thousands of simultaneous requests. B. The minimal network manager is probably chutney. It needs the ability to export information about networks, to "wait until bootstrapped", to export information about servers, and so on. Its network list needs to turn into a database of named networks. C. Testcases need to be independent, and ideally abstracted. They shouldn't run in-process with chutney. For starters, they can duplicate the current functionality of test-network and of dgoulet's hidden service tests. Probing for features can be keyword-based. reporting results can use some logging framework. D. The test driver can do its initial matching by keyword tags exported by the other objects. It should treat testcases and networks as abitrary subprocesses that it can launch, so that they can be written in any language. 3. A short inventory of use-cases - Voting - Controllers - HS * IP+RP up/down * HSDir up/down * Authenticated HS - Bad clients * Wrong handshake * Wrong desc. - Bad relays * dropping cells/circ/traffic * BAD TLS - Pathing * Does it behaves the way we think? * 4 hops for HS - Relay * Up/Down * Multiple IPs for a single key * OOM handling * Scheduling - Client * Does traffic goes through? - For HS and Exit * DNS testing - Caches at the exit * Stream isolation using n SocksPort * AddressMap 4. Longer-term We might look into using "The Internet" (no, not that one. The one at https://github.com/nsec/the-internet) on Linux to simulate latency between nodes. When designing tools and systems, we should do so with an eye to migrating them into shadow. We should refactor or adapt chutney to support using stem, to take advantage of stem's improved templating and control features, so we can better inspect running servers. We should probably retain the original hands-off non-controller chutney design too, to better detect heisenbugs. 5. Immediate steps - Turn the above into a work plan. - Specify initial interfaces, with plan for migration to better ones once we have more experience. - Identify current chutney and test-network issues standing in the way of reliably getting current tests to work. - Refactor our current integration tests to the above framework.

1 0

Of CA-signed certs and .onion URIs
by Tom Ritter 30 Nov '14

30 Nov '14

There's been a spirited debate on irc, so I thought I would try and capture my thoughts in long form. I think it's important to look at the long-term goals rather than how to get there, so that's where I'm going to start, and then at each item maybe talk a little bit about how to get there. So I think the Tor Project and Tor Browser should: a) Eliminate self-signed certificate errors when browsing https:// on an onion site b) Consider how Mixed Content should interact with .onion browsing c) Get .onion IANA reserved d) Address the problems that Facebook is/was concerned about when deploying a .onion e) Consider how EV treatment could be used to improve poor .onion readability (If you're not familiar with DV [Domain Validated] and EV [Extended Validation] certificates and their UI differences, you should take a peek. For example [0]. There are other subtleties and requirements on EV certs like OCSP checking that removes the indicator, and the forthcoming CT effort in Chrome, but that's mostly orthogonal.) -------------------- a) Self Signed Errors on .onion A .onion specifies the key needed. As far as little-t tor is concerned, it got you to the correct endpoint safely, so whatever SSL certificate is presented could be considered valid. However, if the Hidden Service is on box A and the webserver on box B - you'd need to do some out-of-application tricks (like stunnel) to prevent a MITM from attacking that connection. So as Roger suggested, perhaps requiring the SSL certificate to be signed by the .onion key would be a reasonable choice. But if you make that requirement, it also implies that HTTP .onions are less secure than HTTPS .onions. Which may or not be the case - you don't know. I'm not religious about anything other than getting rid of the error: I don't like that users are trained to click through security errors. This is a weakly held opinion right now - but I think it's fair to give DV treatment to http://.onion because it is, from little-t tor's point of view, secure. Following that conclusion, it is therefore fair to accept self-signed certificates and _not_ require a certificate for a https://.onion be signed by the .onion key. (Because otherwise, we're saying that SSL on .onion requires more hoops to achieve security than HTTP on .onion, which isn't the case.) -------------------- b) Mixed Content on .onion This is a can of worms I'm not going to open in this mail. But it's there, and I think it's worth thinking about whether a .onion requesting resources from http://.com or https://.com is acceptable. -------------------- c) Get .onion IANA reserved I think this is fairly apparent in itself, and is in the works [1]. Not sure its status but I would be happy to lend time in whatever IETF group/work is needed if it will help. -------------------- d) Address the problems that Facebook is/was concerned about when deploying a .onion There are reasons, technical and political, why Facebook went and got a HTTPS cert for their .onion. I've copied Alec so hopefully he'll agree, refute, or add. But from my perspective, if I were Facebook or another large company like that: i) I don't want to train my users to click through HTTPS warnings. (Conversely, I like training my users to type https://mysite.com) ii) I don't want to have to do the development and QA work to cut my site over to be sometimes-HTTP if it's normally always HTTPS iii) It would be convenient if I didn't have to do stunnel tricks to encrypt the connection between my Hidden Service server and (e.g.) load balancer, which is on another box iv) I'd really like to get a green box showing my org name, and it's even better that it'd be very difficult a phisher to get that (iii) can contradict with (A) above of course. Because I came to the conclusion of allowing invalid certificates, a MITM could attack Facebook between the HS server and load balancer. I'm not sure there is an elegant solution there. One would probably have to tunnel the connection over a mutually authenticated stunnel connection to prevent a MITM. But frankly, if we assume users are used to clicking through self-signed certs and we want to start the process of training them _not_ to, Facebook would have to do this now _anyway_. So... =/ I guess documenting the crap out of this concern and providing examples may be the best solution based off my mindset right now. It's awesome that Facebook set up a Hidden Service. I'd love to get a lot more large orgs doing that. We should reach out and figure out what the blockers are, what's painful about it, and what we can do to help. I would love doing that, it would be awesome. (And I'm not afraid to NDA myself up if necessary, seeing as I'm under NDA with half of the Bay Area anyway.) -------------------- e) Consider how EV treatment could be used to improve poor .onion readability This is the trickiest one, and it overlaps the most with the question of "Should we encourage CAs to issue certificates?" EV treatment in Tor Browser is a tool in the toolbox. I think it would be wasteful of written code and users who are accustomed to seeing it to not make use of it. I also think it dovetails nicely with how unreadable HS addresses are and how much more unreadable they're going to get soon when they get longer. I don't want a system that _requires_ participating in the DNS or CA model. Free or Paid, you still have to provide identifying information - and for an anonymity project I think we can all agree that's unacceptable. But as we hopefully expand hidden services to more and more corporate services - these organizations are legitimately concerned about (e.g.) phishing, and it's unreasonable to expect users to meticulously validate a .onion address. (Let alone how you find what the address should be validated against.) But a problem is that if we allow a .onion to certify anything it wants, it can certify any fraudulent information it wants. Bootstrapping off the other axis of Zooko's Triangle (Secure and Human Meaningful, but Centralized) is a way to combat that fraudulent information. (Not the only way, but a way.) Syrup-tan had an idea on irc: Have a DV certificate sign a certificate that is valid for the .onion URL, and display the URL of the DV certificate. This doesn't eliminate phishing - I can register facebok.com and then get that displayed. But doing bootstapping off DNS and DV certificates is a fairly low bar in terms of the cost to a .onion operator. (There are other concerns here, I'm not completely comfortable with repurposing the EV indicator in this way. Asa on irc had the good point that if we did this, maybe we'd want to change the EV green to another color just to be a little bit different. Not that I really expect users to notice that though...) Allowing an organization to purchase an EV certificate from a CA, and display the organization's name in the address bar, is another way - albeit a very high bar in terms of cost to an onion operator. A petname system based off who-knows-what (for example the namecoin/sovereign-keys like system of a land-grab, first-to-the-name approach) is a third, and would meet the goal of not requiring participating in the DNS and CA systems. but a high bar in terms of engineering effort for Tor. I think Tor Browser should do several of them. I think the EV certificates + partnering with CAs is dead simple and requires no engineering effort on behalf of Tor Browser. So that's a win, and I think worth doing. But there should be at least one more solution in the short to long term (e.g. a petname approach). Unfortunately, if the time between now and the 'long term' solution is too long, it locks out everyone who can't get an EV cert - which is a legitimate concern. Perhaps after there's a spec Tor likes, some large organization concerned about preventing phishing could throw some engineering time at the problem. Anyway, if it's not clear, I am volunteering to work on these things as I'm able. -tom [0] https://ftt-uploads.s3.amazonaws.com/browser-ssl-ui-comparison.png [1] https://datatracker.ietf.org/doc/draft-grothoff-iesg-special-use-p2p-names/

10 18