[tor-dev] Proposal status, with suggestions for fun stuff to do

Hi, everybody! This is a long mail, but if you're interested in Tor development, it could be a good one to read.

As promised at the dev meeting, I've spent a while going over all the current Tor proposals in state "Accepted", "Open", and "Draft", plus the documents in proposals/ideas. With any luck, this will help us move design discussion forward, and point out some neglected designs/discussions that people could pick up to specify/improve/implement.

I've also summarized the stuff in proposals/ideas: I think we should be more aggressive at putting proposal drafts in "Proposals", and in making a place for design overviews and/or proposal-proposal documents that are not themselves proposals.

Of course, if you're looking for a programming project, this isn't the only place to look: you can also head over to the tracker, and look at the tickets, particularly those with keyword "easy".

If this is useful to people, I'm hoping to do this once every month or two.

(A reminder: We've moved the tor specifications and proposals to their own repository. You can get the latest versions from https://gitweb.torproject.org/torspec.git . For an overview on how the proposal process is supposed to work, see proposal 001, at https://gitweb.torproject.org/torspec.git/HEAD:/proposals/001-process.txt .)

===== ACCEPTED:

110 Avoiding infinite length circuits

This proposal solves a potential DoS attack.

This has been nearly completely implemented for a while. All we need to do is verify that there are no clients that do not send RELAY_EARLY cells correctly, and if so, turn on the code that drops EXTEND cells not sent via RELAY_EARLY. The hard part here is double-checking that all the clients that send EXTEND commands inside non-RELAY_EARLY cells are really and truly obsolete.

117 IPv6 exits [for 0.2.1.x]

This proposal describes how to transmit IPv6 traffic over Tor.

It needs updating to work properly with microdescriptors; it also has some open questions about DNS.

I think this should move to 'needs-revision'.

118 Advertising multiple ORPorts at once

This proposal describes how an OR can advertise more than one address and OR port at a time. It needs to be updated to work with microdescriptors, and to explain how much information can be transmitted in the consensus and how (the original proposal was written before consensus directories were really figured out).

I think this should move to 'needs-revision'

140 Provide diffs between consensuses

This proposal describes a way to transmit less directory traffic by sending only differences between consensuses, rather than the consensuses themselves. It is mainly languishing for lack of an appropriately licensed, well-written, very small, pure-C implementation of the "diff" and "patch" algorithms. (The good diffs seem to be GPL (which we can't use without changing Tor's license), or spaghetti code, or not easily usable as a library, or not written in C, or very large, or some combination of those.)

ACTION: Move to 'needs-revision', or find a suitable library

147 Eliminate the need for v2 directories in generating v3 directories

This proposal explains a way that we can phase out the vestigial use of v2 directory documents in keeping authorities well-informed enough to generating the v3 consensus. It's still correct; somebody should implement it before the v2 directory code rots any further.

157 Make certificate downloads specific

This proposal added cross-certification and signing-key-specific download URLs for directory authority certificates. It is IIRC mostly implemented; there are just some SHOULDs that we should turn into MUSTS if all sufficiently old authority certificates are now obsolete.

166 Including Network Statistics in Extra-Info Documents

I think this one is implemented (or mostly implemented) by Karsten in 0.2.2.x. If so, we just need to make sure it is merged into dir-spec, and closed. Karsten, is there something left here?

(NOTE TO SELF: I should remember to ask Karsten if he doesn't notice the above.)

172 GETINFO controller option for circuit information 173 GETINFO Option Expansion

These would help controllers (particularly arm) provide more useful information about a running Tor process. They're accepted and some parts of 173 are even implemented: somebody just needs to implement the rest.

174 Optimistic Data for Tor: Server Side

This proposal would make streams start faster by allowing clients to send DATA cells immediately, without waiting for a CONNECTED cell. There's a patch under review as trac ticket #1795: it has some issues that need to get fixed before we can merge it. I really want to get it fixed up soon, and merged in the next week or two.

===== OPEN:

143 Improvements of Distributed Storage for Tor Hidden Service Descriptors

Here's a proposal from Karsten about making the hidden service DHT more reliable and secure to use. It could use more discussion and analysis.

145 Separate "suitable as a guard" from "suitable as a new guard"

Currently, the Guard flag means both "You can use this node as a guard if you need to pick a new guard" and "If this node is currently your guard, you can keep using it as a guard." This proposal tries to separate these two concepts, so that clients can stop picking a router once it is full of existing clients using it as a guard, but the clients currently on it won't all drop it.

It's not clear whether this has anonymity issues, and it's not clear whether the imagined performance gains are actually worthwhile.

146 Add new flag to reflect losing-term stability

From time to time we get the idea of having clients ship with a reasonably recent consensus (or a list of directory mirrors), so instead of bootstrapping from one of the authorities, they can bootstrap from a regular directory cache. The problem here is that by the time the client is run, most of the directory mirrors will be down or will have changed their IP. This proposal tries to address that.

It needs analysis based on behavior of actual routers on the network to see whether it could work, and what parameters might work.

156 Tracking blocked ports on the client side

This proposal provides a way for clients to learn which ports they are (and aren't) able to connect to, and connect to the ones that work. It comes with a patch, too. It also lets routers track ports that _they_ can't connect to.

I'm a little unconvinced that : most clients that have some ports blocked will need bridges, not just restriction to a smaller set of ports. This could be good behind restrictive firewalls, though.

The router-side part is a little iffy: routers that can't connect to each other violate one of our network topology assumptions, and even if we do want to track failed router->router connections, the routers need to be sure that they aren't fooled into trying to connect repeatedly to a series of nonexistent addresses in an attempt to make them believe that (say) they can't reach port 443.

This one is a paradigmatic "open" proposal: it needs more discussion. The patch probably also needs to be ported to 0.2.3.x; it touches some code that has changed.

158 Clients download consensus + microdescriptors

This proposal provides a way for clients to download authority-provided summaries of router descriptors instead of the descriptors themselves; I'm working on this one; it's going in 0.2.3.x. It needs some cleanup, but it's basically right.

(NOTE TO SELF: Clean this up and move it to "Accepted!")

159 Exit Scanning

This is an overview of SoaT, with some ideas for how to integrate it into Tor.

Mike, is this implemented? Done? Superseded? Still open?

(NOTE TO SELF: I should remember to ask Mike if he doesn't notice the above.)

162 Publish the consensus in multiple flavors

This proposal describes a way for authorities to migrate from one format of consensus to another without bloating the consensus forever. It is mostly implemented in 0.2.3.x, though some parts of it need to get work. I should clean it up too.

(NOTE TO SELF: Clean this up and move it to "Accepted!")

163 Detecting whether a connection comes from a client

This one describes a number of ways to distinguish clients from non-clients when rejecting single-hop connections. Some are implemented; others are rejected; others are still discussable. Somebody should figure out what happened here, and reopen discussion as interested.

164 Reporting the status of server votes

This proposal explains a way for authorities to provide a slightly more verbose document that relay operators can use to diagnose reasons that their router was or was not listed in the consensus. These documents would be like slightly more verbose versions of the authorities' votes, and would explain *why* the authority voted as it did. It wouldn't be too hard to implement, and would be a fine project for somebody who wants to get to know the directory code.

165 Easy migration for voting authority sets

This is a design for how to change the set of authorities without having a flag day where the authority operators all reconfigure their authorities at once. It needs more discussion. One difficulty here is that we aren't talking much about changing the set of authorities, but that may be a chicken-and-egg issue, since changing the set is so onerous.

If anybody is interested, it would be great to move the discussion ahead here.

168 Reduce default circuit window

This proposal reduces the default window for circuit sendme cells. I think it's implemented, isn't it? If so, we should make sure that tor-spec.txt is updated and close it.

(NOTE TO SELF: Ask Roger; This should probably be called "Finished", or "Closed" if we have updated the spec.)

171 Separate streams across circuits by connection metadata

This proposal describes a way to prevent cross-application linking of different anonymized sessions by isolating different kinds of streams on different circuits based on their characteristics.

I want this one in 0.2.3.x; it would improve client anonymity. A patch would be great; there are efficiency issues to consider, though.

ACTION: This should be "accepted", I believe, unless somebody has lingering objections.

176 Proposed version-3 link handshake for Tor

Here's my proposal for how to improve our TLS handshake to eliminate renegotiation, become less fingerprintable, and generally make the world a better place. It needs more review by actual crypto people, but I would like to get it into 0.2.3.x.

177 Abstaining from votes on individual flags

Here's my proposal for letting authorities have opinions about some (flag,router) combinations without voting on whether _every_ router should have that flag. It's simple, and I think it's basically right. With more discussion and review, somebody could/should build it for 0.2.3.x, I think.

===== DRAFT:

127 Relaying dirport requests to Tor download site / website

The idea here was to make it easier to fetch and learn about Tor by making it easy for relays to automatically act as proxies to the Tor website. It needs more discussion, and there are some significant details to work out. It's not at all clear whether this is actually a good idea or not.

132 A Tor Web Service For Verifying Correct Browser Configuration

This proposal was meant to give users a way to see if their browser and privoxy (yes, it was a while ago) are correctly configured by running a local webserver on 127.0.0.1. I'm not sure the status here.

133 Incorporate Unreachable ORs into the Tor Network

This proposal had an idea for letting ORs that can only make outgoing connections still relay data usefully in the network. It's something we should keep in mind, and it's a pretty neat idea, but it radically changes the network topology. Anybody who wants to analyze new network topologies should definitely have a look.

141 Download server descriptors on demand

The idea of this proposal was for clients to only download the consensus, and later ask nodes for their own server descriptors while building the circuit through them. It would make each circuit more time-consuming to build, but make bootstrapping much cheaper.

A microdescriptor-based version of this would be even better, and microdescriptors would solve a lot of the problem that this was meant to resolve. It still is not wholly superseded by the microdescriptor system, though.

144 Increase the diversity of circuits by detecting nodes belonging the same provider

This is a version of the good idea, "Let's do routing in a way that tries to keep from routing traffic through the same provider too much!" There are complex issues here that the proposal doesn't completely address, but I think it might be a fine idea for somebody to see how much more we know now than we did in 2008, particularly in light of the relevant paper(s) by Matt Edmann and Paul Syverson.

149 Using data from NETINFO cells

NETINFO cells currently contain time and address information that could be used to estimate skew and provide yet another way to learn our IP address. This proposal starts to discuss ways that we could try to use that information safely, but doesn't really get anywhere. See trac issues #2628 for some discussion about this stuff. It would be neat if we had a working design here, though I suspect that with the slow and grinding death of older operating systems, systems with radically skewed clocks may become a thing of the past.

170 Configuration options regarding circuit building

This is Sebastian's take on how the *Nodes options should work. Roger preferred a more minimal approach that may or may not be right; see the discussion starting at https://trac.torproject.org/projects/tor/ticket/1090#comment:21 for his approach. I'm not going to call this "superseded" or "rejected" until my 1090 code is done, though. :(

175 Automatically promoting Tor clients to nodes

Here's Steven's proposal for adding a mode between "client mode" and "relay mode" for "self-test to see if you would be a good relay, and if so become one." It didn't get enough attention when it was posted to the list; more people should review it.

178 Require majority of authorities to vote for consensus parameters

This is a proposal for making it a little harder for a small number of authorities to influence the value of a consensus parameter. It needs more discussion; Sebastian has a draft implementation in his "safer_params" branch.

(NOTE TO SELF: I think this could fairly become "open"; I should ask Sebastian.)

===== NEEDS-REVISION: 131 Help users to verify they are using Tor

Here's a proposal for making a torcheck-like website more reliable. If anybody wants to pick it up (especially somebody working on torcheck) and see whether it should be reopened or rejected, that would be a fine thing.

====== "IDEAS":

xxx-bwrate-algs.txt

Here's a note about better round-robin calculations for rate limiting. Mike wrote it; I don't understand what's going on with it.

xxx-choosing-crypto-in-tor-protocol.txt

Here's the start of a considerations document that Marian was writing about how to pick new crypto algorithms to migrate to.

xxx-controllers-intercept-extends.txt

Here's an old idea from Geoff Goodell about letting controllers intercept EXTEND commands. It is partially superseded by Damian's 172, though there is an additional feature that doesn't make sense for the Tor network, but might make sense for experimental stuff like Geoff was working on.

xxx-crypto-migration.txt

Here's the document I wrote in December about which parts of our crypto there are to migrate, and what might be involved. This isn't a draft or pre-draft of a design proposal; it is more of a new category of "survey of stuff we need to design and think about", or "proposal for proposals" or something.

xxx-crypto-requirements.txt

Here's Robert Ransom's draft for what crypto properties, exactly, we're trying to get out of our circuit crypto. It is not a proposal draft or pre-draft: it is again a "design considerations" document, and one worth reading.

xxx-draft-spec-for-TLS-normalization.txt

Here's Jacob's proposal for certificate normalization. It should get renamed, given a proposal number, and called "Open" or "Draft" depending on how much the details are likely to change.

xxx-encrypted-services.txt

This is a start-of-a-proposal of Rogers that somebody should pick up and finish some time: The idea is to use the hidden service mechanism to provide a secure naming and encrypted connection facility for hosting sites that do not actually want anonymity themselves. There's been more interest in this topic lately. It might also turn into "exit enclaves done right".

xxx-exit-scanning-outline.txt

Looks like this was superseded by 159?

xxx-geoip-survey-plan.txt

Here's an old document I wrote a while ago about tracking usage by country. Probably it should go into the metrics documentation somewhere (if we do this), or get thrown into "old" (if we won't do this), or updated (if we might someday do this).

xxx-grand-scaling-plan.txt

Here are some notes on scaling that Roger wrote in 2008. There might be some smart ideas here! Have a look some time.

xxx-hide-platform.txt

Here's a proposal pre-draft that says we should normalize the platform string. Somebody could turn this into a proposal pretty easily.

xxx-ipv6-plan.txt

Here's a survey of what we need to do for IPv6 support that I started writing. It's not a proposal; it's a survey of proposal and implementation statuses.

xxx-pluggable-transport.txt

Here's the thing Jacob and I have been working on for pluggable obfuscation techniques. It should get turned into "Draft" status asap, if not "Open". We should finish writing the missing sections, like, yesterday.

xxx-port-knocking.txt

Here's a pre-proposal from Jacob about using port knocking to make bridges harder to fingerprint. This could be a good idea for somebody to do in terms of the pluggable transport spec.

xxx-rate-limit-exits.txt

Here are some notes of Roger's from 2008 claiming that we should rate-limit stream creation at exit nodes. It could help avoid port-scans if we do it right, but we would need to be exceedingly careful not to disrupt useful traffic.

xxx-using-spdy.txt

Here's a document from Steven about opportunistic use of SPDY over Tor.

This should be a "Draft" proposal IMO.

xxx-what-uses-sha1.txt

Here's the beginnings of a survey of where Tor uses SHA1, with an eye to stopping. This really isn't a design proposal; it might fall into a new category of "issue survey" or "information" or something.