[tor-dev] Proposal status, with suggestions for fun stuff to do

[Nick Mathewson]

Hi, everybody!  This is a long mail, but if you're interested in Tor
development, it could be a good one to read.

As promised at the dev meeting, I've spent a while going over all the
current Tor proposals in state "Accepted", "Open", and "Draft", plus
the documents in proposals/ideas.  With any luck, this will help us
move design discussion forward, and point out some neglected
designs/discussions that people could pick up to
specify/improve/implement.

I've also summarized the stuff in proposals/ideas: I think we should
be more aggressive at putting proposal drafts in "Proposals", and in
making a place for design overviews and/or proposal-proposal documents
that are not themselves proposals.

Of course, if you're looking for a programming project, this isn't the
only place to look: you can also head over to the tracker, and look at
the tickets, particularly those with keyword "easy".

If this is useful to people, I'm hoping to do this once every month or two.

(A reminder: We've moved the tor specifications and proposals to their
own repository.  You can get the latest versions from
https://gitweb.torproject.org/torspec.git .  For an overview on how
the proposal process is supposed to work, see proposal 001, at
https://gitweb.torproject.org/torspec.git/HEAD:/proposals/001-process.txt
.)

===== ACCEPTED:

   110  Avoiding infinite length circuits

    This proposal solves a potential DoS attack.

    This has been nearly completely implemented for a while.  All we
    need to do is verify that there are no clients that do not send
    RELAY_EARLY cells correctly, and if so, turn on the code that
    drops EXTEND cells not sent via RELAY_EARLY.  The hard part here
    is double-checking that all the clients that send EXTEND commands
    inside non-RELAY_EARLY cells are really and truly obsolete.

   117  IPv6 exits [for 0.2.1.x]

    This proposal describes how to transmit IPv6 traffic over Tor.

    It needs updating to work properly with microdescriptors; it
    also has some open questions about DNS.

    I think this should move to 'needs-revision'.

   118  Advertising multiple ORPorts at once

    This proposal describes how an OR can advertise more than one
    address and OR port at a time.  It needs to be updated to work
    with microdescriptors, and to explain how much information can
    be transmitted in the consensus and how (the original proposal
    was written before consensus directories were really figured
    out).

    I think this should move to 'needs-revision'

   140  Provide diffs between consensuses

    This proposal describes a way to transmit less directory traffic
    by sending only differences between consensuses, rather than the
    consensuses themselves.  It is mainly languishing for lack of an
    appropriately licensed, well-written, very small, pure-C
    implementation of the "diff" and "patch" algorithms.  (The good
    diffs seem to be GPL (which we can't use without changing Tor's
    license), or spaghetti code, or not easily usable as a library,
    or not written in C, or very large, or some combination of
    those.)

    ACTION: Move to 'needs-revision', or find a suitable library

   147  Eliminate the need for v2 directories in generating v3 directories

    This proposal explains a way that we can phase out the vestigial
    use of v2 directory documents in keeping authorities
    well-informed enough to generating the v3 consensus.  It's still
    correct; somebody should implement it before the v2 directory
    code rots any further.

   157  Make certificate downloads specific

    This proposal added cross-certification and signing-key-specific
    download URLs for directory authority certificates.  It is IIRC
    mostly implemented; there are just some SHOULDs that we should
    turn into MUSTS if all sufficiently old authority certificates
    are now obsolete.

   166  Including Network Statistics in Extra-Info Documents

    I think this one is implemented (or mostly implemented) by
    Karsten in 0.2.2.x.  If so, we just need to make sure it is
    merged into dir-spec, and closed.  Karsten, is there something
    left here?

    (NOTE TO SELF: I should remember to ask Karsten if he doesn't
    notice the above.)

   172  GETINFO controller option for circuit information
   173  GETINFO Option Expansion

    These would help controllers (particularly arm) provide more useful
    information about a running Tor process.  They're accepted and
    some parts of 173 are even implemented: somebody just needs to
    implement the rest.

   174  Optimistic Data for Tor: Server Side

     This proposal would make streams start faster by allowing
     clients to send DATA cells immediately, without waiting for a
     CONNECTED cell.  There's a patch under review as trac ticket
     #1795: it has some issues that need to get fixed before we can
     merge it.  I really want to get it fixed up soon, and merged in
     the next week or two.


=====  OPEN:

   143  Improvements of Distributed Storage for Tor Hidden Service Descriptors

     Here's a proposal from Karsten about making the hidden service
     DHT more reliable and secure to use.  It could use more
     discussion and analysis.

   145  Separate "suitable as a guard" from "suitable as a new guard"

     Currently, the Guard flag means both "You can use this node as
     a guard if you need to pick a new guard" and "If this node is
     currently your guard, you can keep using it as a guard."  This
     proposal tries to separate these two concepts, so that clients
     can stop picking a router once it is full of existing clients
     using it as a guard, but the clients currently on it won't all
     drop it.

     It's not clear whether this has anonymity issues, and it's not
     clear whether the imagined performance gains are actually
     worthwhile.

   146  Add new flag to reflect losing-term stability

     From time to time we get the idea of having clients ship with a
     reasonably recent consensus (or a list of directory mirrors),
     so instead of bootstrapping from one of the authorities, they
     can bootstrap from a regular directory cache.  The problem here
     is that by the time the client is run, most of the directory
     mirrors will be down or will have changed their IP.  This
     proposal tries to address that.

     It needs analysis based on behavior of actual routers on the
     network to see whether it could work, and what parameters might
     work.

   156  Tracking blocked ports on the client side

     This proposal provides a way for clients to learn which ports
     they are (and aren't) able to connect to, and connect to the
     ones that work.  It comes with a patch, too.  It also lets
     routers track ports that _they_ can't connect to.

     I'm a little unconvinced that : most clients that have some ports
     blocked will need bridges, not just restriction to a smaller
     set of ports.  This could be good behind restrictive firewalls,
     though.

     The router-side part is a little iffy: routers that can't
     connect to each other violate one of our network topology
     assumptions, and even if we do want to track failed
     router->router connections, the routers need to be sure that
     they aren't fooled into trying to connect repeatedly to a
     series of nonexistent addresses in an attempt to make them
     believe that (say) they can't reach port 443.

     This one is a paradigmatic "open" proposal: it needs more
     discussion.  The patch probably also needs to be ported to
     0.2.3.x; it touches some code that has changed.

   158  Clients download consensus + microdescriptors

     This proposal provides a way for clients to download
     authority-provided summaries of router descriptors instead of
     the descriptors themselves; I'm working on this one; it's going
     in 0.2.3.x.  It needs some cleanup, but it's basically right.

     (NOTE TO SELF: Clean this up and move it to "Accepted!")

   159  Exit Scanning

     This is an overview of SoaT, with some ideas for how to
     integrate it into Tor.

     Mike, is this implemented? Done?  Superseded?  Still open?

     (NOTE TO SELF: I should remember to ask Mike if he doesn't
     notice the above.)

   162  Publish the consensus in multiple flavors

     This proposal describes a way for authorities to migrate from
     one format of consensus to another without bloating the
     consensus forever.  It is mostly implemented in 0.2.3.x, though
     some parts of it need to get work.  I should clean it up too.

     (NOTE TO SELF: Clean this up and move it to "Accepted!")

   163  Detecting whether a connection comes from a client

     This one describes a number of ways to distinguish clients from
     non-clients when rejecting single-hop connections.  Some are
     implemented; others are rejected; others are still
     discussable.  Somebody should figure out what happened here,
     and reopen discussion as interested.

   164  Reporting the status of server votes

     This proposal explains a way for authorities to provide a
     slightly more verbose document that relay operators can use to
     diagnose reasons that their router was or was not listed in the
     consensus.  These documents would be like slightly more verbose
     versions of the authorities' votes, and would explain *why* the
     authority voted as it did.  It wouldn't be too hard to
     implement, and would be a fine project for somebody who wants
     to get to know the directory code.

   165  Easy migration for voting authority sets

     This is a design for how to change the set of authorities
     without having a flag day where the authority operators all
     reconfigure their authorities at once.  It needs more
     discussion.  One difficulty here is that we aren't talking much
     about changing the set of authorities, but that may be a
     chicken-and-egg issue, since changing the set is so onerous.

     If anybody is interested, it would be great to move the
     discussion ahead here.

   168  Reduce default circuit window

     This proposal reduces the default window for circuit sendme
     cells.  I think it's implemented, isn't it?  If so, we should
     make sure that tor-spec.txt is updated and close it.

     (NOTE TO SELF: Ask Roger; This should probably be called
     "Finished", or "Closed" if we have updated the spec.)

   171  Separate streams across circuits by connection metadata

     This proposal describes a way to prevent cross-application
     linking of different anonymized sessions by isolating different
     kinds of streams on different circuits based on their
     characteristics.

     I want this one in 0.2.3.x; it would improve client anonymity.
     A patch would be great; there are efficiency issues to
     consider, though.

     ACTION: This should be "accepted", I believe, unless somebody
     has lingering objections.

   176  Proposed version-3 link handshake for Tor

     Here's my proposal for how to improve our TLS handshake to
     eliminate renegotiation, become less fingerprintable, and
     generally make the world a better place.  It needs more review
     by actual crypto people, but I would like to get it into
     0.2.3.x.

   177  Abstaining from votes on individual flags

     Here's my proposal for letting authorities have opinions about
     some (flag,router) combinations without voting on whether
     _every_ router should have that flag.  It's simple, and I think
     it's basically right.  With more discussion and review,
     somebody could/should build it for 0.2.3.x, I think.

===== DRAFT:

   127  Relaying dirport requests to Tor download site / website

    The idea here was to make it easier to fetch and learn about Tor
    by making it easy for relays to automatically act as proxies to
    the Tor website.  It needs more discussion, and there are some
    significant details to work out.  It's not at all clear whether
    this is actually a good idea or not.

   132  A Tor Web Service For Verifying Correct Browser Configuration

    This proposal was meant to give users a way to see if their
    browser and privoxy (yes, it was a while ago) are correctly
    configured by running a local webserver on 127.0.0.1.  I'm not
    sure the status here.

   133  Incorporate Unreachable ORs into the Tor Network

    This proposal had an idea for letting ORs that can only make
    outgoing connections still relay data usefully in the network.
    It's something we should keep in mind, and it's a pretty neat
    idea, but it radically changes the network topology.  Anybody
    who wants to analyze new network topologies should definitely
    have a look.

   141  Download server descriptors on demand

    The idea of this proposal was for clients to only download the
    consensus, and later ask nodes for their own server descriptors
    while building the circuit through them.  It would make each
    circuit more time-consuming to build, but make bootstrapping
    much cheaper.

    A microdescriptor-based version of this would be even better,
    and microdescriptors would solve a lot of the problem that this
    was meant to resolve.  It still is not wholly superseded by the
    microdescriptor system, though.

   144  Increase the diversity of circuits by detecting nodes
belonging the same provider

    This is a version of the good idea, "Let's do routing in a way
    that tries to keep from routing traffic through the same
    provider too much!"  There are complex issues here that the
    proposal doesn't completely address, but I think it might be a
    fine idea for somebody to see how much more we know now than we
    did in 2008, particularly in light of the relevant paper(s) by
    Matt Edmann and Paul Syverson.

   149  Using data from NETINFO cells

    NETINFO cells currently contain time and address information
    that could be used to estimate skew and provide yet another way
    to learn our IP address.  This proposal starts to discuss ways
    that we could try to use that information safely, but doesn't
    really get anywhere.  See trac issues #2628 for some discussion
    about this stuff.  It would be neat if we had a working design
    here, though I suspect that with the slow and grinding death of
    older operating systems, systems with radically skewed clocks
    may become a thing of the past.

   170  Configuration options regarding circuit building

    This is Sebastian's take on how the *Nodes options should work.
    Roger preferred a more minimal approach that may or may not be
    right; see the discussion starting at
    https://trac.torproject.org/projects/tor/ticket/1090#comment:21
    for his approach.  I'm not going to call this "superseded" or
    "rejected" until  my 1090 code is done, though.  :(

   175  Automatically promoting Tor clients to nodes

    Here's Steven's proposal for adding a mode between "client mode"
    and "relay mode" for "self-test to see if you would be a good
    relay, and if so become one."  It didn't get enough attention
    when it was posted to the list; more people should review it.

   178  Require majority of authorities to vote for consensus parameters

    This is a proposal for making it a little harder for a small
    number of authorities to influence the value of a consensus
    parameter.  It needs more discussion; Sebastian has a draft
    implementation in his "safer_params" branch.

    (NOTE TO SELF: I think this could fairly become "open"; I should ask
    Sebastian.)

=====  NEEDS-REVISION:
   131  Help users to verify they are using Tor

     Here's a proposal for making a torcheck-like website more
     reliable.  If anybody wants to pick it up (especially somebody
     working on torcheck) and see whether it should be reopened or
     rejected, that would be a fine thing.

======  "IDEAS":

   xxx-bwrate-algs.txt

     Here's a note about better round-robin calculations for rate
     limiting.  Mike wrote it; I don't understand what's going on
     with it.

   xxx-choosing-crypto-in-tor-protocol.txt

     Here's the start of a considerations document that Marian was
     writing about how to pick new crypto algorithms to migrate to.

   xxx-controllers-intercept-extends.txt

     Here's an old idea from Geoff Goodell about letting controllers
     intercept EXTEND commands.  It is partially superseded by
     Damian's 172, though there is an additional feature that
     doesn't make sense for the Tor network, but might make sense
     for experimental stuff like Geoff was working on.

   xxx-crypto-migration.txt

     Here's the document I wrote in December about which parts of
     our crypto there are to migrate, and what might be involved.
     This isn't a draft or pre-draft of a design proposal; it is
     more of a new category of "survey of stuff we need to design
     and think about", or "proposal for proposals" or something.

   xxx-crypto-requirements.txt

     Here's Robert Ransom's draft for what crypto properties,
     exactly, we're trying to get out of our circuit crypto.
     It is not a proposal draft or pre-draft: it is again a "design
     considerations" document, and one worth reading.

   xxx-draft-spec-for-TLS-normalization.txt

     Here's Jacob's proposal for certificate normalization.  It
     should get renamed, given a proposal number, and called "Open"
     or "Draft" depending on how much the details are likely to
     change.

   xxx-encrypted-services.txt

     This is a start-of-a-proposal of Rogers that somebody should
     pick up and finish some time: The idea is to use the hidden
     service mechanism to provide a secure naming and encrypted
     connection facility for hosting sites that do not actually want
     anonymity themselves.  There's been more interest in this topic
     lately.  It might also turn into "exit enclaves done right".

   xxx-exit-scanning-outline.txt

     Looks like this was superseded by 159?

   xxx-geoip-survey-plan.txt

     Here's an old document I wrote a while ago about tracking usage
     by country.  Probably it should go into the metrics
     documentation somewhere (if we do this), or get thrown into
     "old" (if we won't do this), or updated (if we might someday do
     this).

   xxx-grand-scaling-plan.txt

     Here are some notes on scaling that Roger wrote in 2008.  There
     might be some smart ideas here!  Have a look some time.

   xxx-hide-platform.txt

     Here's a proposal pre-draft that says we should normalize the
     platform string.  Somebody could turn this into a proposal
     pretty easily.

   xxx-ipv6-plan.txt

     Here's a survey of what we need to do for IPv6 support that
     I started writing.  It's not a proposal; it's a survey of
     proposal and implementation statuses.

   xxx-pluggable-transport.txt

     Here's the thing Jacob and I have been working on for pluggable
     obfuscation techniques.  It should get turned into "Draft"
     status asap, if not "Open".  We should finish writing the
     missing sections, like, yesterday.

   xxx-port-knocking.txt

     Here's a pre-proposal from Jacob about using port knocking to
     make bridges harder to fingerprint.  This could be a good idea
     for somebody to do in terms of the pluggable transport spec.

   xxx-rate-limit-exits.txt

     Here are some notes of Roger's from 2008 claiming that we
     should rate-limit stream creation at exit nodes.  It could help
     avoid port-scans if we do it right, but we would need to be
     exceedingly careful not to disrupt useful traffic.

   xxx-using-spdy.txt

     Here's a document from Steven about opportunistic use of SPDY
     over Tor.

     This should be a "Draft" proposal IMO.

   xxx-what-uses-sha1.txt

     Here's the beginnings of a survey of where Tor uses SHA1, with
     an eye to stopping.  This really isn't a design proposal; it
     might fall into a new category of "issue survey" or
     "information" or something.