[or-cvs] r18163: {} some notes on how exactly to do this microdescriptor thing. (tor/trunk/doc/spec/proposals/ideas)
arma at seul.org
arma at seul.org
Sun Jan 18 09:51:09 UTC 2009
Author: arma
Date: 2009-01-18 04:51:09 -0500 (Sun, 18 Jan 2009)
New Revision: 18163
Added:
tor/trunk/doc/spec/proposals/ideas/xxx-microdescriptors.txt
Log:
some notes on how exactly to do this microdescriptor thing.
Added: tor/trunk/doc/spec/proposals/ideas/xxx-microdescriptors.txt
===================================================================
--- tor/trunk/doc/spec/proposals/ideas/xxx-microdescriptors.txt (rev 0)
+++ tor/trunk/doc/spec/proposals/ideas/xxx-microdescriptors.txt 2009-01-18 09:51:09 UTC (rev 18163)
@@ -0,0 +1,194 @@
+Filename: xxx-microdescriptors.txt
+Title: Clients download consensus + microdescriptors
+Version: $Revision$
+Last-Modified: $Date$
+Author: Roger Dingledine
+Created: 17-Jan-2009
+Status: Open
+
+1. Overview
+
+ This proposal replaces section 3.2 of proposal 141, called "Fetching
+ descriptors on demand". Rather than modifying the circuit-building
+ protocol to fetch a server descriptor inline at each circuit extend,
+ we instead put all of the information that clients need either into
+ the consensus itself, or into a new set of data about each relay
+ called a microdescriptor. The goal is that descriptor elements that
+ are small and frequently changing should go in the consensus itself,
+ descriptor elements that are small and relatively static should go in
+ the microdescriptor, and if we ever end up with descriptor elements
+ that aren't small yet clients need to know them, we'll need to resume
+ considering some design like the one in proposal 141.
+
+2. Motivation
+
+ See
+ http://archives.seul.org/or/dev/Nov-2008/msg00000.html and
+ http://archives.seul.org/or/dev/Nov-2008/msg00001.html and especially
+ http://archives.seul.org/or/dev/Nov-2008/msg00007.html
+ for a discussion of the options and why this is currently the best
+ approach.
+
+3. Design
+
+ There are three pieces to the proposal. First, authorities will list
+ in their votes (and thus in the consensus) what relay elements are
+ included in the microdescriptor, and also list the expected hash of
+ microdescriptor for each relay. Second, directory mirrors will serve
+ microdescriptors. Third, clients will ask for them and then cache them.
+
+3.1. Consensus changes
+
+ V3 votes should include a new line:
+ microdescriptor-elements bar baz foo
+
+ We also need to include the hash of each expected microdescriptor in
+ the routerstatus section. I suggest a new "m" line for each stanza,
+ with the base64 of the hash of the elements that the authority voted
+ for above.
+
+ The consensus microdescriptor-elements and "m" lines are then computed
+ as described in Section 3.1.2 below.
+
+ I believe that means we need a new consensus-method "6" that knows
+ how to compute the microdescriptor-elements and add "m" lines.
+
+3.1.1. Descriptor elements to include for now
+
+ To start, the element list that authorities suggest should be
+ family onion-key
+
+ (Note that the or-dev posts above only mention onion-key, but if
+ we don't also include family then clients will never learn it. It
+ seemed like it should be relatively static, so putting it in the
+ microdescriptor is smarter than trying to fit it into the consensus.)
+
+3.1.2. Computing consensus for microdescriptor-elements and "m" lines
+
+ One approach is for the consensus microdescriptor-elements line to
+ include all elements listed by a majority of authorities, sorted. The
+ problem here is that it will no longer be deterministic what the correct
+ hash for the "m" line should be. We could imagine telling the authority
+ to go look in its descriptor and produce the right hash itself, but
+ we don't want consensus calculation to be based on external data like
+ that. (Plus, the authority may not have the descriptor that everybody
+ else voted to use.)
+
+ The better approach is to take the exact set that has the most votes
+ (breaking ties by the set that has the most elements, and breaking
+ ties after that by whichever is alphabetically first). That will
+ increase the odds that we actually get a microdescriptor hash that
+ is both a) for the descriptor we're putting in the consensus, and b)
+ over the elements that we're declaring it should be for.
+
+ Then the "m" line for a given relay is the one that gets the most votes
+ from authorities that a) voted for the microdescriptor-elements line
+ we're using, and b) voted for the descriptor we're using.
+
+ (If there's a tie, use the smaller hash. But really, if there are
+ multiple such votes and they differ about a microdescriptor, we caught
+ one of them being lying or buggy. We should log it to track down why.)
+
+ If there are no such votes, then we leave out the "m" line for that
+ relay. That means clients should avoid it for this time period. (As
+ an extension it could instead mean that clients should fetch the
+ descriptor and figure out its microdescriptor themselves. But let's
+ not get ahead of ourselves.)
+
+ It would be nice to have a more foolproof way to agree on what
+ microdescriptor hash each authority should vote for, so we can avoid
+ missing "m" lines. Just switching to a new consensus-method each time
+ we change the set of microdescriptor-elements won't help though, since
+ each authority will still have to decide what hash to vote for before
+ knowing what consensus-method will be used.
+
+ Here's one way we could do that. Each vote / consensus includes both
+ the microdescriptor-elements that were used to compute the hashes,
+ and also a preferred-microdescriptor-elements set. If an authority
+ has a consensus from the previous period, then it should use the
+ consensus preferred-microdescriptor-elements when computing its votes
+ for microdescriptor-elements and the appropriate hashes in the upcoming
+ period. (If it has no previous consensus, then it just puts down its
+ own preferences in both lines.)
+
+3.2. Directory mirrors serve microdescriptors
+
+ Directory mirrors should then read the microdescriptor-elements line
+ from the consensus, and learn how to answer requests.
+
+ The microdescriptors with hashes <D1>,<D2>,<D3> should be available at:
+ http://<hostname>/tor/micro/d/<D1>+<D2>+<D3>.z
+
+ All the microdescriptors from the current consensus should also be
+ available at:
+ http://<hostname>/tor/micro/all.z
+ so a client that's bootstrapping doesn't need to send a 70KB URL just
+ to name every microdescriptor it's looking for.
+
+ The format of a microdescriptor is the header line
+ "microdescriptor 1"
+ followed by each element (keyword and body), alphabetically. There's
+ no need to mention what hash it is, since you can hash the elements
+ to learn this.
+
+ (Do we need a footer line to show that it's over, or is the next
+ microdescriptor line or EOF enough of a hint? A footer line wouldn't
+ hurt much. Also, no fair voting for the microdescriptor-element
+ "microdescriptor".)
+
+ The hash of the microdescriptor is simply the hash of the concatenated
+ elements -- not counting the header line or hypothetical footer line.
+ Is this smart?
+
+ Note that I put a "1" up there in the header line. It isn't part
+ of what's hashed, though. Is there a way to put in a version that's
+ more useful?
+
+ Directory mirrors should check to make sure that the microdescriptors
+ they're about to serve match the right hashes (either the hashes from
+ the fetch URL or the hashes from the consensus, respectively).
+
+ We will probably want to consider some sort of smart data structure to
+ be able to quickly convert microdescriptor hashes into the appropriate
+ microdescriptor. Clients will want this anyway when they load their
+ microdescriptor cache and want to match it up with the consensus to
+ see what's missing.
+
+3.3. Clients fetch them and cache them
+
+ When a client gets a new consensus, it looks to see if there are any
+ microdescriptors it needs to learn. If it needs to learn more than
+ some threshold of the microdescriptors (half?), it requests 'all',
+ else it requests only the missing ones.
+
+ Clients maintain a cache of microdescriptors along with metadata like
+ when it was last referenced by a consensus. They keep a microdescriptor
+ until it hasn't been mentioned in any consensus for a week.
+
+3.3.1. Information leaks from clients
+
+ If a client asks you for a set of microdescs, then you know she didn't
+ have them cached before. How much does that leak? What about when
+ we're all using our entry guards as directory guards, and we've seen
+ that user make a bunch of circuits already?
+
+ Fetching "all" when you need at least half is a good first order fix,
+ but might not be all there is to it.
+
+4. Transition and deployment
+
+ Phase one, the directory authorities should start voting on
+ microdescriptors and microdescriptor elements, and putting them in the
+ consensus. This should happen during the 0.2.1.x series, and should
+ be relatively easy to do.
+
+ Phase two, directory mirrors should learn how to serve them, and learn
+ how to read the consensus to find out what they should be serving. It
+ would be great if we can squeeze this in during 0.2.1.x also, so once
+ clients start to fetch them there will be many mirrors to choose from.
+
+ (Are there reasonable ways to build only part of phase two in 0.2.1.x?)
+
+ Phase three, clients should start fetching and caching them instead
+ of normal descriptors. This should happen post 0.2.1.x.
+
Property changes on: tor/trunk/doc/spec/proposals/ideas/xxx-microdescriptors.txt
___________________________________________________________________
Added: svn:keywords
+ Author Date Id Revision
Added: svn:eol-style
+ native
More information about the tor-commits
mailing list