commit 7bd906b6ecef7a0dcf3b420944da0d08db5a2553 Author: Nick Mathewson nickm@torproject.org Date: Thu Jul 17 17:31:59 2014 +0200
Move dir-spec-v2 to attic. (#12645) --- attic/dir-spec-v2.txt | 877 +++++++++++++++++++++++++++++++++++++++++++++++++ dir-spec-v2.txt | 877 ------------------------------------------------- 2 files changed, 877 insertions(+), 877 deletions(-)
diff --git a/attic/dir-spec-v2.txt b/attic/dir-spec-v2.txt new file mode 100644 index 0000000..021532e --- /dev/null +++ b/attic/dir-spec-v2.txt @@ -0,0 +1,877 @@ + + Tor directory protocol, version 2 + +0. Scope and preliminaries + + This directory protocol is used by Tor version 0.1.1.x and 0.1.2.x. See + dir-spec-v1.txt for information on earlier versions, and dir-spec.txt + for information on later versions. + +0.1. Goals and motivation + + There were several problems with the way Tor handles directory information + in version 0.1.0.x and earlier. Here are the problems we try to fix with + this new design, already implemented in 0.1.1.x: + 1. Directories were very large and use up a lot of bandwidth: clients + downloaded descriptors for all router several times an hour. + 2. Every directory authority was a trust bottleneck: if a single + directory authority lied, it could make clients believe for a time an + arbitrarily distorted view of the Tor network. + 3. Our current "verified server" system is kind of nonsensical. + + 4. Getting more directory authorities would add more points of failure + and worsen possible partitioning attacks. + + There are two problems that remain unaddressed by this design. + 5. Requiring every client to know about every router won't scale. + 6. Requiring every directory cache to know every router won't scale. + + We attempt to fix 1-4 here, and to build a solution that will work when we + figure out an answer for 5. We haven't thought at all about what to do + about 6. + +1. Outline + + There is a small set (say, around 10) of semi-trusted directory + authorities. A default list of authorities is shipped with the Tor + software. Users can change this list, but are encouraged not to do so, in + order to avoid partitioning attacks. + + Routers periodically upload signed "descriptors" to the directory + authorities describing their keys, capabilities, and other information. + Routers may act as directory mirrors (also called "caches"), to reduce + load on the directory authorities. They announce this in their + descriptors. + + Each directory authority periodically generates and signs a compact + "network status" document that lists that authority's view of the current + descriptors and status for known routers, but which does not include the + descriptors themselves. + + Directory mirrors download, cache, and re-serve network-status documents + to clients. + + Clients, directory mirrors, and directory authorities all use + network-status documents to find out when their list of routers is + out-of-date. If it is, they download any missing router descriptors. + Clients download missing descriptors from mirrors; mirrors and authorities + download from authorities. Descriptors are downloaded by the hash of the + descriptor, not by the server's identity key: this prevents servers from + attacking clients by giving them descriptors nobody else uses. + + All directory information is uploaded and downloaded with HTTP. + + Coordination among directory authorities is done client-side: clients + compute a vote-like algorithm among the network-status documents they + have, and base their decisions on the result. + +1.1. What's different from 0.1.0.x? + + Clients used to download a signed concatenated set of router descriptors + (called a "directory") from directory mirrors, regardless of which + descriptors had changed. + + Between downloading directories, clients would download "network-status" + documents that would list which servers were supposed to running. + + Clients would always believe the most recently published network-status + document they were served. + + Routers used to upload fresh descriptors all the time, whether their keys + and other information had changed or not. + +1.2. Document meta-format + + Router descriptors, directories, and running-routers documents all obey the + following lightweight extensible information format. + + The highest level object is a Document, which consists of one or more + Items. Every Item begins with a KeywordLine, followed by one or more + Objects. A KeywordLine begins with a Keyword, optionally followed by + whitespace and more non-newline characters, and ends with a newline. A + Keyword is a sequence of one or more characters in the set [A-Za-z0-9-]. + An Object is a block of encoded data in pseudo-Open-PGP-style + armor. (cf. RFC 2440) + + More formally: + + Document ::= (Item | NL)+ + Item ::= KeywordLine Object* + KeywordLine ::= Keyword NL | Keyword WS ArgumentsChar+ NL + Keyword = KeywordChar+ + KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-' + ArgumentChar ::= any printing ASCII character except NL. + WS = (SP | TAB)+ + Object ::= BeginLine Base64-encoded-data EndLine + BeginLine ::= "-----BEGIN " Keyword "-----" NL + EndLine ::= "-----END " Keyword "-----" NL + + The BeginLine and EndLine of an Object must use the same keyword. + + When interpreting a Document, software MUST ignore any KeywordLine that + starts with a keyword it doesn't recognize; future implementations MUST NOT + require current clients to understand any KeywordLine not currently + described. + + Other implementations that want to extend Tor's directory format MAY + introduce their own items. The keywords for extension items SHOULD start + with the characters "x-" or "X-", to guarantee that they will not conflict + with keywords used by future versions of Tor. + +2. Router operation + + ORs SHOULD generate a new router descriptor whenever any of the + following events have occurred: + + - A period of time (18 hrs by default) has passed since the last + time a descriptor was generated. + + - A descriptor field other than bandwidth or uptime has changed. + + - Bandwidth has changed by at least a factor of 2 from the last time a + descriptor was generated, and at least a given interval of time + (20 mins by default) has passed since then. + + - Its uptime has been reset (by restarting). + + After generating a descriptor, ORs upload it to every directory + authority they know, by posting it to the URL + + http://hostname:port/tor/ + +2.1. Router descriptor format + + Every router descriptor MUST start with a "router" Item; MUST end with a + "router-signature" Item and an extra NL; and MUST contain exactly one + instance of each of the following Items: "published" "onion-key" + "signing-key" "bandwidth". + + A router descriptor MAY have zero or one of each of the following Items, + but MUST NOT have more than one: "contact", "uptime", "fingerprint", + "hibernating", "read-history", "write-history", "eventdns", "platform", + "family". + + For historical reasons some options are prefixed with "opt ", for instance + "opt fingerprint". This "opt " prefix should be ignored with the second word + used as the keyword instead. + + Additionally, a router descriptor MAY contain any number of "accept", + "reject", and "opt" Items. Other than "router" and "router-signature", + the items may appear in any order. + + The items' formats are as follows: + "router" nickname address ORPort SocksPort DirPort + + Indicates the beginning of a router descriptor. "address" must be an + IPv4 address in dotted-quad format. The last three numbers indicate + the TCP ports at which this OR exposes functionality. ORPort is a port + at which this OR accepts TLS connections for the main OR protocol; + SocksPort is deprecated and should always be 0; and DirPort is the + port at which this OR accepts directory-related HTTP connections. If + any port is not supported, the value 0 is given instead of a port + number. + + "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed + + Estimated bandwidth for this router, in bytes per second. The + "average" bandwidth is the volume per second that the OR is willing to + sustain over long periods; the "burst" bandwidth is the volume that + the OR is willing to sustain in very short intervals. The "observed" + value is an estimate of the capacity this server can handle. The + server remembers the max bandwidth sustained output over any ten + second period in the past day, and another sustained input. The + "observed" value is the lesser of these two numbers. + + "platform" string + + A human-readable string describing the system on which this OR is + running. This MAY include the operating system, and SHOULD include + the name and version of the software implementing the Tor protocol. + + "published" YYYY-MM-DD HH:MM:SS + + The time, in GMT, when this descriptor was generated. + + "fingerprint" + + A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded in + hex, with a single space after every 4 characters) for this router's + identity key. A descriptor is considered invalid (and MUST be + rejected) if the fingerprint line does not match the public key. + + "hibernating" 0|1 + + If the value is 1, then the Tor server was hibernating when the + descriptor was published, and shouldn't be used to build circuits. + + "uptime" + + The number of seconds that this OR process has been running. + + "onion-key" NL a public key in PEM format + + This key is used to encrypt EXTEND cells for this OR. The key MUST be + accepted for at least 1 week after any new key is published in a + subsequent descriptor. + + "signing-key" NL a public key in PEM format + + The OR's long-term identity key. + + "accept" exitpattern + "reject" exitpattern + + These lines describe the rules that an OR follows when + deciding whether to allow a new stream to a given address. The + 'exitpattern' syntax is described below. The rules are considered in + order; if no rule matches, the address will be accepted. For clarity, + the last such entry SHOULD be accept *:* or reject *:*. + + "router-signature" NL Signature NL + + The "SIGNATURE" object contains a signature of the PKCS1-padded + hash of the entire router descriptor, taken from the beginning of the + "router" line, through the newline after the "router-signature" line. + The router descriptor is invalid unless the signature is performed + with the router's identity key. + + "contact" info NL + + Describes a way to contact the server's administrator, preferably + including an email address and a PGP key fingerprint. + + "family" names NL + + 'Names' is a space-separated list of server nicknames or + hexdigests. If two ORs list one another in their "family" entries, + then OPs should treat them as a single OR for the purpose of path + selection. + + For example, if node A's descriptor contains "family B", and node B's + descriptor contains "family A", then node A and node B should never + be used on the same circuit. + + "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL + "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL + + Declare how much bandwidth the OR has used recently. Usage is divided + into intervals of NSEC seconds. The YYYY-MM-DD HH:MM:SS field + defines the end of the most recent interval. The numbers are the + number of bytes used in the most recent intervals, ordered from + oldest to newest. + + "eventdns" bool NL + + Declare whether this version of Tor is using the newer enhanced + dns logic. Versions of Tor without eventdns SHOULD NOT be used for + reverse hostname lookups. + + [All versions of Tor before 0.1.2.2-alpha should be assumed to have + this option set to 0 if it is not present. All Tor versions at + 0.1.2.2-alpha or later should be assumed to have this option set to + 1 if it is not present. Until 0.1.2.1-alpha-dev, this option was + not generated, even when eventdns was in use. With 0.2.0.1-alpha, the + old 'dnsworker' logic has been removed, rendering this option of + historical interest only.] + +2.2. Nonterminals in router descriptors + + nickname ::= between 1 and 19 alphanumeric characters, case-insensitive. + hexdigest ::= a '$', followed by 20 hexadecimal characters. + [Represents a server by the digest of its identity key.] + + exitpattern ::= addrspec ":" portspec + portspec ::= "*" | port | port "-" port + port ::= an integer between 1 and 65535, inclusive. + [Some implementations incorrectly generate ports with value 0. + Implementations SHOULD accept this, and SHOULD NOT generate it.] + + addrspec ::= "*" | ip4spec | ip6spec + ipv4spec ::= ip4 | ip4 "/" num_ip4_bits | ip4 "/" ip4mask + ip4 ::= an IPv4 address in dotted-quad format + ip4mask ::= an IPv4 mask in dotted-quad format + num_ip4_bits ::= an integer between 0 and 32 + ip6spec ::= ip6 | ip6 "/" num_ip6_bits + ip6 ::= an IPv6 address, surrounded by square brackets. + num_ip6_bits ::= an integer between 0 and 128 + + bool ::= "0" | "1" + + Ports are required; if they are not included in the router + line, they must appear in the "ports" lines. + +3. Network status format + + Directory authorities generate, sign, and compress network-status + documents. Directory servers SHOULD generate a fresh network-status + document when the contents of such a document would be different from the + last one generated, and some time (at least one second, possibly longer) + has passed since the last one was generated. + + The network status document contains a preamble, a set of router status + entries, and a signature, in that order. + + We use the same meta-format as used for directories and router descriptors + in "tor-spec.txt". Implementations MAY insert blank lines + for clarity between sections; these blank lines are ignored. + Implementations MUST NOT depend on blank lines in any particular location. + + As used here, "whitespace" is a sequence of 1 or more tab or space + characters. + + The preamble contains: + + "network-status-version" -- A document format version. For this + specification, the version is "2". + "dir-source" -- The authority's hostname, current IP address, and + directory port, all separated by whitespace. + "fingerprint" -- A base16-encoded hash of the signing key's + fingerprint, with no additional spaces added. + "contact" -- An arbitrary string describing how to contact the + directory server's administrator. Administrators should include at + least an email address and a PGP fingerprint. + "dir-signing-key" -- The directory server's public signing key. + "client-versions" -- A comma-separated list of recommended client + versions. + "server-versions" -- A comma-separated list of recommended server + versions. + "published" -- The publication time for this network-status object. + "dir-options" -- A set of flags, in any order, separated by whitespace: + "Names" if this directory authority performs name bindings. + "Versions" if this directory authority recommends software versions. + "BadExits" if the directory authority flags nodes that it believes + are performing incorrectly as exit nodes. + "BadDirectories" if the directory authority flags nodes that it + believes are performing incorrectly as directory caches. + + The dir-options entry is optional. The "-versions" entries are required if + the "Versions" flag is present. The other entries are required and must + appear exactly once. The "network-status-version" entry must appear first; + the others may appear in any order. Implementations MUST ignore + additional arguments to the items above, and MUST ignore unrecognized + flags. + + For each router, the router entry contains: (This format is designed for + conciseness.) + + "r" -- followed by the following elements, in order, separated by + whitespace: + - The OR's nickname, + - A hash of its identity key, encoded in base64, with trailing = + signs removed. + - A hash of its most recent descriptor, encoded in base64, with + trailing = signs removed. (The hash is calculated as for + computing the signature of a descriptor.) + - The publication time of its most recent descriptor, in the form + YYYY-MM-DD HH:MM:SS, in GMT. + - An IP address + - An OR port + - A directory port (or "0" for none") + "s" -- A series of whitespace-separated status flags, in any order: + "Authority" if the router is a directory authority. + "BadExit" if the router is believed to be useless as an exit node + (because its ISP censors it, because it is behind a restrictive + proxy, or for some similar reason). + "BadDirectory" if the router is believed to be useless as a + directory cache (because its directory port isn't working, + its bandwidth is always throttled, or for some similar + reason). + "Exit" if the router is useful for building general-purpose exit + circuits. + "Fast" if the router is suitable for high-bandwidth circuits. + "Guard" if the router is suitable for use as an entry guard. + "Named" if the router's identity-nickname mapping is canonical, + and this authority binds names. + "Stable" if the router is suitable for long-lived circuits. + "Running" if the router is currently usable. + "Valid" if the router has been 'validated'. + "V2Dir" if the router implements this protocol. + "v" -- The version of the Tor protocol that this server is running. If + the value begins with "Tor" SP, the rest of the string is a Tor + version number, and the protocol is "The Tor protocol as supported + by the given version of Tor." Otherwise, if the value begins with + some other string, Tor has upgraded to a more sophisticated + protocol versioning system, and the protocol is "a version of the + Tor protocol more recent than any we recognize." + + The "r" entry for each router must appear first and is required. The + "s" entry is optional (see Section 3.1 below for how the flags are + decided). Unrecognized flags on the "s" line and extra elements + on the "r" line must be ignored. The "v" line is optional. + + The signature section contains: + + "directory-signature" nickname-of-dirserver NL Signature + + Signature is a signature of this network-status document + (the document up until the signature, including the line + "directory-signature <nick>\n"), using the directory authority's + signing key. + + We compress the network status list with zlib before transmitting it. + +3.1. Establishing server status + + (This section describes how directory authorities choose which status + flags to apply to routers, as of Tor 0.1.1.18-rc. Later directory + authorities MAY do things differently, so long as clients keep working + well. Clients MUST NOT depend on the exact behaviors in this section.) + + In the below definitions, a router is considered "active" if it is + running, valid, and not hibernating. + + "Valid" -- a router is 'Valid' if it is running a version of Tor not + known to be broken, and the directory authority has not blacklisted + it as suspicious. + + "Named" -- Directory authority administrators may decide to support name + binding. If they do, then they must maintain a file of + nickname-to-identity-key mappings, and try to keep this file consistent + with other directory authorities. If they don't, they act as clients, and + report bindings made by other directory authorities (name X is bound to + identity Y if at least one binding directory lists it, and no directory + binds X to some other Y'.) A router is called 'Named' if the router + believes the given name should be bound to the given key. + + "Running" -- A router is 'Running' if the authority managed to connect to + it successfully within the last 30 minutes. + + "Stable" -- A router is 'Stable' if it is active, and either its + uptime is at least the median uptime for known active routers, or + its uptime is at least 30 days. Routers are never called stable if + they are running a version of Tor known to drop circuits stupidly. + (0.1.1.10-alpha through 0.1.1.16-rc are stupid this way.) + + "Fast" -- A router is 'Fast' if it is active, and its bandwidth is + in the top 7/8ths for known active routers. + + "Guard" -- A router is a possible 'Guard' if it is 'Stable' and its + bandwidth is above median for known active routers. If the total + bandwidth of active non-BadExit Exit servers is less than one third + of the total bandwidth of all active servers, no Exit is listed as + a Guard. + + "Authority" -- A router is called an 'Authority' if the authority + generating the network-status document believes it is an authority. + + "V2Dir" -- A router supports the v2 directory protocol if it has an open + directory port, and it is running a version of the directory protocol that + supports the functionality clients need. (Currently, this is + 0.1.1.9-alpha or later.) + + Directory server administrators may label some servers or IPs as + blacklisted, and elect not to include them in their network-status lists. + + Authorities SHOULD 'disable' any servers in excess of 3 on any single IP. + When there are more than 3 to choose from, authorities should first prefer + authorities to non-authorities, then prefer Running to non-Running, and + then prefer high-bandwidth to low-bandwidth. To 'disable' a server, the + authority *should* advertise it without the Running or Valid flag. + + Thus, the network-status list includes all non-blacklisted, + non-expired, non-superseded descriptors. + +4. Directory server operation + + All directory authorities and directory mirrors ("directory servers") + implement this section, except as noted. + +4.1. Accepting uploads (authorities only) + + When a router posts a signed descriptor to a directory authority, the + authority first checks whether it is well-formed and correctly + self-signed. If it is, the authority next verifies that the nickname + in question is not already assigned to a router with a different + public key. + Finally, the authority MAY check that the router is not blacklisted + because of its key, IP, or another reason. + + If the descriptor passes these tests, and the authority does not already + have a descriptor for a router with this public key, it accepts the + descriptor and remembers it. + + If the authority _does_ have a descriptor with the same public key, the + newly uploaded descriptor is remembered if its publication time is more + recent than the most recent old descriptor for that router, and either: + - There are non-cosmetic differences between the old descriptor and the + new one. + - Enough time has passed between the descriptors' publication times. + (Currently, 12 hours.) + + Differences between router descriptors are "non-cosmetic" if they would be + sufficient to force an upload as described in section 2 above. + + Note that the "cosmetic difference" test only applies to uploaded + descriptors, not to descriptors that the authority downloads from other + authorities. + +4.2. Downloading network-status documents (authorities and caches) + + All directory servers (authorities and mirrors) try to keep a fresh + set of network-status documents from every authority. To do so, + every 5 minutes, each authority asks every other authority for its + most recent network-status document. Every 15 minutes, each mirror + picks a random authority and asks it for the most recent network-status + documents for all the authorities the authority knows about (including + the chosen authority itself). + + Directory servers and mirrors remember and serve the most recent + network-status document they have from each authority. Other + network-status documents don't need to be stored. If the most recent + network-status document is over 10 days old, it is discarded anyway. + Mirrors SHOULD store and serve network-status documents from authorities + they don't recognize, but SHOULD NOT use such documents for any other + purpose. Mirrors SHOULD discard network-status documents older than 48 + hours. + +4.3. Downloading and storing router descriptors (authorities and caches) + + Periodically (currently, every 10 seconds), directory servers check + whether there are any specific descriptors (as identified by descriptor + hash in a network-status document) that they do not have and that they + are not currently trying to download. + + If so, the directory server launches requests to the authorities for these + descriptors, such that each authority is only asked for descriptors listed + in its most recent network-status. When more than one authority lists the + descriptor, we choose which to ask at random. + + If one of these downloads fails, we do not try to download that descriptor + from the authority that failed to serve it again unless we receive a newer + network-status from that authority that lists the same descriptor. + + Directory servers must potentially cache multiple descriptors for each + router. Servers must not discard any descriptor listed by any current + network-status document from any authority. If there is enough space to + store additional descriptors, servers SHOULD try to hold those which + clients are likely to download the most. (Currently, this is judged + based on the interval for which each descriptor seemed newest.) + + Authorities SHOULD NOT download descriptors for routers that they would + immediately reject for reasons listed in 3.1. + +4.4. HTTP URLs + + "Fingerprints" in these URLs are base16-encoded SHA1 hashes. + + The authoritative network-status published by a host should be available at: + http://<hostname>/tor/status/authority.z + + The network-status published by a host with fingerprint + <F> should be available at: + http://<hostname>/tor/status/fp/<F>.z + + The network-status documents published by hosts with fingerprints + <F1>,<F2>,<F3> should be available at: + http://<hostname>/tor/status/fp/<F1>+<F2>+<F3>.z + + The most recent network-status documents from all known authorities, + concatenated, should be available at: + http://<hostname>/tor/status/all.z + + The most recent descriptor for a server whose identity key has a + fingerprint of <F> should be available at: + http://<hostname>/tor/server/fp/<F>.z + + The most recent descriptors for servers with identity fingerprints + <F1>,<F2>,<F3> should be available at: + http://<hostname>/tor/server/fp/<F1>+<F2>+<F3>.z + + (NOTE: Implementations SHOULD NOT download descriptors by identity key + fingerprint. This allows a corrupted server (in collusion with a cache) to + provide a unique descriptor to a client, and thereby partition that client + from the rest of the network.) + + The server descriptor with (descriptor) digest <D> (in hex) should be + available at: + http://<hostname>/tor/server/d/<D>.z + + The most recent descriptors with digests <D1>,<D2>,<D3> should be + available at: + http://<hostname>/tor/server/d/<D1>+<D2>+<D3>.z + + The most recent descriptor for this server should be at: + http://<hostname>/tor/server/authority.z + [Nothing in the Tor protocol uses this resource yet, but it is useful + for debugging purposes. Also, the official Tor implementations + (starting at 0.1.1.x) use this resource to test whether a server's + own DirPort is reachable.] + + A concatenated set of the most recent descriptors for all known servers + should be available at: + http://<hostname>/tor/server/all.z + + For debugging, directories SHOULD expose non-compressed objects at URLs like + the above, but without the final ".z". + Clients MUST handle compressed concatenated information in two forms: + - A concatenated list of zlib-compressed objects. + - A zlib-compressed concatenated list of objects. + Directory servers MAY generate either format: the former requires less + CPU, but the latter requires less bandwidth. + + Clients SHOULD use upper case letters (A-F) when base16-encoding + fingerprints. Servers MUST accept both upper and lower case fingerprints + in requests. + +5. Client operation: downloading information + + Every Tor that is not a directory server (that is, those that do + not have a DirPort set) implements this section. + +5.1. Downloading network-status documents + + Each client maintains an ordered list of directory authorities. + Insofar as possible, clients SHOULD all use the same ordered list. + + For each network-status document a client has, it keeps track of its + publication time *and* the time when the client retrieved it. Clients + consider a network-status document "live" if it was published within the + last 24 hours. + + Clients try to have a live network-status document hours from *every* + authority, and try to periodically get new network-status documents from + each authority in rotation as follows: + + If a client is missing a live network-status document for any + authority, it tries to fetch it from a directory cache. On failure, + the client waits briefly, then tries that network-status document + again from another cache. The client does not build circuits until it + has live network-status documents from more than half the authorities + it trusts, and it has descriptors for more than 1/4 of the routers + that it believes are running. + + If the most recently _retrieved_ network-status document is over 30 + minutes old, the client attempts to download a network-status document. + When choosing which documents to download, clients treat their list of + directory authorities as a circular ring, and begin with the authority + appearing immediately after the authority for their most recently + retrieved network-status document. If this attempt fails (either it + fails to download at all, or the one it gets is not as good as the + one it has), the client retries at other caches several times, before + moving on to the next network-status document in sequence. + + Clients discard all network-status documents over 24 hours old. + + If enough mirrors (currently 4) claim not to have a given network status, + we stop trying to download that authority's network-status, until we + download a new network-status that makes us believe that the authority in + question is running. Clients should wait a little longer after each + failure. + + Clients SHOULD try to batch as many network-status requests as possible + into each HTTP GET. + + (Note: clients can and should pick caches based on the network-status + information they have: once they have first fetched network-status info + from an authority, they should not need to go to the authority directly + again.) + +5.2. Downloading and storing router descriptors + + Clients try to have the best descriptor for each router. A descriptor is + "best" if: + * It is the most recently published descriptor listed for that router + by at least two network-status documents. + OR, + * No descriptor for that router is listed by two or more + network-status documents, and it is the most recently published + descriptor listed by any network-status document. + + Periodically (currently every 10 seconds) clients check whether there are + any "downloadable" descriptors. A descriptor is downloadable if: + - It is the "best" descriptor for some router. + - The descriptor was published at least 10 minutes in the past. + (This prevents clients from trying to fetch descriptors that the + mirrors have probably not yet retrieved and cached.) + - The client does not currently have it. + - The client is not currently trying to download it. + - The client would not discard it immediately upon receiving it. + - The client thinks it is running and valid (see 6.1 below). + + If at least 16 known routers have downloadable descriptors, or if + enough time (currently 10 minutes) has passed since the last time the + client tried to download descriptors, it launches requests for all + downloadable descriptors, as described in 5.3 below. + + When a descriptor download fails, the client notes it, and does not + consider the descriptor downloadable again until a certain amount of time + has passed. (Currently 0 seconds for the first failure, 60 seconds for the + second, 5 minutes for the third, 10 minutes for the fourth, and 1 day + thereafter.) Periodically (currently once an hour) clients reset the + failure count. + + No descriptors are downloaded until the client has downloaded more than + half of the network-status documents. + + Clients retain the most recent descriptor they have downloaded for each + router so long as it is not too old (currently, 48 hours), OR so long as + it is recommended by at least one networkstatus AND no "better" + descriptor has been downloaded. [Versions of Tor before 0.1.2.3-alpha + would discard descriptors simply for being published too far in the past.] + [The code seems to discard descriptors in all cases after they're 5 + days old. True? -RD] + +5.3. Managing downloads + + When a client has no live network-status documents, it downloads + network-status documents from a randomly chosen authority. In all other + cases, the client downloads from mirrors randomly chosen from among those + believed to be V2 directory servers. (This information comes from the + network-status documents; see 6 below.) + + When downloading multiple router descriptors, the client chooses multiple + mirrors so that: + - At least 3 different mirrors are used, except when this would result + in more than one request for under 4 descriptors. + - No more than 128 descriptors are requested from a single mirror. + - Otherwise, as few mirrors as possible are used. + After choosing mirrors, the client divides the descriptors among them + randomly. + + After receiving any response client MUST discard any network-status + documents and descriptors that it did not request. + +6. Using directory information + + Everyone besides directory authorities uses the approaches in this section + to decide which servers to use and what their keys are likely to be. + (Directory authorities just believe their own opinions, as in 3.1 above.) + +6.1. Choosing routers for circuits. + + Tor implementations only pay attention to "live" network-status documents. + A network status is "live" if it is the most recently downloaded network + status document for a given directory server, and the server is a + directory server trusted by the client, and the network-status document is + no more than 1 day old. + + For time-sensitive information, Tor implementations focus on "recent" + network-status documents. A network status is "recent" if it is live, and + if it was published in the last 60 minutes. If there are fewer + than 3 such documents, the most recently published 3 are "recent." If + there are fewer than 3 in all, all are "recent.") + + Circuits SHOULD NOT be built until the client has enough directory + information: network-statuses (or failed attempts to download + network-statuses) for all authorities, network-statuses for at more than + half of the authorities, and descriptors for at least 1/4 of the servers + believed to be running. + + A server is "listed" if it is included by more than half of the live + network status documents. Clients SHOULD NOT use unlisted servers. + + Clients believe the flags "Valid", "Exit", "Fast", "Guard", "Stable", and + "V2Dir" about a given router when they are asserted by more than half of + the live network-status documents. Clients believe the flag "Running" if + it is listed by more than half of the recent network-status documents. + + These flags are used as follows: + + - Clients SHOULD NOT use non-'Valid' or non-'Running' routers unless + requested to do so. + + - Clients SHOULD NOT use non-'Fast' routers for any purpose other than + very-low-bandwidth circuits (such as introduction circuits). + + - Clients SHOULD NOT use non-'Stable' routers for circuits that are + likely to need to be open for a very long time (such as those used for + IRC or SSH connections). + + - Clients SHOULD NOT choose non-'Guard' nodes when picking entry guard + nodes. + + - Clients SHOULD NOT download directory information from non-'V2Dir' + caches. + +6.2. Managing naming + + In order to provide human-memorable names for individual server + identities, some directory servers bind names to IDs. Clients handle + names in two ways: + + When a client encounters a name it has not mapped before: + + If all the live "Naming" network-status documents the client has + claim that the name binds to some identity ID, and the client has at + least three live network-status documents, the client maps the name to + ID. + + When a user tries to refer to a router with a name that does not have a + mapping under the above rules, the implementation SHOULD warn the user. + After giving the warning, the implementation MAY use a router that at + least one Naming authority maps the name to, so long as no other naming + authority maps that name to a different router. If no Naming authority + maps the name to a router, the implementation MAY use any router that + advertises the name. + + Not every router needs a nickname. When a router doesn't configure a + nickname, it publishes with the default nickname "Unnamed". Authorities + SHOULD NOT ever mark a router with this nickname as Named; client software + SHOULD NOT ever use a router in response to a user request for a router + called "Unnamed". + +6.3. Software versions + + An implementation of Tor SHOULD warn when it has fetched (or has + attempted to fetch and failed four consecutive times) a network-status + for each authority, and it is running a software version + not listed on more than half of the live "Versioning" network-status + documents. + +6.4. Warning about a router's status. + + If a router tries to publish its descriptor to a Naming authority + that has its nickname mapped to another key, the router SHOULD + warn the operator that it is either using the wrong key or is using + an already claimed nickname. + + If a router has fetched (or attempted to fetch and failed four + consecutive times) a network-status for every authority, and at + least one of the authorities is "Naming", and no live "Naming" + authorities publish a binding for the router's nickname, the + router MAY remind the operator that the chosen nickname is not + bound to this key at the authorities, and suggest contacting the + authority operators. + + ... + +6.5. Router protocol versions + + A client should believe that a router supports a given feature if that + feature is supported by the router or protocol versions in more than half + of the live networkstatus's "v" entries for that router. In other words, + if the "v" entries for some router are: + v Tor 0.0.8pre1 (from authority 1) + v Tor 0.1.2.11 (from authority 2) + v FutureProtocolDescription 99 (from authority 3) + then the client should believe that the router supports any feature + supported by 0.1.2.11. + + This is currently equivalent to believing the median declared version for + a router in all live networkstatuses. + +7. Standards compliance + + All clients and servers MUST support HTTP 1.0. + +7.1. HTTP headers + + Servers MAY set the Content-Length: header. Servers SHOULD set + Content-Encoding to "deflate" or "identity". + + Servers MAY include an X-Your-Address-Is: header, whose value is the + apparent IP address of the client connecting to them (as a dotted quad). + For directory connections tunneled over a BEGIN_DIR stream, servers SHOULD + report the IP from which the circuit carrying the BEGIN_DIR stream reached + them. + + Servers SHOULD disable caching of multiple network statuses or multiple + router descriptors. Servers MAY enable caching of single descriptors, + single network statuses, the list of all router descriptors, a v1 + directory, or a v1 running routers document. XXX mention times. + +7.2. HTTP status codes + + XXX We should write down what return codes dirservers send in what + situations. + diff --git a/dir-spec-v2.txt b/dir-spec-v2.txt deleted file mode 100644 index 021532e..0000000 --- a/dir-spec-v2.txt +++ /dev/null @@ -1,877 +0,0 @@ - - Tor directory protocol, version 2 - -0. Scope and preliminaries - - This directory protocol is used by Tor version 0.1.1.x and 0.1.2.x. See - dir-spec-v1.txt for information on earlier versions, and dir-spec.txt - for information on later versions. - -0.1. Goals and motivation - - There were several problems with the way Tor handles directory information - in version 0.1.0.x and earlier. Here are the problems we try to fix with - this new design, already implemented in 0.1.1.x: - 1. Directories were very large and use up a lot of bandwidth: clients - downloaded descriptors for all router several times an hour. - 2. Every directory authority was a trust bottleneck: if a single - directory authority lied, it could make clients believe for a time an - arbitrarily distorted view of the Tor network. - 3. Our current "verified server" system is kind of nonsensical. - - 4. Getting more directory authorities would add more points of failure - and worsen possible partitioning attacks. - - There are two problems that remain unaddressed by this design. - 5. Requiring every client to know about every router won't scale. - 6. Requiring every directory cache to know every router won't scale. - - We attempt to fix 1-4 here, and to build a solution that will work when we - figure out an answer for 5. We haven't thought at all about what to do - about 6. - -1. Outline - - There is a small set (say, around 10) of semi-trusted directory - authorities. A default list of authorities is shipped with the Tor - software. Users can change this list, but are encouraged not to do so, in - order to avoid partitioning attacks. - - Routers periodically upload signed "descriptors" to the directory - authorities describing their keys, capabilities, and other information. - Routers may act as directory mirrors (also called "caches"), to reduce - load on the directory authorities. They announce this in their - descriptors. - - Each directory authority periodically generates and signs a compact - "network status" document that lists that authority's view of the current - descriptors and status for known routers, but which does not include the - descriptors themselves. - - Directory mirrors download, cache, and re-serve network-status documents - to clients. - - Clients, directory mirrors, and directory authorities all use - network-status documents to find out when their list of routers is - out-of-date. If it is, they download any missing router descriptors. - Clients download missing descriptors from mirrors; mirrors and authorities - download from authorities. Descriptors are downloaded by the hash of the - descriptor, not by the server's identity key: this prevents servers from - attacking clients by giving them descriptors nobody else uses. - - All directory information is uploaded and downloaded with HTTP. - - Coordination among directory authorities is done client-side: clients - compute a vote-like algorithm among the network-status documents they - have, and base their decisions on the result. - -1.1. What's different from 0.1.0.x? - - Clients used to download a signed concatenated set of router descriptors - (called a "directory") from directory mirrors, regardless of which - descriptors had changed. - - Between downloading directories, clients would download "network-status" - documents that would list which servers were supposed to running. - - Clients would always believe the most recently published network-status - document they were served. - - Routers used to upload fresh descriptors all the time, whether their keys - and other information had changed or not. - -1.2. Document meta-format - - Router descriptors, directories, and running-routers documents all obey the - following lightweight extensible information format. - - The highest level object is a Document, which consists of one or more - Items. Every Item begins with a KeywordLine, followed by one or more - Objects. A KeywordLine begins with a Keyword, optionally followed by - whitespace and more non-newline characters, and ends with a newline. A - Keyword is a sequence of one or more characters in the set [A-Za-z0-9-]. - An Object is a block of encoded data in pseudo-Open-PGP-style - armor. (cf. RFC 2440) - - More formally: - - Document ::= (Item | NL)+ - Item ::= KeywordLine Object* - KeywordLine ::= Keyword NL | Keyword WS ArgumentsChar+ NL - Keyword = KeywordChar+ - KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-' - ArgumentChar ::= any printing ASCII character except NL. - WS = (SP | TAB)+ - Object ::= BeginLine Base64-encoded-data EndLine - BeginLine ::= "-----BEGIN " Keyword "-----" NL - EndLine ::= "-----END " Keyword "-----" NL - - The BeginLine and EndLine of an Object must use the same keyword. - - When interpreting a Document, software MUST ignore any KeywordLine that - starts with a keyword it doesn't recognize; future implementations MUST NOT - require current clients to understand any KeywordLine not currently - described. - - Other implementations that want to extend Tor's directory format MAY - introduce their own items. The keywords for extension items SHOULD start - with the characters "x-" or "X-", to guarantee that they will not conflict - with keywords used by future versions of Tor. - -2. Router operation - - ORs SHOULD generate a new router descriptor whenever any of the - following events have occurred: - - - A period of time (18 hrs by default) has passed since the last - time a descriptor was generated. - - - A descriptor field other than bandwidth or uptime has changed. - - - Bandwidth has changed by at least a factor of 2 from the last time a - descriptor was generated, and at least a given interval of time - (20 mins by default) has passed since then. - - - Its uptime has been reset (by restarting). - - After generating a descriptor, ORs upload it to every directory - authority they know, by posting it to the URL - - http://hostname:port/tor/ - -2.1. Router descriptor format - - Every router descriptor MUST start with a "router" Item; MUST end with a - "router-signature" Item and an extra NL; and MUST contain exactly one - instance of each of the following Items: "published" "onion-key" - "signing-key" "bandwidth". - - A router descriptor MAY have zero or one of each of the following Items, - but MUST NOT have more than one: "contact", "uptime", "fingerprint", - "hibernating", "read-history", "write-history", "eventdns", "platform", - "family". - - For historical reasons some options are prefixed with "opt ", for instance - "opt fingerprint". This "opt " prefix should be ignored with the second word - used as the keyword instead. - - Additionally, a router descriptor MAY contain any number of "accept", - "reject", and "opt" Items. Other than "router" and "router-signature", - the items may appear in any order. - - The items' formats are as follows: - "router" nickname address ORPort SocksPort DirPort - - Indicates the beginning of a router descriptor. "address" must be an - IPv4 address in dotted-quad format. The last three numbers indicate - the TCP ports at which this OR exposes functionality. ORPort is a port - at which this OR accepts TLS connections for the main OR protocol; - SocksPort is deprecated and should always be 0; and DirPort is the - port at which this OR accepts directory-related HTTP connections. If - any port is not supported, the value 0 is given instead of a port - number. - - "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed - - Estimated bandwidth for this router, in bytes per second. The - "average" bandwidth is the volume per second that the OR is willing to - sustain over long periods; the "burst" bandwidth is the volume that - the OR is willing to sustain in very short intervals. The "observed" - value is an estimate of the capacity this server can handle. The - server remembers the max bandwidth sustained output over any ten - second period in the past day, and another sustained input. The - "observed" value is the lesser of these two numbers. - - "platform" string - - A human-readable string describing the system on which this OR is - running. This MAY include the operating system, and SHOULD include - the name and version of the software implementing the Tor protocol. - - "published" YYYY-MM-DD HH:MM:SS - - The time, in GMT, when this descriptor was generated. - - "fingerprint" - - A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded in - hex, with a single space after every 4 characters) for this router's - identity key. A descriptor is considered invalid (and MUST be - rejected) if the fingerprint line does not match the public key. - - "hibernating" 0|1 - - If the value is 1, then the Tor server was hibernating when the - descriptor was published, and shouldn't be used to build circuits. - - "uptime" - - The number of seconds that this OR process has been running. - - "onion-key" NL a public key in PEM format - - This key is used to encrypt EXTEND cells for this OR. The key MUST be - accepted for at least 1 week after any new key is published in a - subsequent descriptor. - - "signing-key" NL a public key in PEM format - - The OR's long-term identity key. - - "accept" exitpattern - "reject" exitpattern - - These lines describe the rules that an OR follows when - deciding whether to allow a new stream to a given address. The - 'exitpattern' syntax is described below. The rules are considered in - order; if no rule matches, the address will be accepted. For clarity, - the last such entry SHOULD be accept *:* or reject *:*. - - "router-signature" NL Signature NL - - The "SIGNATURE" object contains a signature of the PKCS1-padded - hash of the entire router descriptor, taken from the beginning of the - "router" line, through the newline after the "router-signature" line. - The router descriptor is invalid unless the signature is performed - with the router's identity key. - - "contact" info NL - - Describes a way to contact the server's administrator, preferably - including an email address and a PGP key fingerprint. - - "family" names NL - - 'Names' is a space-separated list of server nicknames or - hexdigests. If two ORs list one another in their "family" entries, - then OPs should treat them as a single OR for the purpose of path - selection. - - For example, if node A's descriptor contains "family B", and node B's - descriptor contains "family A", then node A and node B should never - be used on the same circuit. - - "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL - - Declare how much bandwidth the OR has used recently. Usage is divided - into intervals of NSEC seconds. The YYYY-MM-DD HH:MM:SS field - defines the end of the most recent interval. The numbers are the - number of bytes used in the most recent intervals, ordered from - oldest to newest. - - "eventdns" bool NL - - Declare whether this version of Tor is using the newer enhanced - dns logic. Versions of Tor without eventdns SHOULD NOT be used for - reverse hostname lookups. - - [All versions of Tor before 0.1.2.2-alpha should be assumed to have - this option set to 0 if it is not present. All Tor versions at - 0.1.2.2-alpha or later should be assumed to have this option set to - 1 if it is not present. Until 0.1.2.1-alpha-dev, this option was - not generated, even when eventdns was in use. With 0.2.0.1-alpha, the - old 'dnsworker' logic has been removed, rendering this option of - historical interest only.] - -2.2. Nonterminals in router descriptors - - nickname ::= between 1 and 19 alphanumeric characters, case-insensitive. - hexdigest ::= a '$', followed by 20 hexadecimal characters. - [Represents a server by the digest of its identity key.] - - exitpattern ::= addrspec ":" portspec - portspec ::= "*" | port | port "-" port - port ::= an integer between 1 and 65535, inclusive. - [Some implementations incorrectly generate ports with value 0. - Implementations SHOULD accept this, and SHOULD NOT generate it.] - - addrspec ::= "*" | ip4spec | ip6spec - ipv4spec ::= ip4 | ip4 "/" num_ip4_bits | ip4 "/" ip4mask - ip4 ::= an IPv4 address in dotted-quad format - ip4mask ::= an IPv4 mask in dotted-quad format - num_ip4_bits ::= an integer between 0 and 32 - ip6spec ::= ip6 | ip6 "/" num_ip6_bits - ip6 ::= an IPv6 address, surrounded by square brackets. - num_ip6_bits ::= an integer between 0 and 128 - - bool ::= "0" | "1" - - Ports are required; if they are not included in the router - line, they must appear in the "ports" lines. - -3. Network status format - - Directory authorities generate, sign, and compress network-status - documents. Directory servers SHOULD generate a fresh network-status - document when the contents of such a document would be different from the - last one generated, and some time (at least one second, possibly longer) - has passed since the last one was generated. - - The network status document contains a preamble, a set of router status - entries, and a signature, in that order. - - We use the same meta-format as used for directories and router descriptors - in "tor-spec.txt". Implementations MAY insert blank lines - for clarity between sections; these blank lines are ignored. - Implementations MUST NOT depend on blank lines in any particular location. - - As used here, "whitespace" is a sequence of 1 or more tab or space - characters. - - The preamble contains: - - "network-status-version" -- A document format version. For this - specification, the version is "2". - "dir-source" -- The authority's hostname, current IP address, and - directory port, all separated by whitespace. - "fingerprint" -- A base16-encoded hash of the signing key's - fingerprint, with no additional spaces added. - "contact" -- An arbitrary string describing how to contact the - directory server's administrator. Administrators should include at - least an email address and a PGP fingerprint. - "dir-signing-key" -- The directory server's public signing key. - "client-versions" -- A comma-separated list of recommended client - versions. - "server-versions" -- A comma-separated list of recommended server - versions. - "published" -- The publication time for this network-status object. - "dir-options" -- A set of flags, in any order, separated by whitespace: - "Names" if this directory authority performs name bindings. - "Versions" if this directory authority recommends software versions. - "BadExits" if the directory authority flags nodes that it believes - are performing incorrectly as exit nodes. - "BadDirectories" if the directory authority flags nodes that it - believes are performing incorrectly as directory caches. - - The dir-options entry is optional. The "-versions" entries are required if - the "Versions" flag is present. The other entries are required and must - appear exactly once. The "network-status-version" entry must appear first; - the others may appear in any order. Implementations MUST ignore - additional arguments to the items above, and MUST ignore unrecognized - flags. - - For each router, the router entry contains: (This format is designed for - conciseness.) - - "r" -- followed by the following elements, in order, separated by - whitespace: - - The OR's nickname, - - A hash of its identity key, encoded in base64, with trailing = - signs removed. - - A hash of its most recent descriptor, encoded in base64, with - trailing = signs removed. (The hash is calculated as for - computing the signature of a descriptor.) - - The publication time of its most recent descriptor, in the form - YYYY-MM-DD HH:MM:SS, in GMT. - - An IP address - - An OR port - - A directory port (or "0" for none") - "s" -- A series of whitespace-separated status flags, in any order: - "Authority" if the router is a directory authority. - "BadExit" if the router is believed to be useless as an exit node - (because its ISP censors it, because it is behind a restrictive - proxy, or for some similar reason). - "BadDirectory" if the router is believed to be useless as a - directory cache (because its directory port isn't working, - its bandwidth is always throttled, or for some similar - reason). - "Exit" if the router is useful for building general-purpose exit - circuits. - "Fast" if the router is suitable for high-bandwidth circuits. - "Guard" if the router is suitable for use as an entry guard. - "Named" if the router's identity-nickname mapping is canonical, - and this authority binds names. - "Stable" if the router is suitable for long-lived circuits. - "Running" if the router is currently usable. - "Valid" if the router has been 'validated'. - "V2Dir" if the router implements this protocol. - "v" -- The version of the Tor protocol that this server is running. If - the value begins with "Tor" SP, the rest of the string is a Tor - version number, and the protocol is "The Tor protocol as supported - by the given version of Tor." Otherwise, if the value begins with - some other string, Tor has upgraded to a more sophisticated - protocol versioning system, and the protocol is "a version of the - Tor protocol more recent than any we recognize." - - The "r" entry for each router must appear first and is required. The - "s" entry is optional (see Section 3.1 below for how the flags are - decided). Unrecognized flags on the "s" line and extra elements - on the "r" line must be ignored. The "v" line is optional. - - The signature section contains: - - "directory-signature" nickname-of-dirserver NL Signature - - Signature is a signature of this network-status document - (the document up until the signature, including the line - "directory-signature <nick>\n"), using the directory authority's - signing key. - - We compress the network status list with zlib before transmitting it. - -3.1. Establishing server status - - (This section describes how directory authorities choose which status - flags to apply to routers, as of Tor 0.1.1.18-rc. Later directory - authorities MAY do things differently, so long as clients keep working - well. Clients MUST NOT depend on the exact behaviors in this section.) - - In the below definitions, a router is considered "active" if it is - running, valid, and not hibernating. - - "Valid" -- a router is 'Valid' if it is running a version of Tor not - known to be broken, and the directory authority has not blacklisted - it as suspicious. - - "Named" -- Directory authority administrators may decide to support name - binding. If they do, then they must maintain a file of - nickname-to-identity-key mappings, and try to keep this file consistent - with other directory authorities. If they don't, they act as clients, and - report bindings made by other directory authorities (name X is bound to - identity Y if at least one binding directory lists it, and no directory - binds X to some other Y'.) A router is called 'Named' if the router - believes the given name should be bound to the given key. - - "Running" -- A router is 'Running' if the authority managed to connect to - it successfully within the last 30 minutes. - - "Stable" -- A router is 'Stable' if it is active, and either its - uptime is at least the median uptime for known active routers, or - its uptime is at least 30 days. Routers are never called stable if - they are running a version of Tor known to drop circuits stupidly. - (0.1.1.10-alpha through 0.1.1.16-rc are stupid this way.) - - "Fast" -- A router is 'Fast' if it is active, and its bandwidth is - in the top 7/8ths for known active routers. - - "Guard" -- A router is a possible 'Guard' if it is 'Stable' and its - bandwidth is above median for known active routers. If the total - bandwidth of active non-BadExit Exit servers is less than one third - of the total bandwidth of all active servers, no Exit is listed as - a Guard. - - "Authority" -- A router is called an 'Authority' if the authority - generating the network-status document believes it is an authority. - - "V2Dir" -- A router supports the v2 directory protocol if it has an open - directory port, and it is running a version of the directory protocol that - supports the functionality clients need. (Currently, this is - 0.1.1.9-alpha or later.) - - Directory server administrators may label some servers or IPs as - blacklisted, and elect not to include them in their network-status lists. - - Authorities SHOULD 'disable' any servers in excess of 3 on any single IP. - When there are more than 3 to choose from, authorities should first prefer - authorities to non-authorities, then prefer Running to non-Running, and - then prefer high-bandwidth to low-bandwidth. To 'disable' a server, the - authority *should* advertise it without the Running or Valid flag. - - Thus, the network-status list includes all non-blacklisted, - non-expired, non-superseded descriptors. - -4. Directory server operation - - All directory authorities and directory mirrors ("directory servers") - implement this section, except as noted. - -4.1. Accepting uploads (authorities only) - - When a router posts a signed descriptor to a directory authority, the - authority first checks whether it is well-formed and correctly - self-signed. If it is, the authority next verifies that the nickname - in question is not already assigned to a router with a different - public key. - Finally, the authority MAY check that the router is not blacklisted - because of its key, IP, or another reason. - - If the descriptor passes these tests, and the authority does not already - have a descriptor for a router with this public key, it accepts the - descriptor and remembers it. - - If the authority _does_ have a descriptor with the same public key, the - newly uploaded descriptor is remembered if its publication time is more - recent than the most recent old descriptor for that router, and either: - - There are non-cosmetic differences between the old descriptor and the - new one. - - Enough time has passed between the descriptors' publication times. - (Currently, 12 hours.) - - Differences between router descriptors are "non-cosmetic" if they would be - sufficient to force an upload as described in section 2 above. - - Note that the "cosmetic difference" test only applies to uploaded - descriptors, not to descriptors that the authority downloads from other - authorities. - -4.2. Downloading network-status documents (authorities and caches) - - All directory servers (authorities and mirrors) try to keep a fresh - set of network-status documents from every authority. To do so, - every 5 minutes, each authority asks every other authority for its - most recent network-status document. Every 15 minutes, each mirror - picks a random authority and asks it for the most recent network-status - documents for all the authorities the authority knows about (including - the chosen authority itself). - - Directory servers and mirrors remember and serve the most recent - network-status document they have from each authority. Other - network-status documents don't need to be stored. If the most recent - network-status document is over 10 days old, it is discarded anyway. - Mirrors SHOULD store and serve network-status documents from authorities - they don't recognize, but SHOULD NOT use such documents for any other - purpose. Mirrors SHOULD discard network-status documents older than 48 - hours. - -4.3. Downloading and storing router descriptors (authorities and caches) - - Periodically (currently, every 10 seconds), directory servers check - whether there are any specific descriptors (as identified by descriptor - hash in a network-status document) that they do not have and that they - are not currently trying to download. - - If so, the directory server launches requests to the authorities for these - descriptors, such that each authority is only asked for descriptors listed - in its most recent network-status. When more than one authority lists the - descriptor, we choose which to ask at random. - - If one of these downloads fails, we do not try to download that descriptor - from the authority that failed to serve it again unless we receive a newer - network-status from that authority that lists the same descriptor. - - Directory servers must potentially cache multiple descriptors for each - router. Servers must not discard any descriptor listed by any current - network-status document from any authority. If there is enough space to - store additional descriptors, servers SHOULD try to hold those which - clients are likely to download the most. (Currently, this is judged - based on the interval for which each descriptor seemed newest.) - - Authorities SHOULD NOT download descriptors for routers that they would - immediately reject for reasons listed in 3.1. - -4.4. HTTP URLs - - "Fingerprints" in these URLs are base16-encoded SHA1 hashes. - - The authoritative network-status published by a host should be available at: - http://<hostname>/tor/status/authority.z - - The network-status published by a host with fingerprint - <F> should be available at: - http://<hostname>/tor/status/fp/<F>.z - - The network-status documents published by hosts with fingerprints - <F1>,<F2>,<F3> should be available at: - http://<hostname>/tor/status/fp/<F1>+<F2>+<F3>.z - - The most recent network-status documents from all known authorities, - concatenated, should be available at: - http://<hostname>/tor/status/all.z - - The most recent descriptor for a server whose identity key has a - fingerprint of <F> should be available at: - http://<hostname>/tor/server/fp/<F>.z - - The most recent descriptors for servers with identity fingerprints - <F1>,<F2>,<F3> should be available at: - http://<hostname>/tor/server/fp/<F1>+<F2>+<F3>.z - - (NOTE: Implementations SHOULD NOT download descriptors by identity key - fingerprint. This allows a corrupted server (in collusion with a cache) to - provide a unique descriptor to a client, and thereby partition that client - from the rest of the network.) - - The server descriptor with (descriptor) digest <D> (in hex) should be - available at: - http://<hostname>/tor/server/d/<D>.z - - The most recent descriptors with digests <D1>,<D2>,<D3> should be - available at: - http://<hostname>/tor/server/d/<D1>+<D2>+<D3>.z - - The most recent descriptor for this server should be at: - http://<hostname>/tor/server/authority.z - [Nothing in the Tor protocol uses this resource yet, but it is useful - for debugging purposes. Also, the official Tor implementations - (starting at 0.1.1.x) use this resource to test whether a server's - own DirPort is reachable.] - - A concatenated set of the most recent descriptors for all known servers - should be available at: - http://<hostname>/tor/server/all.z - - For debugging, directories SHOULD expose non-compressed objects at URLs like - the above, but without the final ".z". - Clients MUST handle compressed concatenated information in two forms: - - A concatenated list of zlib-compressed objects. - - A zlib-compressed concatenated list of objects. - Directory servers MAY generate either format: the former requires less - CPU, but the latter requires less bandwidth. - - Clients SHOULD use upper case letters (A-F) when base16-encoding - fingerprints. Servers MUST accept both upper and lower case fingerprints - in requests. - -5. Client operation: downloading information - - Every Tor that is not a directory server (that is, those that do - not have a DirPort set) implements this section. - -5.1. Downloading network-status documents - - Each client maintains an ordered list of directory authorities. - Insofar as possible, clients SHOULD all use the same ordered list. - - For each network-status document a client has, it keeps track of its - publication time *and* the time when the client retrieved it. Clients - consider a network-status document "live" if it was published within the - last 24 hours. - - Clients try to have a live network-status document hours from *every* - authority, and try to periodically get new network-status documents from - each authority in rotation as follows: - - If a client is missing a live network-status document for any - authority, it tries to fetch it from a directory cache. On failure, - the client waits briefly, then tries that network-status document - again from another cache. The client does not build circuits until it - has live network-status documents from more than half the authorities - it trusts, and it has descriptors for more than 1/4 of the routers - that it believes are running. - - If the most recently _retrieved_ network-status document is over 30 - minutes old, the client attempts to download a network-status document. - When choosing which documents to download, clients treat their list of - directory authorities as a circular ring, and begin with the authority - appearing immediately after the authority for their most recently - retrieved network-status document. If this attempt fails (either it - fails to download at all, or the one it gets is not as good as the - one it has), the client retries at other caches several times, before - moving on to the next network-status document in sequence. - - Clients discard all network-status documents over 24 hours old. - - If enough mirrors (currently 4) claim not to have a given network status, - we stop trying to download that authority's network-status, until we - download a new network-status that makes us believe that the authority in - question is running. Clients should wait a little longer after each - failure. - - Clients SHOULD try to batch as many network-status requests as possible - into each HTTP GET. - - (Note: clients can and should pick caches based on the network-status - information they have: once they have first fetched network-status info - from an authority, they should not need to go to the authority directly - again.) - -5.2. Downloading and storing router descriptors - - Clients try to have the best descriptor for each router. A descriptor is - "best" if: - * It is the most recently published descriptor listed for that router - by at least two network-status documents. - OR, - * No descriptor for that router is listed by two or more - network-status documents, and it is the most recently published - descriptor listed by any network-status document. - - Periodically (currently every 10 seconds) clients check whether there are - any "downloadable" descriptors. A descriptor is downloadable if: - - It is the "best" descriptor for some router. - - The descriptor was published at least 10 minutes in the past. - (This prevents clients from trying to fetch descriptors that the - mirrors have probably not yet retrieved and cached.) - - The client does not currently have it. - - The client is not currently trying to download it. - - The client would not discard it immediately upon receiving it. - - The client thinks it is running and valid (see 6.1 below). - - If at least 16 known routers have downloadable descriptors, or if - enough time (currently 10 minutes) has passed since the last time the - client tried to download descriptors, it launches requests for all - downloadable descriptors, as described in 5.3 below. - - When a descriptor download fails, the client notes it, and does not - consider the descriptor downloadable again until a certain amount of time - has passed. (Currently 0 seconds for the first failure, 60 seconds for the - second, 5 minutes for the third, 10 minutes for the fourth, and 1 day - thereafter.) Periodically (currently once an hour) clients reset the - failure count. - - No descriptors are downloaded until the client has downloaded more than - half of the network-status documents. - - Clients retain the most recent descriptor they have downloaded for each - router so long as it is not too old (currently, 48 hours), OR so long as - it is recommended by at least one networkstatus AND no "better" - descriptor has been downloaded. [Versions of Tor before 0.1.2.3-alpha - would discard descriptors simply for being published too far in the past.] - [The code seems to discard descriptors in all cases after they're 5 - days old. True? -RD] - -5.3. Managing downloads - - When a client has no live network-status documents, it downloads - network-status documents from a randomly chosen authority. In all other - cases, the client downloads from mirrors randomly chosen from among those - believed to be V2 directory servers. (This information comes from the - network-status documents; see 6 below.) - - When downloading multiple router descriptors, the client chooses multiple - mirrors so that: - - At least 3 different mirrors are used, except when this would result - in more than one request for under 4 descriptors. - - No more than 128 descriptors are requested from a single mirror. - - Otherwise, as few mirrors as possible are used. - After choosing mirrors, the client divides the descriptors among them - randomly. - - After receiving any response client MUST discard any network-status - documents and descriptors that it did not request. - -6. Using directory information - - Everyone besides directory authorities uses the approaches in this section - to decide which servers to use and what their keys are likely to be. - (Directory authorities just believe their own opinions, as in 3.1 above.) - -6.1. Choosing routers for circuits. - - Tor implementations only pay attention to "live" network-status documents. - A network status is "live" if it is the most recently downloaded network - status document for a given directory server, and the server is a - directory server trusted by the client, and the network-status document is - no more than 1 day old. - - For time-sensitive information, Tor implementations focus on "recent" - network-status documents. A network status is "recent" if it is live, and - if it was published in the last 60 minutes. If there are fewer - than 3 such documents, the most recently published 3 are "recent." If - there are fewer than 3 in all, all are "recent.") - - Circuits SHOULD NOT be built until the client has enough directory - information: network-statuses (or failed attempts to download - network-statuses) for all authorities, network-statuses for at more than - half of the authorities, and descriptors for at least 1/4 of the servers - believed to be running. - - A server is "listed" if it is included by more than half of the live - network status documents. Clients SHOULD NOT use unlisted servers. - - Clients believe the flags "Valid", "Exit", "Fast", "Guard", "Stable", and - "V2Dir" about a given router when they are asserted by more than half of - the live network-status documents. Clients believe the flag "Running" if - it is listed by more than half of the recent network-status documents. - - These flags are used as follows: - - - Clients SHOULD NOT use non-'Valid' or non-'Running' routers unless - requested to do so. - - - Clients SHOULD NOT use non-'Fast' routers for any purpose other than - very-low-bandwidth circuits (such as introduction circuits). - - - Clients SHOULD NOT use non-'Stable' routers for circuits that are - likely to need to be open for a very long time (such as those used for - IRC or SSH connections). - - - Clients SHOULD NOT choose non-'Guard' nodes when picking entry guard - nodes. - - - Clients SHOULD NOT download directory information from non-'V2Dir' - caches. - -6.2. Managing naming - - In order to provide human-memorable names for individual server - identities, some directory servers bind names to IDs. Clients handle - names in two ways: - - When a client encounters a name it has not mapped before: - - If all the live "Naming" network-status documents the client has - claim that the name binds to some identity ID, and the client has at - least three live network-status documents, the client maps the name to - ID. - - When a user tries to refer to a router with a name that does not have a - mapping under the above rules, the implementation SHOULD warn the user. - After giving the warning, the implementation MAY use a router that at - least one Naming authority maps the name to, so long as no other naming - authority maps that name to a different router. If no Naming authority - maps the name to a router, the implementation MAY use any router that - advertises the name. - - Not every router needs a nickname. When a router doesn't configure a - nickname, it publishes with the default nickname "Unnamed". Authorities - SHOULD NOT ever mark a router with this nickname as Named; client software - SHOULD NOT ever use a router in response to a user request for a router - called "Unnamed". - -6.3. Software versions - - An implementation of Tor SHOULD warn when it has fetched (or has - attempted to fetch and failed four consecutive times) a network-status - for each authority, and it is running a software version - not listed on more than half of the live "Versioning" network-status - documents. - -6.4. Warning about a router's status. - - If a router tries to publish its descriptor to a Naming authority - that has its nickname mapped to another key, the router SHOULD - warn the operator that it is either using the wrong key or is using - an already claimed nickname. - - If a router has fetched (or attempted to fetch and failed four - consecutive times) a network-status for every authority, and at - least one of the authorities is "Naming", and no live "Naming" - authorities publish a binding for the router's nickname, the - router MAY remind the operator that the chosen nickname is not - bound to this key at the authorities, and suggest contacting the - authority operators. - - ... - -6.5. Router protocol versions - - A client should believe that a router supports a given feature if that - feature is supported by the router or protocol versions in more than half - of the live networkstatus's "v" entries for that router. In other words, - if the "v" entries for some router are: - v Tor 0.0.8pre1 (from authority 1) - v Tor 0.1.2.11 (from authority 2) - v FutureProtocolDescription 99 (from authority 3) - then the client should believe that the router supports any feature - supported by 0.1.2.11. - - This is currently equivalent to believing the median declared version for - a router in all live networkstatuses. - -7. Standards compliance - - All clients and servers MUST support HTTP 1.0. - -7.1. HTTP headers - - Servers MAY set the Content-Length: header. Servers SHOULD set - Content-Encoding to "deflate" or "identity". - - Servers MAY include an X-Your-Address-Is: header, whose value is the - apparent IP address of the client connecting to them (as a dotted quad). - For directory connections tunneled over a BEGIN_DIR stream, servers SHOULD - report the IP from which the circuit carrying the BEGIN_DIR stream reached - them. - - Servers SHOULD disable caching of multiple network statuses or multiple - router descriptors. Servers MAY enable caching of single descriptors, - single network statuses, the list of all router descriptors, a v1 - directory, or a v1 running routers document. XXX mention times. - -7.2. HTTP status codes - - XXX We should write down what return codes dirservers send in what - situations. -