January 2014 - tor-commits - lists.torproject.org

[torspec/master] Remove now empty section 4.1, and renumber sections.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit 592827b25b161b388fe888013f67d138d97ad971 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 14:04:05 2014 +0100 Remove now empty section 4.1, and renumber sections. --- dir-spec.txt | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/dir-spec.txt b/dir-spec.txt index e4e2f4a..765d076 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2501,9 +2501,7 @@ All directory caches implement this section, except as noted. -4.1. Voting (authorities only) - -4.2. Downloading consensus status documents (caches only) +4.1. Downloading consensus status documents (caches only) All directory servers (authorities and caches) try to keep a recent network-status consensus document to serve to clients. A cache ALWAYS @@ -2527,7 +2525,7 @@ length. Caches serve all consensus flavors from the same locations as the directory authorities. -4.3. Downloading router descriptors +4.2. Downloading router descriptors Periodically (currently, every 10 seconds), directory caches check whether there are any specific descriptors that they do not have and that @@ -2550,7 +2548,7 @@ [XXXX define recent] -4.4. Downloading and storing microdescriptors (caches only) +4.3. Downloading and storing microdescriptors (caches only) Directory mirrors should fetch, cache, and serve each microdescriptor from the authorities. @@ -2571,7 +2569,7 @@ (NOTE: Due to squid proxy url limitations at most 92 microdescrriptor hashes can be retrieved in a single request.) -4.5. Downloading and storing extra-info documents +4.4. Downloading and storing extra-info documents Any cache that chooses to cache extra-info documents and any client that uses extra-info documents should implement this @@ -2585,9 +2583,9 @@ documents currently held. If so, it downloads whatever extra-info documents are missing. Caches download from authorities; non-caches try to download from caches. We follow the same splitting and back-off rules - as in section 4.3 (if a cache) or section 5.3 (if a client). + as in section 4.2 (if a cache) or section 5.3 (if a client). -4.6. General-use HTTP URLs +4.5. General-use HTTP URLs "Fingerprints" in these URLs are base16-encoded SHA1 hashes.

1 0

[torspec/master] Fix section number of new appendix B.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit 103b2a9cc58f75a56227b3994b252bd01cace4b4 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 19:11:27 2014 +0100 Fix section number of new appendix B. --- dir-spec.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir-spec.txt b/dir-spec.txt index 8862f43..aba072e 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2893,7 +2893,7 @@ A. Consensus-negotiation timeline. Valid-after/valid-until switchover -4.5. General-use HTTP URLs +B. General-use HTTP URLs "Fingerprints" in these URLs are base16-encoded SHA1 hashes.

1 0

[torspec/master] Rename section 4.3.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit 5de33f38b0a6e071573c02a72f0aedaae17cdeec Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 15:28:33 2014 +0100 Rename section 4.3. --- dir-spec.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir-spec.txt b/dir-spec.txt index dbf48d0..6a8c764 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2548,7 +2548,7 @@ [XXXX define recent] -4.3. Downloading and storing microdescriptors (caches only) +4.3. Downloading microdescriptors Directory mirrors should fetch, cache, and serve each microdescriptor from the authorities.

1 0

[torspec/master] Rename section 4.4.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit 1ea9eb75d632896dbe329e979584650789855826 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 15:29:29 2014 +0100 Rename section 4.4. --- dir-spec.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir-spec.txt b/dir-spec.txt index 6a8c764..3ae0162 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2569,7 +2569,7 @@ (NOTE: Due to squid proxy url limitations at most 92 microdescrriptor hashes can be retrieved in a single request.) -4.4. Downloading and storing extra-info documents +4.4. Downloading extra-info documents Any cache that chooses to cache extra-info documents and any client that uses extra-info documents should implement this

1 0

[torspec/master] Make section 6 and its subsections the new section 5.4.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit 82d9b977ac264fa7b5b1003e092c539f3ee1e0e3 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 19:42:29 2014 +0100 Make section 6 and its subsections the new section 5.4. This specification should end at the point where clients have downloaded all directory information they need. Using this information should be covered in path-spec. This results in the following changes to section numbers: * 6 -> 5.4 * 6.1 -> 5.4.1 * 6.2 -> 5.4.2 * 6.3 -> 5.4.3 * 6.4 -> 5.4.4 * 6.5 -> 5.4.5 This commit does not yet repair subsequent section numbers. --- dir-spec.txt | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/dir-spec.txt b/dir-spec.txt index 7429446..f60beea 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2648,7 +2648,7 @@ - The client does not currently have it. - The client is not currently trying to download it. - The client would not discard it immediately upon receiving it. - - The client thinks it is running and valid (see section 6.1 below). + - The client thinks it is running and valid (see section 5.4.1 below). If at least 16 known routers have downloadable descriptors, or if enough time (currently 10 minutes) has passed since the last time the @@ -2718,14 +2718,16 @@ documents are missing. Clients try to download from caches. We follow the same splitting and back-off rules as in section 5.2. -6. Using directory information +5.4. Using directory information + + [XXX This subsection really belongs in path-spec.txt, not here. -KL] Everyone besides directory authorities uses the approaches in this section to decide which relays to use and what their keys are likely to be. (Directory authorities just believe their own opinions, as in section 3.4.2 above.) -6.1. Choosing routers for circuits. +5.4.1. Choosing routers for circuits. Circuits SHOULD NOT be built until the client has enough directory information: a live consensus network status [XXXX fallback?] and @@ -2754,7 +2756,7 @@ See the "path-spec.txt" document for more details. -6.2. Managing naming +5.4.2. Managing naming In order to provide human-memorable names for individual router identities, some directory servers bind names to IDs. Clients handle @@ -2782,12 +2784,12 @@ SHOULD NOT ever use a router in response to a user request for a router called "Unnamed". -6.3. Software versions +5.4.3. Software versions An implementation of Tor SHOULD warn when it has fetched a consensus network-status, and it is running a software version not listed. -6.4. Warning about a router's status. +5.4.4. Warning about a router's status. If a router tries to publish its descriptor to a Naming authority that has its nickname mapped to another key, the router SHOULD @@ -2802,7 +2804,7 @@ ... -6.5. Router protocol versions +5.4.5. Router protocol versions A client should believe that a router supports a given feature if that feature is supported by the router or protocol versions in more than half

1 0

[torspec/master] Distribute contents of 5.3 to 5.1 and 5.2.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit b44a3b2a841fade8ca58c53814942b9b76cef87f Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 19:22:25 2014 +0100 Distribute contents of 5.3 to 5.1 and 5.2. "Managing downloads" is not an operation of its own. Distribute the consensus specific parts to 5.1 and the router descriptor specific parts to 5.2. This commit does not yet repair section numbering or references. --- dir-spec.txt | 58 +++++++++++++++++++++++++++++----------------------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/dir-spec.txt b/dir-spec.txt index 2dacd0e..8cd666a 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2595,8 +2595,15 @@ A network-status document is "live" if the time in its valid-until field has not passed. - If a client is missing a live network-status document, it tries to fetch - it from a directory cache (or from an authority if it knows no caches). + When a client has no consensus network-status document, it downloads it + from a randomly chosen authority. In all other cases, the client + downloads from caches randomly chosen from among those believed to be V2 + directory servers. (This information comes from the network-status + documents; see 6 below.) + + After receiving any response client MUST discard any network-status + documents that it did not request. + On failure, the client waits briefly, then tries that network-status document again from another cache. The client does not build circuits until it has a live network-status consensus document, and it has @@ -2646,7 +2653,19 @@ If at least 16 known routers have downloadable descriptors, or if enough time (currently 10 minutes) has passed since the last time the client tried to download descriptors, it launches requests for all - downloadable descriptors, as described in section 5.3 below. + downloadable descriptors. + + When downloading multiple router descriptors, the client chooses multiple + mirrors so that: + - At least 3 different mirrors are used, except when this would result + in more than one request for under 4 descriptors. + - No more than 128 descriptors are requested from a single mirror. + - Otherwise, as few mirrors as possible are used. + After choosing mirrors, the client divides the descriptors among them + randomly. + + After receiving any response client MUST discard any descriptors that it + did not request. When a descriptor download fails, the client notes it, and does not consider the descriptor downloadable again until a certain amount of time @@ -2670,32 +2689,6 @@ they currently fetch descriptors. After bootstrapping, clients only need to fetch the microdescriptors that have changed. - Clients maintain a cache of microdescriptors along with metadata like - when it was last referenced by a consensus, and which identity key - it corresponds to. They keep a microdescriptor until it hasn't been - mentioned in any consensus for a week. Future clients might cache them - for longer or shorter times. - -5.3. Managing downloads - - When a client has no consensus network-status document, it downloads it - from a randomly chosen authority. In all other cases, the client - downloads from caches randomly chosen from among those believed to be V2 - directory servers. (This information comes from the network-status - documents; see 6 below.) - - When downloading multiple router descriptors, the client chooses multiple - mirrors so that: - - At least 3 different mirrors are used, except when this would result - in more than one request for under 4 descriptors. - - No more than 128 descriptors are requested from a single mirror. - - Otherwise, as few mirrors as possible are used. - After choosing mirrors, the client divides the descriptors among them - randomly. - - After receiving any response client MUST discard any network-status - documents and descriptors that it did not request. - When a client gets a new microdescriptor consensus, it looks to see if there are any microdescriptors it needs to learn. If it needs to learn more than half of the microdescriptors, it requests 'all', else it @@ -2703,6 +2696,13 @@ the upload bandwidth for listing the microdescriptors they want is more or less than the download bandwidth for the microdescriptors they do not want. + [XXX The 'all' URL is not implemented yet. -KL] + + Clients maintain a cache of microdescriptors along with metadata like + when it was last referenced by a consensus, and which identity key + it corresponds to. They keep a microdescriptor until it hasn't been + mentioned in any consensus for a week. Future clients might cache them + for longer or shorter times. 5.4. Downloading extra-info documents

1 0

[torspec/master] Rename section 5.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit a2a1fc66ba352794c64b1237d34d69e936e10444 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 19:12:14 2014 +0100 Rename section 5. --- dir-spec.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir-spec.txt b/dir-spec.txt index aba072e..b29598d 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2581,7 +2581,7 @@ documents are missing. Caches download from authorities. We follow the same splitting and back-off rules as in section 4.2. -5. Client operation: downloading information +5. Client operation Every Tor that is not a directory server (that is, those that do not have a DirPort set) implements this section.

1 0

[torspec/master] Fix section references from removing section 5.3.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit 4a9224500dfaa06232cac4be3df798e3fd548b13 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 19:24:15 2014 +0100 Fix section references from removing section 5.3. --- dir-spec.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/dir-spec.txt b/dir-spec.txt index 8cd666a..7429446 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2704,7 +2704,7 @@ mentioned in any consensus for a week. Future clients might cache them for longer or shorter times. -5.4. Downloading extra-info documents +5.3. Downloading extra-info documents Any client that uses extra-info documents should implement this section. @@ -2716,7 +2716,7 @@ extra-info-digest field that does not match any of the extra-info documents currently held. If so, it downloads whatever extra-info documents are missing. Clients try to download from caches. - We follow the same splitting and back-off rules as in section 5.3. + We follow the same splitting and back-off rules as in section 5.2. 6. Using directory information

1 0

[torspec/master] Move section 4.5 to the appendix.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit a8f52765f394a6e23866ba28b4781fa530f03114 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 19:09:56 2014 +0100 Move section 4.5 to the appendix. "General-use HTTP URLs" is not an operation but a reference, and references belong in the appendix. This commit does not yet repair section numbering or references. --- dir-spec.txt | 232 +++++++++++++++++++++++++++++----------------------------- 1 file changed, 116 insertions(+), 116 deletions(-) diff --git a/dir-spec.txt b/dir-spec.txt index 17b694e..8862f43 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2581,122 +2581,6 @@ documents are missing. Caches download from authorities. We follow the same splitting and back-off rules as in section 4.2. -4.5. General-use HTTP URLs - - "Fingerprints" in these URLs are base16-encoded SHA1 hashes. - - The most recent v3 consensus should be available at: - http://<hostname>/tor/status-vote/current/consensus.z - - Starting with Tor version 0.2.1.1-alpha is also available at: - http://<hostname>/tor/status-vote/current/consensus/<F1>+<F2>+<F3>.z - - (NOTE: Due to squid proxy url limitations at most 96 fingerprints can be - retrieved in a single request.) - - Where F1, F2, etc. are authority identity fingerprints the client trusts. - Servers will only return a consensus if more than half of the requested - authorities have signed the document, otherwise a 404 error will be sent - back. The fingerprints can be shortened to a length of any multiple of - two, using only the leftmost part of the encoded fingerprint. Tor uses - 3 bytes (6 hex characters) of the fingerprint. - - Clients SHOULD sort the fingerprints in ascending order. Server MUST - accept any order. - - Clients SHOULD use this format when requesting consensus documents from - directory authority servers and from caches running a version of Tor - that is known to support this URL format. - - A concatenated set of all the current key certificates should be available - at: - http://<hostname>/tor/keys/all.z - - The key certificate for this server (if it is an authority) should be - available at: - http://<hostname>/tor/keys/authority.z - - The key certificate for an authority whose authority identity fingerprint - is <F> should be available at: - http://<hostname>/tor/keys/fp/<F>.z - - The key certificate whose signing key fingerprint is <F> should be - available at: - http://<hostname>/tor/keys/sk/<F>.z - - The key certificate whose identity key fingerprint is <F> and whose signing - key fingerprint is <S> should be available at: - - http://<hostname>/tor/keys/fp-sk/<F>-<S>.z - - (As usual, clients may request multiple certificates using: - http://<hostname>/tor/keys/fp-sk/<F1>-<S1>+<F2>-<S2>.z ) - [The above fp-sk format was not supported before Tor 0.2.1.9-alpha.] - - The most recent descriptor for a server whose identity key has a - fingerprint of <F> should be available at: - http://<hostname>/tor/server/fp/<F>.z - - The most recent descriptors for servers with identity fingerprints - <F1>,<F2>,<F3> should be available at: - http://<hostname>/tor/server/fp/<F1>+<F2>+<F3>.z - - (NOTE: Due to squid proxy url limitations at most 96 fingerprints can be - retrieved in a single request. - - Implementations SHOULD NOT download descriptors by identity key - fingerprint. This allows a corrupted server (in collusion with a cache) to - provide a unique descriptor to a client, and thereby partition that client - from the rest of the network.) - - The server descriptor with (descriptor) digest <D> (in hex) should be - available at: - http://<hostname>/tor/server/d/<D>.z - - The most recent descriptors with digests <D1>,<D2>,<D3> should be - available at: - http://<hostname>/tor/server/d/<D1>+<D2>+<D3>.z - - The most recent descriptor for this server should be at: - http://<hostname>/tor/server/authority.z - [Nothing in the Tor protocol uses this resource yet, but it is useful - for debugging purposes. Also, the official Tor implementations - (starting at 0.1.1.x) use this resource to test whether a server's - own DirPort is reachable.] - - A concatenated set of the most recent descriptors for all known servers - should be available at: - http://<hostname>/tor/server/all.z - - Extra-info documents are available at the URLS - http://<hostname>/tor/extra/d/... - http://<hostname>/tor/extra/fp/... - http://<hostname>/tor/extra/all[.z] - http://<hostname>/tor/extra/authority[.z] - (As for /tor/server/ URLs: supports fetching extra-info - documents by their digest, by the fingerprint of their servers, - or all at once. When serving by fingerprint, we serve the - extra-info that corresponds to the descriptor we would serve by - that fingerprint. Only directory authorities of version - 0.2.0.1-alpha or later are guaranteed to support the first - three classes of URLs. Caches may support them, and MUST - support them if they have advertised "caches-extra-info".) - - For debugging, directories SHOULD expose non-compressed objects at URLs like - the above, but without the final ".z". - Clients MUST handle compressed concatenated information in two forms: - - A concatenated list of zlib-compressed objects. - - A zlib-compressed concatenated list of objects. - Directory servers MAY generate either format: the former requires less - CPU, but the latter requires less bandwidth. - - Clients SHOULD use upper case letters (A-F) when base16-encoding - fingerprints. Servers MUST accept both upper and lower case fingerprints - in requests. - - [XXX Add new URLs for microdescriptors, consensus flavors, and - microdescriptor consensus. -KL] - 5. Client operation: downloading information Every Tor that is not a directory server (that is, those that do @@ -3009,3 +2893,119 @@ A. Consensus-negotiation timeline. Valid-after/valid-until switchover +4.5. General-use HTTP URLs + + "Fingerprints" in these URLs are base16-encoded SHA1 hashes. + + The most recent v3 consensus should be available at: + http://<hostname>/tor/status-vote/current/consensus.z + + Starting with Tor version 0.2.1.1-alpha is also available at: + http://<hostname>/tor/status-vote/current/consensus/<F1>+<F2>+<F3>.z + + (NOTE: Due to squid proxy url limitations at most 96 fingerprints can be + retrieved in a single request.) + + Where F1, F2, etc. are authority identity fingerprints the client trusts. + Servers will only return a consensus if more than half of the requested + authorities have signed the document, otherwise a 404 error will be sent + back. The fingerprints can be shortened to a length of any multiple of + two, using only the leftmost part of the encoded fingerprint. Tor uses + 3 bytes (6 hex characters) of the fingerprint. + + Clients SHOULD sort the fingerprints in ascending order. Server MUST + accept any order. + + Clients SHOULD use this format when requesting consensus documents from + directory authority servers and from caches running a version of Tor + that is known to support this URL format. + + A concatenated set of all the current key certificates should be available + at: + http://<hostname>/tor/keys/all.z + + The key certificate for this server (if it is an authority) should be + available at: + http://<hostname>/tor/keys/authority.z + + The key certificate for an authority whose authority identity fingerprint + is <F> should be available at: + http://<hostname>/tor/keys/fp/<F>.z + + The key certificate whose signing key fingerprint is <F> should be + available at: + http://<hostname>/tor/keys/sk/<F>.z + + The key certificate whose identity key fingerprint is <F> and whose signing + key fingerprint is <S> should be available at: + + http://<hostname>/tor/keys/fp-sk/<F>-<S>.z + + (As usual, clients may request multiple certificates using: + http://<hostname>/tor/keys/fp-sk/<F1>-<S1>+<F2>-<S2>.z ) + [The above fp-sk format was not supported before Tor 0.2.1.9-alpha.] + + The most recent descriptor for a server whose identity key has a + fingerprint of <F> should be available at: + http://<hostname>/tor/server/fp/<F>.z + + The most recent descriptors for servers with identity fingerprints + <F1>,<F2>,<F3> should be available at: + http://<hostname>/tor/server/fp/<F1>+<F2>+<F3>.z + + (NOTE: Due to squid proxy url limitations at most 96 fingerprints can be + retrieved in a single request. + + Implementations SHOULD NOT download descriptors by identity key + fingerprint. This allows a corrupted server (in collusion with a cache) to + provide a unique descriptor to a client, and thereby partition that client + from the rest of the network.) + + The server descriptor with (descriptor) digest <D> (in hex) should be + available at: + http://<hostname>/tor/server/d/<D>.z + + The most recent descriptors with digests <D1>,<D2>,<D3> should be + available at: + http://<hostname>/tor/server/d/<D1>+<D2>+<D3>.z + + The most recent descriptor for this server should be at: + http://<hostname>/tor/server/authority.z + [Nothing in the Tor protocol uses this resource yet, but it is useful + for debugging purposes. Also, the official Tor implementations + (starting at 0.1.1.x) use this resource to test whether a server's + own DirPort is reachable.] + + A concatenated set of the most recent descriptors for all known servers + should be available at: + http://<hostname>/tor/server/all.z + + Extra-info documents are available at the URLS + http://<hostname>/tor/extra/d/... + http://<hostname>/tor/extra/fp/... + http://<hostname>/tor/extra/all[.z] + http://<hostname>/tor/extra/authority[.z] + (As for /tor/server/ URLs: supports fetching extra-info + documents by their digest, by the fingerprint of their servers, + or all at once. When serving by fingerprint, we serve the + extra-info that corresponds to the descriptor we would serve by + that fingerprint. Only directory authorities of version + 0.2.0.1-alpha or later are guaranteed to support the first + three classes of URLs. Caches may support them, and MUST + support them if they have advertised "caches-extra-info".) + + For debugging, directories SHOULD expose non-compressed objects at URLs like + the above, but without the final ".z". + Clients MUST handle compressed concatenated information in two forms: + - A concatenated list of zlib-compressed objects. + - A zlib-compressed concatenated list of objects. + Directory servers MAY generate either format: the former requires less + CPU, but the latter requires less bandwidth. + + Clients SHOULD use upper case letters (A-F) when base16-encoding + fingerprints. Servers MUST accept both upper and lower case fingerprints + in requests. + + [XXX Add new URLs for microdescriptors, consensus flavors, and + microdescriptor consensus. -KL] +

1 0

[torspec/master] Rename section 5.2.
by nickm＠torproject.org 17 Jan '14

17 Jan '14

commit cd0c8cf1c48ec5e58a5f1f0268c9d75f04f29d35 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Jan 14 19:14:28 2014 +0100 Rename section 5.2. It's obvious that clients are storing router descriptors they downloaded. --- dir-spec.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir-spec.txt b/dir-spec.txt index b29598d..2dacd0e 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2626,7 +2626,7 @@ the same update strategy as for the normal consensus. They should not download more than one consensus flavor. -5.2. Downloading and storing router descriptors or microdescriptors +5.2. Downloading router descriptors or microdescriptors Clients try to have the best descriptor for each router. A descriptor is "best" if:

1 0