[tor-commits] [torspec/master] Create rend-spec-v3.txt based on proposal 224

commit 471af27b55ff3894551109b45848f2ce1002441b Author: Nick Mathewson nickm@torproject.org Date: Tue Sep 19 14:33:33 2017 -0400

Create rend-spec-v3.txt based on proposal 224 --- proposals/000-index.txt | 4 +- proposals/224-rend-spec-ng.txt | 3 +- rend-spec.txt => rend-spec-v2.txt | 0 rend-spec-v3.txt | 2369 +++++++++++++++++++++++++++++++++++++ 4 files changed, 2373 insertions(+), 3 deletions(-)

diff --git a/proposals/000-index.txt b/proposals/000-index.txt index e184aab..6f6fca0 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -144,7 +144,7 @@ Proposals by number: 221 Stop using CREATE_FAST [CLOSED] 222 Stop sending client timestamps [CLOSED] 223 Ace: Improved circuit-creation key exchange [RESERVE] -224 Next-Generation Hidden Services in Tor [ACCEPTED] +224 Next-Generation Hidden Services in Tor [CLOSED] 225 Strawman proposal: commit-and-reveal shared rng [SUPERSEDED] 226 "Scalability and Stability Improvements to BridgeDB: Switching to a Distributed Database System and RDBMS" [OPEN] 227 Include package fingerprints in consensus documents [CLOSED] @@ -263,7 +263,6 @@ Proposals by status: 172 GETINFO controller option for circuit information 173 GETINFO Option Expansion 188 Bridge Guards and other anti-enumeration defenses - 224 Next-Generation Hidden Services in Tor 258 Denial-of-service resistance for directory authorities 265 Load Balancing with Overhead Parameters [for 0.2.9.x] META: @@ -345,6 +344,7 @@ Proposals by status: 218 Controller events to better understand connection/circuit usage [in 0.2.5.2-alpha] 221 Stop using CREATE_FAST [for 0.2.5.x] 222 Stop sending client timestamps [in 0.2.4.18] + 224 Next-Generation Hidden Services in Tor [in 0.3.2.1-alpha] 227 Include package fingerprints in consensus documents [in 0.2.6.3-alpha] 228 Cross-certifying identity keys with onion keys 235 Stop assigning (and eventually supporting) the Named flag [in 0.2.6, 0.2.7] diff --git a/proposals/224-rend-spec-ng.txt b/proposals/224-rend-spec-ng.txt index 73f8936..b2c0fea 100644 --- a/proposals/224-rend-spec-ng.txt +++ b/proposals/224-rend-spec-ng.txt @@ -2,7 +2,8 @@ Filename: 224-rend-spec-ng.txt Title: Next-Generation Hidden Services in Tor Author: David Goulet, George Kadianakis, Nick Mathewson Created: 2013-11-29 -Status: Accepted +Status: Closed +Implemented-In: 0.3.2.1-alpha

Table of contents:

diff --git a/rend-spec.txt b/rend-spec-v2.txt similarity index 100% rename from rend-spec.txt rename to rend-spec-v2.txt diff --git a/rend-spec-v3.txt b/rend-spec-v3.txt new file mode 100644 index 0000000..5540f2e --- /dev/null +++ b/rend-spec-v3.txt @@ -0,0 +1,2369 @@ + Tor Rendezvous Specification - Version 3 + +This document specifies how the hidden service version 3 protocol works. This +text used to be proposal 224-rend-spec-ng.txt. + + +Table of contents: + + 0. Hidden services: overview and preliminaries. + 0.1. Improvements over previous versions. + 0.2. Notation and vocabulary + 0.3. Cryptographic building blocks + 0.4. Protocol building blocks [BUILDING-BLOCKS] + 0.5. Assigned relay cell types + 0.6. Acknowledgments + 1. Protocol overview + 1.1. View from 10,000 feet + 1.2. In more detail: naming hidden services [NAMING] + 1.3. In more detail: Access control [IMD:AC] + 1.4. In more detail: Distributing hidden service descriptors. [IMD:DIST] + 1.5. In more detail: Scaling to multiple hosts + 1.6. In more detail: Backward compatibility with older hidden service + 1.7. In more detail: Keeping crypto keys offline + 1.8. In more detail: Encryption Keys And Replay Resistance + 1.9. In more detail: A menagerie of keys + 1.9.1. In even more detail: Client authorization [CLIENT-AUTH] + 2. Generating and publishing hidden service descriptors [HSDIR] + 2.1. Deriving blinded keys and subcredentials [SUBCRED] + 2.2. Locating, uploading, and downloading hidden service descriptors + 2.2.1. Dividing time into periods [TIME-PERIODS] + 2.2.2. When to publish a hidden service descriptor [WHEN-HSDESC] + 2.2.3. Where to publish a hidden service descriptor [WHERE-HSDESC] + 2.2.4. Using time periods and SRVs to fetch/upload HS descriptors + 2.2.5. Expiring hidden service descriptors [EXPIRE-DESC] + 2.2.6. URLs for anonymous uploading and downloading + 2.3. Publishing shared random values [PUB-SHAREDRANDOM] + 2.3.1. Client behavior in the absense of shared random values + 2.3.2. Hidden services and changing shared random values + 2.4. Hidden service descriptors: outer wrapper [DESC-OUTER] + 2.5. Hidden service descriptors: encryption format [HS-DESC-ENC] + 2.5.1. First layer of encryption [HS-DESC-FIRST-LAYER] + 2.5.1.1. First layer encryption logic + 2.5.1.2. First layer plaintext format + 2.5.1.3. Client behavior + 2.5.1.4. Obfuscating the number of authorized clients + 2.5.2. Second layer of encryption [HS-DESC-SECOND-LAYER] + 2.5.2.1. Second layer encryption keys + 2.5.2.2. Second layer plaintext format + 2.5.3. Deriving hidden service descriptor encryption keys [HS-DESC-ENCRYPTION-KEYS] + 3. The introduction protocol [INTRO-PROTOCOL] + 3.1. Registering an introduction point [REG_INTRO_POINT] + 3.1.1. Extensible ESTABLISH_INTRO protocol. [EST_INTRO] + 3.1.2. Registering an introduction point on a legacy Tor node [LEGACY_EST_INTRO] + 3.1.3. Acknowledging establishment of introduction point [INTRO_ESTABLISHED] + 3.2. Sending an INTRODUCE1 cell to the introduction point. [SEND_INTRO1] + 3.2.1. INTRODUCE1 cell format [FMT_INTRO1] + 3.2.2. INTRODUCE_ACK cell format. [INTRO_ACK] + 3.3. Processing an INTRODUCE2 cell at the hidden service. [PROCESS_INTRO2] + 3.3.1. Introduction handshake encryption requirements [INTRO-HANDSHAKE-REQS] + 3.3.2. Example encryption handshake: ntor with extra data [NTOR-WITH-EXTRA-DATA] + 3.4. Authentication during the introduction phase. [INTRO-AUTH] + 3.4.1. Ed25519-based authentication. + 4. The rendezvous protocol + 4.1. Establishing a rendezvous point [EST_REND_POINT] + 4.2. Joining to a rendezvous point [JOIN_REND] + 4.2.1. Key expansion + 4.3. Using legacy hosts as rendezvous points + 5. Encrypting data between client and host + 6. Encoding onion addresses [ONIONADDRESS] + 7. Open Questions: + +-1. Draft notes + + This document describes a proposed design and specification for + hidden services in Tor version 0.2.5.x or later. It's a replacement + for the current rend-spec.txt, rewritten for clarity and for improved + design. + + Look for the string "TODO" below: it describes gaps or uncertainties + in the design. + + Change history: + + 2013-11-29: Proposal first numbered. Some TODO and XXX items remain. + + 2014-01-04: Clarify some unclear sections. + + 2014-01-21: Fix a typo. + + 2014-02-20: Move more things to the revised certificate format in the + new updated proposal 220. + + 2015-05-26: Fix two typos. + + +0. Hidden services: overview and preliminaries. + + Hidden services aim to provide responder anonymity for bidirectional + stream-based communication on the Tor network. Unlike regular Tor + connections, where the connection initiator receives anonymity but + the responder does not, hidden services attempt to provide + bidirectional anonymity. + + Participants: + + Operator -- A person running a hidden service + + Host, "Server" -- The Tor software run by the operator to provide + a hidden service. + + User -- A person contacting a hidden service. + + Client -- The Tor software running on the User's computer + + Hidden Service Directory (HSDir) -- A Tor node that hosts signed + statements from hidden service hosts so that users can make + contact with them. + + Introduction Point -- A Tor node that accepts connection requests + for hidden services and anonymously relays those requests to the + hidden service. + + Rendezvous Point -- A Tor node to which clients and servers + connect and which relays traffic between them. + +0.1. Improvements over previous versions. + + Here is a list of improvements of this proposal over the legacy hidden + services: + + a) Better crypto (replaced SHA1/DH/RSA1024 with SHA3/ed25519/curve25519) + b) Improved directory protocol leaking less to directory servers. + c) Improved directory protocol with smaller surface for targeted attacks. + d) Better onion address security against impersonation. + e) More extensible introduction/rendezvous protocol. + f) Offline keys for onion services + g) Advanced client authorization + +0.2. Notation and vocabulary + + Unless specified otherwise, all multi-octet integers are big-endian. + + We write sequences of bytes in two ways: + + 1. A sequence of two-digit hexadecimal values in square brackets, + as in [AB AD 1D EA]. + + 2. A string of characters enclosed in quotes, as in "Hello". The + characters in these strings are encoded in their ascii + representations; strings are NOT nul-terminated unless + explicitly described as NUL terminated. + + We use the words "byte" and "octet" interchangeably. + + We use the vertical bar | to denote concatenation. + + We use INT_N(val) to denote the network (big-endian) encoding of the + unsigned integer "val" in N bytes. For example, INT_4(1337) is [00 00 + 05 39]. Values are truncated like so: val % (2 ^ (N * 8)). For example, + INT_4(42) is 42 % 4294967296 (32 bit). + +0.3. Cryptographic building blocks + + This specification uses the following cryptographic building blocks: + + * A pseudorandom number generator backed by a strong entropy source. + The output of the PRNG should always be hashed before being posted on + the network to avoid leaking raw PRNG bytes to the network + (see [PRNG-REFS]). + + * A stream cipher STREAM(iv, k) where iv is a nonce of length + S_IV_LEN bytes and k is a key of length S_KEY_LEN bytes. + + * A public key signature system SIGN_KEYGEN()->seckey, pubkey; + SIGN_SIGN(seckey,msg)->sig; and SIGN_CHECK(pubkey, sig, msg) -> + { "OK", "BAD" }; where secret keys are of length SIGN_SECKEY_LEN + bytes, public keys are of length SIGN_PUBKEY_LEN bytes, and + signatures are of length SIGN_SIG_LEN bytes. + + This signature system must also support key blinding operations + as discussed in appendix [KEYBLIND] and in section [SUBCRED]: + SIGN_BLIND_SECKEY(seckey, blind)->seckey2 and + SIGN_BLIND_PUBKEY(pubkey, blind)->pubkey2 . + + * A public key agreement system "PK", providing + PK_KEYGEN()->seckey, pubkey; PK_VALID(pubkey) -> {"OK", "BAD"}; + and PK_HANDSHAKE(seckey, pubkey)->output; where secret keys are + of length PK_SECKEY_LEN bytes, public keys are of length + PK_PUBKEY_LEN bytes, and the handshake produces outputs of + length PK_OUTPUT_LEN bytes. + + * A cryptographic hash function H(d), which should be preimage and + collision resistant. It produces hashes of length HASH_LEN + bytes. + + * A cryptographic message authentication code MAC(key,msg) that + produces outputs of length MAC_LEN bytes. + + * A key derivation function KDF(message, n) that outputs n bytes. + + As a first pass, I suggest: + + * Instantiate STREAM with AES256-CTR. + + * Instantiate SIGN with Ed25519 and the blinding protocol in + [KEYBLIND]. + + * Instantiate PK with Curve25519. + + * Instantiate H with SHA3-256. + + * Instantiate KDF with SHAKE-256. + + * Instantiate MAC(key=k, message=m) with H(k_len | k | m), + where k_len is htonll(len(k)). + + For legacy purposes, we specify compatibility with older versions of + the Tor introduction point and rendezvous point protocols. These used + RSA1024, DH1024, AES128, and SHA1, as discussed in + rend-spec.txt. + + As in [proposal 220], all signatures are generated not over strings + themselves, but over those strings prefixed with a distinguishing + value. + +0.4. Protocol building blocks [BUILDING-BLOCKS] + + In sections below, we need to transmit the locations and identities + of Tor nodes. We do so in the link identification format used by + EXTEND2 cells in the Tor protocol. + + NSPEC (Number of link specifiers) [1 byte] + NSPEC times: + LSTYPE (Link specifier type) [1 byte] + LSLEN (Link specifier length) [1 byte] + LSPEC (Link specifier) [LSLEN bytes] + + Link specifier types are as described in tor-spec.txt. Every set of + link specifiers MUST include at minimum specifiers of type [00] + (TLS-over-TCP, IPv4), [02] (legacy node identity) and [03] (ed25519 + identity key). + + We also incorporate Tor's circuit extension handshakes, as used in + the CREATE2 and CREATED2 cells described in tor-spec.txt. In these + handshakes, a client who knows a public key for a server sends a + message and receives a message from that server. Once the exchange is + done, the two parties have a shared set of forward-secure key + material, and the client knows that nobody else shares that key + material unless they control the secret key corresponding to the + server's public key. + +0.5. Assigned relay cell types + + These relay cell types are reserved for use in the hidden service + protocol. + + 32 -- RELAY_COMMAND_ESTABLISH_INTRO + + Sent from hidden service host to introduction point; + establishes introduction point. Discussed in + [REG_INTRO_POINT]. + + 33 -- RELAY_COMMAND_ESTABLISH_RENDEZVOUS + + Sent from client to rendezvous point; creates rendezvous + point. Discussed in [EST_REND_POINT]. + + 34 -- RELAY_COMMAND_INTRODUCE1 + + Sent from client to introduction point; requests + introduction. Discussed in [SEND_INTRO1] + + 35 -- RELAY_COMMAND_INTRODUCE2 + + Sent from introduction point to hidden service host; requests + introduction. Same format as INTRODUCE1. Discussed in + [FMT_INTRO1] and [PROCESS_INTRO2] + + 36 -- RELAY_COMMAND_RENDEZVOUS1 + + Sent from hidden service host to rendezvous point; + attempts to join host's circuit to + client's circuit. Discussed in [JOIN_REND] + + 37 -- RELAY_COMMAND_RENDEZVOUS2 + + Sent from rendezvous point to client; + reports join of host's circuit to + client's circuit. Discussed in [JOIN_REND] + + 38 -- RELAY_COMMAND_INTRO_ESTABLISHED + + Sent from introduction point to hidden service host; + reports status of attempt to establish introduction + point. Discussed in [INTRO_ESTABLISHED] + + 39 -- RELAY_COMMAND_RENDEZVOUS_ESTABLISHED + + Sent from rendezvous point to client; acknowledges + receipt of ESTABLISH_RENDEZVOUS cell. Discussed in + [EST_REND_POINT] + + 40 -- RELAY_COMMAND_INTRODUCE_ACK + + Sent from introduction point to client; acknowledges + receipt of INTRODUCE1 cell and reports success/failure. + Discussed in [INTRO_ACK] + +0.6. Acknowledgments + + This design includes ideas from many people, including + Christopher Baines, + Daniel J. Bernstein, + Matthew Finkel, + Ian Goldberg, + George Kadianakis, + Aniket Kate, + Tanja Lange, + Robert Ransom, + Roger Dingledine, + Aaron Johnson, + Tim Wilson-Brown ("teor"), + special (John Brooks), + s7r + + It's based on Tor's original hidden service design by Roger + Dingledine, Nick Mathewson, and Paul Syverson, and on improvements to + that design over the years by people including + Tobias Kamm, + Thomas Lauterbach, + Karsten Loesing, + Alessandro Preite Martinez, + Robert Ransom, + Ferdinand Rieger, + Christoph Weingarten, + Christian Wilms, + + We wouldn't be able to do any of this work without good attack + designs from researchers including + Alex Biryukov, + Lasse Øverlier, + Ivan Pustogarov, + Paul Syverson + Ralf-Philipp Weinmann, + See [ATTACK-REFS] for their papers. + + Several of these ideas have come from conversations with + Christian Grothoff, + Brian Warner, + Zooko Wilcox-O'Hearn, + + And if this document makes any sense at all, it's thanks to + editing help from + Matthew Finkel + George Kadianakis, + Peter Palfrader, + Tim Wilson-Brown ("teor"), + + + [XXX Acknowledge the huge bunch of people working on 8106.] + [XXX Acknowledge the huge bunch of people working on 8244.] + + + Please forgive me if I've missed you; please forgive me if I've + misunderstood your best ideas here too. + + +1. Protocol overview + + In this section, we outline the hidden service protocol. This section + omits some details in the name of simplicity; those are given more + fully below, when we specify the protocol in more detail. + +1.1. View from 10,000 feet + + A hidden service host prepares to offer a hidden service by choosing + several Tor nodes to serve as its introduction points. It builds + circuits to those nodes, and tells them to forward introduction + requests to it using those circuits. + + Once introduction points have been picked, the host builds a set of + documents called "hidden service descriptors" (or just "descriptors" + for short) and uploads them to a set of HSDir nodes. These documents + list the hidden service's current introduction points and describe + how to make contact with the hidden service. + + When a client wants to connect to a hidden service, it first chooses + a Tor node at random to be its "rendezvous point" and builds a + circuit to that rendezvous point. If the client does not have an + up-to-date descriptor for the service, it contacts an appropriate + HSDir and requests such a descriptor. + + The client then builds an anonymous circuit to one of the hidden + service's introduction points listed in its descriptor, and gives the + introduction point an introduction request to pass to the hidden + service. This introduction request includes the target rendezvous + point and the first part of a cryptographic handshake. + + Upon receiving the introduction request, the hidden service host + makes an anonymous circuit to the rendezvous point and completes the + cryptographic handshake. The rendezvous point connects the two + circuits, and the cryptographic handshake gives the two parties a + shared key and proves to the client that it is indeed talking to the + hidden service. + + Once the two circuits are joined, the client can send Tor RELAY cells + to the server. RELAY_BEGIN cells open streams to an external process + or processes configured by the server; RELAY_DATA cells are used to + communicate data on those streams, and so forth. + +1.2. In more detail: naming hidden services [NAMING] + + A hidden service's name is its long term master identity key. This is + encoded as a hostname by encoding the entire key in Base 32, including a + version byte and a checksum, and then appending the string ".onion" at the + end. The result is a 56-character domain name. + + (This is a change from older versions of the hidden service protocol, + where we used an 80-bit truncated SHA1 hash of a 1024 bit RSA key.) + + The names in this format are distinct from earlier names because of + their length. An older name might look like: + + unlikelynamefora.onion + yyhws9optuwiwsns.onion + + And a new name following this specification might look like: + + l5satjgud6gucryazcyvyvhuxhr74u6ygigiuyixe3a6ysis67ororad.onion + + Please see section [ONIONADDRESS] for the encoding specification. + +1.3. In more detail: Access control [IMD:AC] + + Access control for a hidden service is imposed at multiple points through + the process above. Furthermore, there is also the option to impose + additional client authorization access control using pre-shared secrets + exchanged out-of-band between the hidden service and its clients. + + The first stage of access control happens when downloading HS descriptors. + Specifically, in order to download a descriptor, clients must know which + blinded signing key was used to sign it. (See the next section for more info + on key blinding.) + + To learn the introduction points, clients must decrypt the body of the + hidden service descriptor. To do so, clients must know the _unblinded_ + public key of the service, which makes the descriptor unuseable by entities + without that knowledge (e.g. HSDirs that don't know the onion address). + + Also, if optional client authorization is enabled, hidden service + descriptors are superencrypted using each authorized user's identity x25519 + key, to further ensure that unauthorized entities cannot decrypt it. + + In order to make the introduction point send a rendezvous request to the + service, the client needs to use the per-introduction-point authentication + key found in the hidden service descriptor. + + The final level of access control happens at the server itself, which may + decide to respond or not respond to the client's request depending on the + contents of the request. The protocol is extensible at this point: at a + minimum, the server requires that the client demonstrate knowledge of the + contents of the encrypted portion of the hidden service descriptor. If + optional client authorization is enabled, the service may additionally + require the client to prove knowledge of a pre-shared private key. + +1.4. In more detail: Distributing hidden service descriptors. [IMD:DIST] + + Periodically, hidden service descriptors become stored at different + locations to prevent a single directory or small set of directories + from becoming a good DoS target for removing a hidden service. + + For each period, the Tor directory authorities agree upon a + collaboratively generated random value. (See section 2.3 for a + description of how to incorporate this value into the voting + practice; generating the value is described in other proposals, + including [SHAREDRANDOM-REFS].) That value, combined with hidden service + directories' public identity keys, determines each HSDir's position + in the hash ring for descriptors made in that period. + + Each hidden service's descriptors are placed into the ring in + positions based on the key that was used to sign them. Note that + hidden service descriptors are not signed with the services' public + keys directly. Instead, we use a key-blinding system [KEYBLIND] to + create a new key-of-the-day for each hidden service. Any client that + knows the hidden service's credential can derive these blinded + signing keys for a given period. It should be impossible to derive + the blinded signing key lacking that credential. + + The body of each descriptor is also encrypted with a key derived from + the credential. + + To avoid a "thundering herd" problem where every service generates + and uploads a new descriptor at the start of each period, each + descriptor comes online at a time during the period that depends on + its blinded signing key. The keys for the last period remain valid + until the new keys come online. + +1.5. In more detail: Scaling to multiple hosts + + This design is compatible with our current approaches for scaling hidden + services. Specifically, hidden service operators can use onionbalance to + achieve high availability between multiple nodes on the HSDir + layer. Furthermore, operators can use proposal 255 to load balance their + hidden services on the introduction layer. See [SCALING-REFS] for further + discussions on this topic and alternative designs. + +1.6. In more detail: Backward compatibility with older hidden service + protocols + + This design is incompatible with the clients, server, and hsdir node + protocols from older versions of the hidden service protocol as + described in rend-spec.txt. On the other hand, it is designed to + enable the use of older Tor nodes as rendezvous points and + introduction points. + +1.7. In more detail: Keeping crypto keys offline + + In this design, a hidden service's secret identity key may be + stored offline. It's used only to generate blinded signing keys, + which are used to sign descriptor signing keys. + + In order to operate a hidden service, the operator can generate in + advance a number of blinded signing keys and descriptor signing + keys (and their credentials; see [DESC-OUTER] and [HS-DESC-ENC] + below), and their corresponding descriptor encryption keys, and + export those to the hidden service hosts. + + As a result, in the scenario where the Hidden Service gets + compromised, the adversary can only impersonate it for a limited + period of time (depending on how many signing keys were generated + in advance). + + It's important to not send the private part of the blinded signing + key to the Hidden Service since an attacker can derive from it the + secret master identity key. The secret blinded signing key should + only be used to create credentials for the descriptor signing keys. + +1.8. In more detail: Encryption Keys And Replay Resistance + + To avoid replays of an introduction request by an introduction point, + a hidden service host must never accept the same request + twice. Earlier versions of the hidden service design used an + authenticated timestamp here, but including a view of the current + time can create a problematic fingerprint. (See proposal 222 for more + discussion.) + +1.9. In more detail: A menagerie of keys + + [In the text below, an "encryption keypair" is roughly "a keypair you + can do Diffie-Hellman with" and a "signing keypair" is roughly "a + keypair you can do ECDSA with."] + + Public/private keypairs defined in this document: + + Master (hidden service) identity key -- A master signing keypair + used as the identity for a hidden service. This key is long + term and not used on its own to sign anything; it is only used + to generate blinded signing keys as described in [KEYBLIND] + and [SUBCRED]. The public key is encoded in the ".onion" + address according to [NAMING]. + + Blinded signing key -- A keypair derived from the identity key, + used to sign descriptor signing keys. It changes periodically for + each service. Clients who know a 'credential' consisting of the + service's public identity key and an optional secret can derive + the public blinded identity key for a service. This key is used + as an index in the DHT-like structure of the directory system + (see [SUBCRED]). + + Descriptor signing key -- A key used to sign hidden service + descriptors. This is signed by blinded signing keys. Unlike + blinded signing keys and master identity keys, the secret part + of this key must be stored online by hidden service hosts. The + public part of this key is included in the unencrypted section + of HS descriptors (see [DESC-OUTER]). + + Introduction point authentication key -- A short-term signing + keypair used to identify a hidden service to a given + introduction point. A fresh keypair is made for each + introduction point; these are used to sign the request that a + hidden service host makes when establishing an introduction + point, so that clients who know the public component of this key + can get their introduction requests sent to the right + service. No keypair is ever used with more than one introduction + point. (previously called a "service key" in rend-spec.txt) + + Introduction point encryption key -- A short-term encryption + keypair used when establishing connections via an introduction + point. Plays a role analogous to Tor nodes' onion keys. A fresh + keypair is made for each introduction point. + + Symmetric keys defined in this document: + + Descriptor encryption keys -- A symmetric encryption key used to + encrypt the body of hidden service descriptors. Derived from the + current period and the hidden service credential. + + Public/private keypairs defined elsewhere: + + Onion key -- Short-term encryption keypair + + (Node) identity key + + Symmetric key-like things defined elsewhere: + + KH from circuit handshake -- An unpredictable value derived as + part of the Tor circuit extension handshake, used to tie a request + to a particular circuit. + +1.9.1. In even more detail: Client authorization keys [CLIENT-AUTH] + + When client authorization is enabled, each authorized client of a hidden + service has two more assymetric keypairs which are shared with the hidden + service. An entity without those keys is not able to use the hidden + service. Throughout this document, we assume that these pre-shared keys are + exchanged between the hidden service and its clients in a secure out-of-band + fashion. + + Specifically, each authorized client possesses: + + - An x25519 keypair used to compute decryption keys that allow the client to + decrypt the hidden service descriptor. See [HS-DESC-ENC]. + + - An ed25519 keypair which allows the client to compute signatures which + prove to the hidden service that the client is authorized. These + signatures are inserted into the INTRODUCE1 cell, and without them the + introduction to the hidden service cannot be completed. See [INTRO-AUTH]. + + The right way to exchange these keys is to have the client generate keys and + send the corresponding public keys to the hidden service out-of-band. An + easier but less secure way of doing this exchange would be to have the + hidden service generate the keypairs and pass the corresponding private keys + to its clients. See section [CLIENT-AUTH-MGMT] for more details on how these + keys should be managed. + + [TODO: Also specify stealth client authorization.] + +2. Generating and publishing hidden service descriptors [HSDIR] + + Hidden service descriptors follow the same metaformat as other Tor + directory objects. They are published anonymously to Tor servers with the + HSDir flag, HSDir=2 protocol version and tor version >= 0.3.0.8 (because a + bug was fixed in this version). + +2.1. Deriving blinded keys and subcredentials [SUBCRED] + + In each time period (see [TIME-PERIODS] for a definition of time + periods), a hidden service host uses a different blinded private key + to sign its directory information, and clients use a different + blinded public key as the index for fetching that information. + + For a candidate for a key derivation method, see Appendix [KEYBLIND]. + + Additionally, clients and hosts derive a subcredential for each + period. Knowledge of the subcredential is needed to decrypt hidden + service descriptors for each period and to authenticate with the + hidden service host in the introduction process. Unlike the + credential, it changes each period. Knowing the subcredential, even + in combination with the blinded private key, does not enable the + hidden service host to derive the main credential--therefore, it is + safe to put the subcredential on the hidden service host while + leaving the hidden service's private key offline. + + The subcredential for a period is derived as: + + subcredential = H("subcredential" | credential | blinded-public-key). + + In the above formula, credential corresponds to: + + credential = H("credential" | public-identity-key) + + where public-identity-key is the public identity master key of the hidden + service. + +2.2. Locating, uploading, and downloading hidden service descriptors + [HASHRING] + + To avoid attacks where a hidden service's descriptor is easily + targeted for censorship, we store them at different directories over + time, and use shared random values to prevent those directories from + being predictable far in advance. + + Which Tor servers hosts a hidden service depends on: + + * the current time period, + * the daily subcredential, + * the hidden service directories' public keys, + * a shared random value that changes in each time period, + * a set of network-wide networkstatus consensus parameters. + (Consensus parameters are integer values voted on by authorities + and published in the consensus documents, described in + dir-spec.txt, section 3.3.) + + Below we explain in more detail. + +2.2.1. Dividing time into periods [TIME-PERIODS] + + To prevent a single set of hidden service directory from becoming a + target by adversaries looking to permanently censor a hidden service, + hidden service descriptors are uploaded to different locations that + change over time. + + The length of a "time period" is controlled by the consensus + parameter 'hsdir-interval', and is a number of minutes between 30 and + 14400 (10 days). The default time period length is 1440 (one day). + + Time periods start at the Unix epoch (Jan 1, 1970), and are computed by + taking the number of minutes since the epoch and dividing by the time + period. However, we want our time periods to start at 12:00UTC every day, so + we subtract a "rotation time offset" of 12*60 minutes from the number of + minutes since the epoch, before dividing by the time period (effectively + making "our" epoch start at Jan 1, 1970 12:00UTC). + + Example: If the current time is 2016-04-13 11:15:01 UTC, making the seconds + since the epoch 1460546101, and the number of minutes since the epoch + 24342435. We then subtract the "rotation time offset" of 12*60 minutes from + the minutes since the epoch, to get 24341715. If the current time period + length is 1440 minutes, by doing the division we see that we are currently + in time period number 16903. + + Specifically, time period #16903 began 16903*1440*60 + (12*60*60) seconds + after the epoch, at 2016-04-12 12:00 UTC, and ended at 16904*1440*60 + + (12*60*60) seconds after the epoch, at 2016-04-13 12:00 UTC. + +2.2.2. When to publish a hidden service descriptor [WHEN-HSDESC] + + Hidden services periodically publish their descriptor to the responsible + HSDirs. The set of responsible HSDirs is determined as specified in + [WHERE-HSDESC]. + + Specifically, everytime a hidden service publishes its descriptor, it also + sets up a timer for a random time between 60 minutes and 120 minutes in the + future. When the timer triggers, the hidden service needs to publish its + descriptor again to the responsible HSDirs for that time period. + [TODO: Control republish period using a consensus parameter?] + +2.2.2.1. Overlapping descriptors + + Hidden services need to upload multiple descriptors so that they can be + reachable to clients with older or newer consensuses than them. Services + need to upload their descriptors to the HSDirs _before_ the beginning of + each upcoming time period, so that they are readily available for clients to + fetch them. Furthermore, services should keep uploading their old descriptor + even after the end of a time period, so that they can be reachable by + clients that still have consensuses from the previous time period. + + Hence, services maintain two active descriptors at every point. Clients on + the other hand, don't have a notion of overlapping descriptors, and instead + always download the descriptor for the current time period and shared random + value. It's the job of the service to ensure that descriptors will be + available for all clients. See section [FETCHUPLOADDESC] for how this is + achieved. + + [TODO: What to do when we run multiple hidden services in a single host?] + +2.2.3. Where to publish a hidden service descriptor [WHERE-HSDESC] + + This section specifies how the HSDir hash ring is formed at any given + time. Whenever a time value is needed (e.g. to get the current time period + number), we assume that clients and services use the valid-after time from + their latest live consensus. + + The following consensus parameters control where a hidden service + descriptor is stored; + + hsdir_n_replicas = an integer in range [1,16] with default value 2. + hsdir_spread_fetch = an integer in range [1,128] with default value 3. + hsdir_spread_store = an integer in range [1,128] with default value 3. + + To determine where a given hidden service descriptor will be stored + in a given period, after the blinded public key for that period is + derived, the uploading or downloading party calculates: + + for replicanum in 1...hsdir_n_replicas: + hs_index(replicanum) = H("store-at-idx" | + blinded_public_key | + INT_8(replicanum) | + INT_8(period_length) | + INT_8(period_num) ) + + where blinded_public_key is specified in section [KEYBLIND], period_length + is the length of the time period in minutes, and period_num is calculated + using the current consensus "valid-after" as specified in section + [TIME-PERIODS]. + + Then, for each node listed in the current consensus with the HSDirV3 flag, + we compute a directory index for that node as: + + hsdir_index(node) = H("node-idx" | node_identity | + shared_random_value | + INT_8(period_num) | + INT_8(period_length) ) + + where shared_random_value is the shared value generated by the authorities + in section [PUB-SHAREDRANDOM], and node_identity is the ed25519 identity + key of the node. + + Finally, for replicanum in 1...hsdir_n_replicas, the hidden service + host uploads descriptors to the first hsdir_spread_store nodes whose + indices immediately follow hs_index(replicanum). If any of those + nodes have already been selected for a lower-numbered replica of the + service, any nodes already chosen are disregarded (i.e. skipped over) + when choosing a replica's hsdir_spread_store nodes. + + When choosing an HSDir to download from, clients choose randomly from + among the first hsdir_spread_fetch nodes after the indices. (Note + that, in order to make the system better tolerate disappearing + HSDirs, hsdir_spread_fetch may be less than hsdir_spread_store.) + Again, nodes from lower-numbered replicas are disregarded when + choosing the spread for a replica. + +2.2.4. Using time periods and SRVs to fetch/upload HS descriptors [FETCHUPLOADDESC] + + Hidden services and clients need to make correct use of time periods (TP) + and shared random values (SRVs) to successfuly fetch and upload + descriptors. Furthermore, to avoid problems with skewed clocks, both clients + and services use the 'valid-after' time of a live consensus as a way to take + decisions with regards to uploading and fetching descriptors. By using the + consensus times as the ground truth here, we minimize the desynchronization + of clients and services due to system clock. Whenever time-based decisions + are taken in this section, assume that they are consensus times and not + system times. + + As [PUB-SHAREDRANDOM] specifies, consensuses contain two shared random + values (the current one and the previous one). Hidden services and clients + are asked to match these shared random values with descriptor time periods + and use the right SRV when fetching/uploading descriptors. This section + attempts to precisely specify how this works. + + Let's start with an illustration of the system: + + +------------------------------------------------------------------+ + | | + | 00:00 12:00 00:00 12:00 00:00 12:00 | + | SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | + | | + | $==========|-----------$===========|-----------$===========| | + | | + | | + +------------------------------------------------------------------+ + + Legend: [TP#1 = Time Period #1] + [SRV#1 = Shared Random Value #1] + ["$" = descriptor rotation moment] + +2.2.4.1. Client behavior for fetching descriptors [CLIENTFETCH] + + And here is how clients use TPs and SRVs to fetch descriptors: + + Clients always aim to synchronize their TP with SRV, so they always want to + use TP#N with SRV#N: To achieve this wrt time periods, clients always use + the current time period when fetching descriptors. Now wrt SRVs, if a client + is in the time segment between a new time period and a new SRV (i.e. the + segments drawn with "-") it uses the current SRV, else if the client is in a + time segment between a new SRV and a new time period (i.e. the segments + drawn with "="), it uses the previous SRV. + + Example: + + +------------------------------------------------------------------+ + | | + | 00:00 12:00 00:00 12:00 00:00 12:00 | + | SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | + | | + | $==========|-----------$===========|-----------$===========| | + | ^ ^ | + | C1 C2 | + +------------------------------------------------------------------+ + + If a client (C1) is at 13:00 right after TP#1, then it will use TP#1 and + SRV#1 for fetching descriptors. Also, if a client (C2) is at 01:00 right + after SRV#2, it will still use TP#1 and SRV#1. + +2.2.4.2. Service behavior for uploading descriptors [SERVICEUPLOAD] + + As discussed above, services maintain two active descriptors at any time. We + call these the "first" and "second" service descriptors. Services rotate + their descriptor everytime they receive a consensus with a valid_after time + past the next SRV calculation time. They rotate their descriptors by + discarding their first descriptor, pushing the second descriptor to the + first, and rebuilding their second descriptor with the latest data. + + Services like clients also employ a different logic for picking SRV and TP + values based on their position in the graph above. Here is the logic: + +2.2.4.2.1. First descriptor upload logic [FIRSTDESCUPLOAD] + + Here is the service logic for uploading its first descriptor: + + When a service is in the time segment between a new time period a new SRV + (i.e. the segments drawn with "-"), it uses the previous time period and + previous SRV for uploading its first descriptor: that's meant to cover + for clients that have a consensus that is still in the previous time period. + + Example: Consider in the above illustration that the service is at 13:00 + right after TP#1. It will upload its first descriptor using TP#0 and SRV#0. + So if a client still has a 11:00 consensus it will be able to access it + based on the client logic above. + + Now if a service is in the time segment between a new SRV and a new time + period (i.e. the segments drawn with "=") it uses the current time period + and the previous SRV for its first descriptor: that's meant to cover clients + with an up-to-date consensus in the same time period as the service. + + Example: + + +------------------------------------------------------------------+ + | | + | 00:00 12:00 00:00 12:00 00:00 12:00 | + | SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | + | | + | $==========|-----------$===========|-----------$===========| | + | ^ | + | S | + +------------------------------------------------------------------+ + + Consider that the service is at 01:00 right after SRV#2: it will upload its + first descriptor using TP#1 and SRV#1. + +2.2.4.2.2. Second descriptor upload logic [SECONDDESCUPLOAD] + + Here is the service logic for uploading its second descriptor: + + When a service is in the time segment between a new time period a new SRV + (i.e. the segments drawn with "-"), it uses the current time period and + current SRV for uploading its second descriptor: that's meant to cover for + clients that have an up-to-date consensus on the same TP as the service. + + Example: Consider in the above illustration that the service is at 13:00 + right after TP#1: it will upload its second descriptor using TP#1 and SRV#1. + + Now if a service is in the time segment between a new SRV and a new time + period (i.e. the segments drawn with "=") it uses the next time period and + the current SRV for its second descriptor: that's meant to cover clients + with a newer consensus than the service (in the next time period). + + Example: + + +------------------------------------------------------------------+ + | | + | 00:00 12:00 00:00 12:00 00:00 12:00 | + | SRV#1 TP#1 SRV#2 TP#2 SRV#3 TP#3 | + | | + | $==========|-----------$===========|-----------$===========| | + | ^ | + | S | + +------------------------------------------------------------------+ + + Consider that the service is at 01:00 right after SRV#2: it will upload its + second descriptor using TP#2 and SRV#2. + +2.2.5. Expiring hidden service descriptors [EXPIRE-DESC] + + Hidden services set their descriptor's "descriptor-lifetime" field to 180 + minutes (3 hours). Hidden services ensure that their descriptor will remain + valid in the HSDir caches, by republishing their descriptors periodically as + specified in [WHEN-HSDESC]. + + Hidden services MUST also keep their introduction circuits alive for as long + as descriptors including those intro points are valid (even if that's after + the time period has changed). + +2.2.6. URLs for anonymous uploading and downloading + + Hidden service descriptors conforming to this specification are uploaded + with an HTTP POST request to the URL /tor/hs/<version>/publish relative to + the hidden service directory's root, and downloaded with an HTTP GET + request for the URL /tor/hs/<version>/<z> where <z> is a base64 encoding of + the hidden service's blinded public key and <version> is the protocol + version which is "3" in this case. + + These requests must be made anonymously, on circuits not used for + anything else. + +2.2.7. Client-side validation of onion addresses + + When a Tor client receives a prop224 onion address from the user, it + MUST first validate the onion address before attempting to connect or + fetch its descriptor. If the validation fails, the client MUST + refuse to connect. + + As part of the address validation, Tor clients should check that the + underlying ed25519 key does not have a torsion component. If Tor accepted + ed25519 keys with torsion components, attackers could create multiple + equivalent onion addresses for a single ed25519 key, which would map to the + same service. We want to avoid that because it could lead to phishing + attacks and surprising behaviors (e.g. imagine a browser plugin that blocks + onion addresses, but could be bypassed using an equivalent onion address + with a torsion component). + + The right way for clients to detect such fraudulent addresses (which should + only occur malevolently and never natutally) is to extract the ed25519 + public key from the onion address and multiply it by the ed25519 group order + and ensure that the result is the ed25519 identity element. For more + details, please see [TORSION-REFS]. + +2.3. Publishing shared random values [PUB-SHAREDRANDOM] + + Our design for limiting the predictability of HSDir upload locations + relies on a shared random value (SRV) that isn't predictable in advance or + too influenceable by an attacker. The authorities must run a protocol + to generate such a value at least once per hsdir period. Here we + describe how they publish these values; the procedure they use to + generate them can change independently of the rest of this + specification. For more information see [SHAREDRANDOM-REFS]. + + According to proposal 250, we add two new lines in consensuses: + + "shared-rand-previous-value" SP NUM_REVEALS SP VALUE NL + "shared-rand-current-value" SP NUM_REVEALS SP VALUE NL + +2.3.1. Client behavior in the absense of shared random values + + If the previous or current shared random value cannot be found in a + consensus, then Tor clients and services need to generate their own random + value for use when choosing HSDirs. + + To do so, Tor clients and services use: + + SRV = H("shared-random-disaster" | INT_8(period_length) | INT_8(period_num)) + + where period_length is the length of a time period in minutes, period_num is + calculated as specified in [TIME-PERIODS] for the wanted shared random value + that could not be found originally. + +2.3.2. Hidden services and changing shared random values + + It's theoretically possible that the consensus shared random values will + change or disappear in the middle of a time period because of directory + authorities dropping offline or misbehaving. + + To avoid client reachability issues in this rare event, hidden services + should use the new shared random values to find the new responsible HSDirs + and upload their descriptors there. + + XXX How long should they upload descriptors there for? + +2.4. Hidden service descriptors: outer wrapper [DESC-OUTER] + + The format for a hidden service descriptor is as follows, using the + meta-format from dir-spec.txt. + + "hs-descriptor" SP version-number NL + + [At start, exactly once.] + + The version-number is a 32 bit unsigned integer indicating the version + of the descriptor. Current version is "3". + + "descriptor-lifetime" SP LifetimeMinutes NL + + [Exactly once] + + The lifetime of a descriptor in minutes. An HSDir SHOULD expire the + hidden service descriptor at least LifetimeMinutes after it was + uploaded. + + The LifetimeMinutes field can take values between 30 and 3000 (50 hours). + + "descriptor-signing-key-cert" NL certificate NL + + [Exactly once.] + + The 'certificate' field contains a certificate in the format from + proposal 220, wrapped with "-----BEGIN ED25519 CERT-----". The + certificate cross-certifies the short-term descriptor signing key with + the blinded public key. The certificate type must be [08], and the + blinded public key must be present as the signing-key extension. + + "revision-counter" SP Integer NL + + [Exactly once.] + + The revision number of the descriptor. If an HSDir receives a + second descriptor for a key that it already has a descriptor for, + it should retain and serve the descriptor with the higher + revision-counter. + + (Checking for monotonically increasing revision-counter values + prevents an attacker from replacing a newer descriptor signed by + a given key with a copy of an older version.) + + "superencrypted" NL encrypted-string + + [Exactly once.] + + An encrypted blob, whose format is discussed in [HS-DESC-ENC] below. The + blob is base64 encoded and enclosed in -----BEGIN MESSAGE---- and + ----END MESSAGE---- wrappers. + + "signature" SP signature NL + + [exactly once, at end.] + + A signature of all previous fields, using the signing key in the + descriptor-signing-key-cert line, prefixed by the string "Tor onion + service descriptor sig v3". We use a separate key for signing, so that + the hidden service host does not need to have its private blinded key + online. + + HSDirs accept hidden service descriptors of up to 50k bytes (a consensus + parameter should also be introduced to control this value). + +2.5. Hidden service descriptors: encryption format [HS-DESC-ENC] + + Hidden service descriptors are protected by two layers of encryption. + Clients need to decrypt both layers to connect to the hidden service. + + The first layer of encryption provides confidentiality against entities who + don't know the public key of the hidden service (e.g. HSDirs), while the + second layer of encryption is only useful when client authorization is enabled + and protects against entities that do not possess valid client credentials. + +2.5.1. First layer of encryption [HS-DESC-FIRST-LAYER] + + The first layer of HS descriptor encryption is designed to protect + descriptor confidentiality against entities who don't know the blinded + public key of the hidden service. + +2.5.1.1. First layer encryption logic + + The encryption keys and format for the first layer of encryption are + generated as specified in [HS-DESC-ENCRYPTION-KEYS] with customization + parameters: + + SECRET_DATA = blinded-public-key + STRING_CONSTANT = "hsdir-superencrypted-data" + + The ciphertext is placed on the "superencrypted" field of the descriptor. + + Before encryption the plaintext is padded with NUL bytes to the nearest + multiple of 10k bytes. + +2.5.1.2. First layer plaintext format + + After clients decrypt the first layer of encryption, they need to parse the + plaintext to get to the second layer ciphertext which is contained in the + "encrypted" field. + + If client auth is enabled, the hidden service generates a fresh + descriptor_cookie key (32 random bytes) and encrypts it using each + authorized client's identity x25519 key. Authorized clients can use the + descriptor cookie to decrypt the second layer of encryption. Our encryption + scheme requires the hidden service to also generate an ephemeral x25519 + keypair for each new descriptor. + + If client auth is disabled, fake data is placed in each of the fields below + to obfuscate whether client authorization is enabled. + + Here are all the supported fields: + + "desc-auth-type" SP type NL + + [Exactly once] + + This field contains the type of authorization used to protect the + descriptor. The only recognized type is "x25519" and specifies the + encryption scheme described in this section. + + If client authorization is disabled, the value here should be "x25519". + + "desc-auth-ephemeral-key" SP key NL + + [Exactly once] + + This field contains an ephemeral x25519 public key generated by the + hidden service and encoded in base64. The key is used by the encryption + scheme below. + + If client authorization is disabled, the value here should be a fresh + x25519 pubkey that will remain unused. + + "auth-client" SP client-id SP iv SP encrypted-cookie + + [Any number] + + When client authorization is enabled, the hidden service inserts an + "auth-client" line for each of its authorized clients. If client + authorization is disabled, the fields here can be populated with random + data of the right size (that's 8 bytes for 'client-id', 16 bytes for 'iv' + and 16 bytes for 'encrypted-cookie' all encoded with base64). + + When client authorization is enabled, each "auth-client" line contains + the descriptor cookie encrypted to each individual client. We assume that + each authorized client possesses a pre-shared x25519 keypair which is + used to decrypt the descriptor cookie. + + We now describe the descriptor cookie encryption scheme. Here are the + relevant keys: + + client_x = private x25519 key of authorized client + client_X = public x25519 key of authorized client + hs_y = private key of ephemeral x25519 keypair of hidden service + hs_Y = public key of ephemeral x25519 keypair of hidden service + descriptor_cookie = descriptor cookie used to encrypt the descriptor + + And here is what the hidden service computes: + + SECRET_SEED = x25519(hs_y, client_X) + KEYS = KDF(SECRET_SEED, 40) + CLIENT-ID = fist 8 bytes of KEYS + COOKIE-KEY = last 32 bytes of KEYS + + Here is a description of the fields in the "auth-client" line: + + - The "client-id" field is CLIENT-ID from above encoded in base64. + + - The "iv" field is 16 random bytes encoded in base64. + + - The "encrypted-cookie" field contains the descriptor cookie ciphertext + as follows and is encoded in base64: + encrypted-cookie = STREAM(iv, COOKIE-KEY) XOR descriptor_cookie + + See section [FIRST-LAYER-CLIENT-BEHAVIOR] for the client-side logic of + how to decrypt the descriptor cookie. + + "encrypted" NL encrypted-string + + [Exactly once] + + An encrypted blob containing the second layer ciphertext, whose format is + discussed in [HS-DESC-SECOND-LAYER] below. The blob is base64 encoded + and enclosed in -----BEGIN MESSAGE---- and ----END MESSAGE---- wrappers. + +2.5.1.3. Client behavior [FIRST-LAYER-CLIENT-BEHAVIOR] + + The goal of clients at this stage is to decrypt the "encrypted" field as + described in [HS-DESC-SECOND-LAYER]. + + If client authorization is enabled, authorized clients need to extract the + descriptor cookie to proceed with decryption of the second layer as + follows: + + An authorized client parsing the first layer of an encrypted descriptor, + extracts the ephemeral key from "desc-auth-ephemeral-key" and calculates + CLIENT-ID and COOKIE-KEY as described in the section above using their + x25519 private key. The client then uses CLIENT-ID to find the right + "auth-client" field which contains the ciphertext of the descriptor + cookie. The client then uses COOKIE-KEY and the iv to decrypt the + descriptor_cookie, which is used to decrypt the second layer of descriptor + encryption as described in [HS-DESC-SECOND-LAYER]. + +2.5.1.4. Hiding client authorization data + + Hidden services should avoid leaking whether client authorization is + enabled or how many authorized clients there are. + + Hence even when client authorization is disabled, the hidden service adds + fake "desc-auth-type", "desc-auth-ephemeral-key" and "auth-client" lines to + the descriptor, as described in [HS-DESC-FIRST-LAYER]. + + The hidden service also avoids leaking the number of authorized clients by + adding fake "auth-client" entries to its descriptor. Specifically, + descriptors always contain a number of authorized clients that is a + multiple of 16 by adding fake "auth-client" entries if needed. + [XXX consider randomization of the value 16] + + Clients MUST accept descriptors with any number of "auth-client" lines as + long as the total descriptor size is within the max limit of 50k (also + controlled with a consensus parameter). + +2.5.2. Second layer of encryption [HS-DESC-SECOND-LAYER] + + The second layer of descriptor encryption is designed to protect descriptor + confidentiality against unauthorized clients. If client authorization is + enabled, it's encrypted using the descriptor_cookie, and contains needed + information for connecting to the hidden service, like the list of its + introduction points. + + If client authorization is disabled, then the second layer of HS encryption + does not offer any additional security, but is still used. + +2.5.2.1. Second layer encryption keys + + The encryption keys and format for the second layer of encryption are + generated as specified in [HS-DESC-ENCRYPTION-KEYS] with customization + parameters as follows: + + SECRET_DATA = blinded-public-key | descriptor_cookie + STRING_CONSTANT = "hsdir-encrypted-data" + + If client authorization is disabled the 'descriptor_cookie' field is left blank. + + The ciphertext is placed on the "encrypted" field of the descriptor. + +2.5.2.2. Second layer plaintext format + + After decrypting the second layer ciphertext, clients can finally learn the + list of intro points etc. The plaintext has the following format: + + "create2-formats" SP formats NL + + [Exactly once] + + A space-separated list of integers denoting CREATE2 cell format numbers + that the server recognizes. Must include at least ntor as described in + tor-spec.txt. See tor-spec section 5.1 for a list of recognized + handshake types. + + "intro-auth-required" SP types NL + + [At most once] + + A space-separated list of introduction-layer authentication types; see + section [INTRO-AUTH] for more info. A client that does not support at + least one of these authentication types will not be able to contact the + host. Recognized types are: 'password' and 'ed25519'. + + "single-onion-service" + + [None or at most once] + + If present, this line indicates that the service is a Single Onion + Service (see prop260 for more details about that type of service). This + field has been introduced in 0.3.0 meaning 0.2.9 service don't include + this. + + Followed by zero or more introduction points as follows (see section + [NUM_INTRO_POINT] below for accepted values): + + "introduction-point" SP link-specifiers NL + + [Exactly once per introduction point at start of introduction + point section] + + The link-specifiers is a base64 encoding of a link specifier + block in the format described in BUILDING-BLOCKS. + + "onion-key" SP "ntor" SP key NL + + [Exactly once per introduction point] + + The key is a base64 encoded curve25519 public key which is the onion + key of the introduction point Tor node used for the ntor handshake + when a client extends to it. + + "auth-key" NL certificate NL + + [Exactly once per introduction point] + + The certificate is a proposal 220 certificate wrapped in + "-----BEGIN ED25519 CERT-----", cross-certifying the descriptor + signing key with the introduction point authentication key, which + is included in the mandatory signing-key extension. The certificate + type must be [09]. + + "enc-key" SP "ntor" SP key NL + + [Exactly once per introduction point] + + The key is a base64 encoded curve25519 public key used to encrypt + the introduction request to service. + + "enc-key-cert" NL certificate NL + + [Exactly once per introduction point] + + Cross-certification of the descriptor signing key by the encryption + key. + + For "ntor" keys, certificate is a proposal 220 certificate wrapped + in "-----BEGIN ED25519 CERT-----" armor, cross-certifying the + descriptor signing key with the ed25519 equivalent of a curve25519 + public encryption key derived using the process in proposal 228 + appendix A. The certificate type must be [0B], and the signing-key + extension is mandatory. + + "legacy-key" NL key NL + + [None or at most once per introduction point] + + The key is an ASN.1 encoded RSA public key in PEM format used for a + legacy introduction point as described in [LEGACY_EST_INTRO]. + + This field is only present if the introduction point only supports + legacy protocol (v2) that is <= 0.2.9 or the protocol version value + "HSIntro 3". + + "legacy-key-cert NL certificate NL + + [None or at most once per introduction point] + + MUST be present if "legacy-key" is present. + + The certificate is a proposal 220 RSA->Ed cross-certificate wrapped + in "-----BEGIN CROSSCERT-----" armor, cross-certifying the + descriptor signing key with the RSA public key found in + "legacy-key". + + To remain compatible with future revisions to the descriptor format, + clients should ignore unrecognized lines in the descriptor. + Other encryption and authentication key formats are allowed; clients + should ignore ones they do not recognize. + + Clients who manage to extract the introduction points of the hidden service + can prroceed with the introduction protocol as specified in [INTRO-PROTOCOL]. + +2.5.3. Deriving hidden service descriptor encryption keys [HS-DESC-ENCRYPTION-KEYS] + + In this section we present the generic encryption format for hidden service + descriptors. We use the same encryption format in both encryption layers, + hence we introduce two customization parameters SECRET_DATA and + STRING_CONSTANT which vary between the layers. + + The SECRET_DATA parameter specifies the secret data that are used during + encryption key generation, while STRING_CONSTANT is merely a string constant + that is used as part of the KDF. + + Here is the key generation logic: + + SALT = 16 bytes from H(random), changes each time we rebuld the + descriptor even if the content of the descriptor hasn't changed. + (So that we don't leak whether the intro point list etc. changed) + + secret_input = SECRET_DATA | subcredential | INT_8(revision_counter) + + keys = KDF(secret_input | salt | STRING_CONSTANT, S_KEY_LEN + S_IV_LEN + MAC_KEY_LEN) + + SECRET_KEY = first S_KEY_LEN bytes of keys + SECRET_IV = next S_IV_LEN bytes of keys + MAC_KEY = last MAC_KEY_LEN bytes of keys + + The encrypted data has the format: + + SALT hashed random bytes from above [16 bytes] + ENCRYPTED The ciphertext [variable] + MAC MAC of both above fields [32 bytes] + + The final encryption format is ENCRYPTED = STREAM(SECRET_IV,SECRET_KEY) XOR Plaintext + +2.5.4. Number of introduction points [NUM_INTRO_POINT] + + This section defines how many introduction points an hidden service + descriptor can have at minimum, by default and the maximum: + + Minimum: 0 - Default: 3 - Maximum: 20 + + A value of 0 would means that the service is still alive but doesn't want + to be reached by any client at the moment. Note that the descriptor size + increases considerably as more introduction points are added. + + The reason for a maximum value of 20 is to give enough scalability to tools + like OnionBalance to be able to load balance up to 120 servers (20 x 6 + HSDirs) but also in order for the descriptor size to not overwhelmed hidden + service directories with user defined values that could be gigantic. + +3. The introduction protocol [INTRO-PROTOCOL] + + The introduction protocol proceeds in three steps. + + First, a hidden service host builds an anonymous circuit to a Tor + node and registers that circuit as an introduction point. + + [After 'First' and before 'Second', the hidden service publishes its + introduction points and associated keys, and the client fetches + them as described in section [HSDIR] above.] + + Second, a client builds an anonymous circuit to the introduction + point, and sends an introduction request. + + Third, the introduction point relays the introduction request along + the introduction circuit to the hidden service host, and acknowledges + the introduction request to the client. + +3.1. Registering an introduction point [REG_INTRO_POINT] + +3.1.1. Extensible ESTABLISH_INTRO protocol. [EST_INTRO] + + When a hidden service is establishing a new introduction point, it + sends an ESTABLISH_INTRO cell with the following contents: + + AUTH_KEY_TYPE [1 byte] + AUTH_KEY_LEN [2 bytes] + AUTH_KEY [AUTH_KEY_LEN bytes] + N_EXTENSIONS [1 byte] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + HANDSHAKE_AUTH [MAC_LEN bytes] + SIG_LEN [2 bytes] + SIG [SIG_LEN bytes] + + The AUTH_KEY_TYPE field indicates the type of the introduction point + authentication key and the type of the MAC to use in + HANDSHAKE_AUTH. Recognized types are: + + [00, 01] -- Reserved for legacy introduction cells; see + [LEGACY_EST_INTRO below] + [02] -- Ed25519; SHA3-256. + + The AUTH_KEY_LEN field determines the length of the AUTH_KEY + field. The AUTH_KEY field contains the public introduction point + authentication key. + + The EXT_FIELD_TYPE, EXT_FIELD_LEN, EXT_FIELD entries are reserved for + future extensions to the introduction protocol. Extensions with + unrecognized EXT_FIELD_TYPE values must be ignored. + + The HANDSHAKE_AUTH field contains the MAC of all earlier fields in + the cell using as its key the shared per-circuit material ("KH") + generated during the circuit extension protocol; see tor-spec.txt + section 5.2, "Setting circuit keys". It prevents replays of + ESTABLISH_INTRO cells. + + SIG_LEN is the length of the signature. + + SIG is a signature, using AUTH_KEY, of all contents of the cell, up + to but not including SIG. These contents are prefixed with the string + "Tor establish-intro cell v1". + + Upon receiving an ESTABLISH_INTRO cell, a Tor node first decodes the + key and the signature, and checks the signature. The node must reject + the ESTABLISH_INTRO cell and destroy the circuit in these cases: + + * If the key type is unrecognized + * If the key is ill-formatted + * If the signature is incorrect + * If the HANDSHAKE_AUTH value is incorrect + + * If the circuit is already a rendezvous circuit. + * If the circuit is already an introduction circuit. + [TODO: some scalability designs fail there.] + * If the key is already in use by another circuit. + + Otherwise, the node must associate the key with the circuit, for use + later in INTRODUCE1 cells. + +3.1.2. Registering an introduction point on a legacy Tor node + [LEGACY_EST_INTRO] + + Tor nodes should also support an older version of the ESTABLISH_INTRO + cell, first documented in rend-spec.txt. New hidden service hosts + must use this format when establishing introduction points at older + Tor nodes that do not support the format above in [EST_INTRO]. + + In this older protocol, an ESTABLISH_INTRO cell contains: + + KEY_LEN [2 bytes] + KEY [KEY_LEN bytes] + HANDSHAKE_AUTH [20 bytes] + SIG [variable, up to end of relay payload] + + The KEY_LEN variable determines the length of the KEY field. + + The KEY field is the ASN1-encoded legacy RSA public key that was also + included in the hidden service descriptor. + + The HANDSHAKE_AUTH field contains the SHA1 digest of (KH | "INTRODUCE"). + + The SIG field contains an RSA signature, using PKCS1 padding, of all + earlier fields. + + Older versions of Tor always use a 1024-bit RSA key for these introduction + authentication keys. + +3.1.3. Acknowledging establishment of introduction point [INTRO_ESTABLISHED] + + After setting up an introduction circuit, the introduction point reports its + status back to the hidden service host with an INTRO_ESTABLISHED cell. + + The INTRO_ESTABLISHED cell has the following contents: + + N_EXTENSIONS [1 byte] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + + Older versions of Tor send back an empty INTRO_ESTABLISHED cell instead. + Services must accept an empty INTRO_ESTABLISHED cell from a legacy relay. + +3.2. Sending an INTRODUCE1 cell to the introduction point. [SEND_INTRO1] + + In order to participate in the introduction protocol, a client must + know the following: + + * An introduction point for a service. + * The introduction authentication key for that introduction point. + * The introduction encryption key for that introduction point. + + The client sends an INTRODUCE1 cell to the introduction point, + containing an identifier for the service, an identifier for the + encryption key that the client intends to use, and an opaque blob to + be relayed to the hidden service host. + + In reply, the introduction point sends an INTRODUCE_ACK cell back to + the client, either informing it that its request has been delivered, + or that its request will not succeed. + + [TODO: specify what tor should do when receiving a malformed cell. Drop it? + Kill circuit? This goes for all possible cells.] + +3.2.1. INTRODUCE1 cell format [FMT_INTRO1] + + When a client is connecting to an introduction point, INTRODUCE1 cells + should be of the form: + + LEGACY_KEY_ID [20 bytes] + AUTH_KEY_TYPE [1 byte] + AUTH_KEY_LEN [2 bytes] + AUTH_KEY [AUTH_KEY_LEN bytes] + N_EXTENSIONS [1 byte] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + ENCRYPTED [Up to end of relay payload] + + AUTH_KEY_TYPE is defined as in [EST_INTRO]. Currently, the only value of + AUTH_KEY_TYPE for this cell is an Ed25519 public key [02]. + + The LEGACY_KEY_ID field is used to distinguish between legacy and new style + INTRODUCE1 cells. In new style INTRODUCE1 cells, LEGACY_KEY_ID is 20 zero + bytes. Upon receiving an INTRODUCE1 cell, the introduction point checks the + LEGACY_KEY_ID field. If LEGACY_KEY_ID is non-zero, the INTRODUCE1 cell + should be handled as a legacy INTRODUCE1 cell by the intro point. + + Upon receiving a INTRODUCE1 cell, the introduction point checks + whether AUTH_KEY matches the introduction point authentication key for an + active introduction circuit. If so, the introduction point sends an + INTRODUCE2 cell with exactly the same contents to the service, and sends an + INTRODUCE_ACK response to the client. + +3.2.2. INTRODUCE_ACK cell format. [INTRO_ACK] + + An INTRODUCE_ACK cell has the following fields: + + STATUS [2 bytes] + N_EXTENSIONS [1 bytes] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + + Recognized status values are: + [00 00] -- Success: cell relayed to hidden service host. + [00 01] -- Failure: service ID not recognized + [00 02] -- Bad message format + [00 03] -- Can't relay cell to service + +3.3. Processing an INTRODUCE2 cell at the hidden service. [PROCESS_INTRO2] + + Upon receiving an INTRODUCE2 cell, the hidden service host checks whether + the AUTH_KEY or LEGACY_KEY_ID field matches the keys for this + introduction circuit. + + The service host then checks whether it has received a cell with these + contents or rendezvous cookie before. If it has, it silently drops it as a + replay. (It must maintain a replay cache for as long as it accepts cells + with the same encryption key. Note that the encryption format below should + be non-malleable.) + + If the cell is not a replay, it decrypts the ENCRYPTED field, + establishes a shared key with the client, and authenticates the whole + contents of the cell as having been unmodified since they left the + client. There may be multiple ways of decrypting the ENCRYTPED field, + depending on the chosen type of the encryption key. Requirements for + an introduction handshake protocol are described in + [INTRO-HANDSHAKE-REQS]. We specify one below in section + [NTOR-WITH-EXTRA-DATA]. + + The decrypted plaintext must have the form: + + RENDEZVOUS_COOKIE [20 bytes] + N_EXTENSIONS [1 byte] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + ONION_KEY_TYPE [1 bytes] + ONION_KEY_LEN [2 bytes] + ONION_KEY [ONION_KEY_LEN bytes] + NSPEC (Number of link specifiers) [1 byte] + NSPEC times: + LSTYPE (Link specifier type) [1 byte] + LSLEN (Link specifier length) [1 byte] + LSPEC (Link specifier) [LSLEN bytes] + PAD (optional padding) [up to end of plaintext] + + Upon processing this plaintext, the hidden service makes sure that + any required authentication is present in the extension fields, and + then extends a rendezvous circuit to the node described in the LSPEC + fields, using the ONION_KEY to complete the extension. As mentioned + in [BUILDING-BLOCKS], the "TLS-over-TCP, IPv4" and "Legacy node + identity" specifiers must be present. + + The hidden service SHOULD NOT reject any LSTYPE fields which it + doesn't recognize; instead, it should use them verbatim in its EXTEND + request to the rendezvous point. + + The ONION_KEY_TYPE field is: + + [01] NTOR: ONION_KEY is 32 bytes long. + + The ONION_KEY field describes the onion key that must be used when + extending to the rendezvous point. It must be of a type listed as + supported in the hidden service descriptor. + + When using a legacy introduction point, the INTRODUCE cells must be padded + to a certain length using the PAD field in the encrypted portion. + + Upon receiving a well-formed INTRODUCE2 cell, the hidden service host + will have: + + * The information needed to connect to the client's chosen + rendezvous point. + * The second half of a handshake to authenticate and establish a + shared key with the hidden service client. + * A set of shared keys to use for end-to-end encryption. + +3.3.1. Introduction handshake encryption requirements [INTRO-HANDSHAKE-REQS] + + When decoding the encrypted information in an INTRODUCE2 cell, a + hidden service host must be able to: + + * Decrypt additional information included in the INTRODUCE2 cell, + to include the rendezvous token and the information needed to + extend to the rendezvous point. + + * Establish a set of shared keys for use with the client. + + * Authenticate that the cell has not been modified since the client + generated it. + + Note that the old TAP-derived protocol of the previous hidden service + design achieved the first two requirements, but not the third. + +3.3.2. Example encryption handshake: ntor with extra data + [NTOR-WITH-EXTRA-DATA] + + [TODO: relocate this] + + This is a variant of the ntor handshake (see tor-spec.txt, section + 5.1.4; see proposal 216; and see "Anonymity and one-way + authentication in key-exchange protocols" by Goldberg, Stebila, and + Ustaoglu). + + It behaves the same as the ntor handshake, except that, in addition + to negotiating forward secure keys, it also provides a means for + encrypting non-forward-secure data to the server (in this case, to + the hidden service host) as part of the handshake. + + Notation here is as in section 5.1.4 of tor-spec.txt, which defines + the ntor handshake. + + The PROTOID for this variant is "tor-hs-ntor-curve25519-sha3-256-1". + We also use the following tweak values: + + t_hsenc = PROTOID | ":hs_key_extract" + t_hsverify = PROTOID | ":hs_verify" + t_hsmac = PROTOID | ":hs_mac" + m_hsexpand = PROTOID | ":hs_key_expand" + + To make an INTRODUCE1 cell, the client must know a public encryption + key B for the hidden service on this introduction circuit. The client + generates a single-use keypair: + x,X = KEYGEN() + and computes: + intro_secret_hs_input = EXP(B,x) | AUTH_KEY | X | B | PROTOID + info = m_hsexpand | subcredential + hs_keys = KDF(intro_secret_hs_input | t_hsenc | info, S_KEY_LEN+MAC_LEN) + ENC_KEY = hs_keys[0:S_KEY_LEN] + MAC_KEY = hs_keys[S_KEY_LEN:S_KEY_LEN+MAC_KEY_LEN] + + and sends, as the ENCRYPTED part of the INTRODUCE1 cell: + + CLIENT_PK [PK_PUBKEY_LEN bytes] + ENCRYPTED_DATA [Padded to length of plaintext] + MAC [MAC_LEN bytes] + + + Substituting those fields into the INTRODUCE1 cell body format + described in [FMT_INTRO1] above, we have + + LEGACY_KEY_ID [20 bytes] + AUTH_KEY_TYPE [1 byte] + AUTH_KEY_LEN [2 bytes] + AUTH_KEY [AUTH_KEY_LEN bytes] + N_EXTENSIONS [1 bytes] + N_EXTENSIONS times: + EXT_FIELD_TYPE [1 byte] + EXT_FIELD_LEN [1 byte] + EXT_FIELD [EXT_FIELD_LEN bytes] + ENCRYPTED: + CLIENT_PK [PK_PUBKEY_LEN bytes] + ENCRYPTED_DATA [Padded to length of plaintext] + MAC [MAC_LEN bytes] + + + (This format is as documented in [FMT_INTRO1] above, except that here + we describe how to build the ENCRYPTED portion.) + + Here, the encryption key plays the role of B in the regular ntor + handshake, and the AUTH_KEY field plays the role of the node ID. + The CLIENT_PK field is the public key X. The ENCRYPTED_DATA field is + the message plaintext, encrypted with the symmetric key ENC_KEY. The + MAC field is a MAC of all of the cell from the AUTH_KEY through the + end of ENCRYPTED_DATA, using the MAC_KEY value as its key. + + To process this format, the hidden service checks PK_VALID(CLIENT_PK) + as necessary, and then computes ENC_KEY and MAC_KEY as the client did + above, except using EXP(CLIENT_PK,b) in the calculation of + intro_secret_hs_input. The service host then checks whether the MAC is + correct. If it is invalid, it drops the cell. Otherwise, it computes + the plaintext by decrypting ENCRYPTED_DATA. + + The hidden service host now completes the service side of the + extended ntor handshake, as described in tor-spec.txt section 5.1.4, + with the modified PROTOID as given above. To be explicit, the hidden + service host generates a keypair of y,Y = KEYGEN(), and uses its + introduction point encryption key 'b' to computes: + + intro_secret_hs_input = EXP(X,b) | AUTH_KEY | X | B | PROTOID + info = m_hsexpand | subcredential + hs_keys = KDF(intro_secret_hs_input | t_hsenc | info, S_KEY_LEN+MAC_LEN) + HS_DEC_KEY = hs_keys[0:S_KEY_LEN] + HS_MAC_KEY = hs_keys[S_KEY_LEN:S_KEY_LEN+MAC_KEY_LEN] + + (The above are used to check the MAC and then decrypt the + encrypted data.) + + rend_secret_hs_input = EXP(X,y) | EXP(X,b) | AUTH_KEY | B | X | Y | PROTOID + NTOR_KEY_SEED = MAC(rend_secret_hs_input, t_hsenc) + verify = MAC(rend_secret_hs_input, t_hsverify) + auth_input = verify | AUTH_KEY | B | Y | X | PROTOID | "Server" + AUTH_INPUT_MAC = MAC(auth_input, t_hsmac) + + (The above are used to finish the ntor handshake.) + + The server's handshake reply is: + SERVER_PK Y [PK_PUBKEY_LEN bytes] + AUTH AUTH_INPUT_MAC [MAC_LEN bytes] + + These fields will be sent to the client in a RENDEZVOUS1 cell using the + HANDSHAKE_INFO element (see [JOIN_REND]). + + The hidden service host now also knows the keys generated by the + handshake, which it will use to encrypt and authenticate data + end-to-end between the client and the server. These keys are as + computed in tor-spec.txt section 5.1.4. + +3.4. Authentication during the introduction phase. [INTRO-AUTH] + + Hidden services may restrict access only to authorized users. + One mechanism to do so is the credential mechanism, where only users who + know the credential for a hidden service may connect at all. + +3.4.1. Ed25519-based authentication. + + To authenticate with an Ed25519 private key, the user must include an + extension field in the encrypted part of the INTRODUCE1 cell with an + EXT_FIELD_TYPE type of [02] and the contents: + + Nonce [16 bytes] + Pubkey [32 bytes] + Signature [64 bytes] + + Nonce is a random value. Pubkey is the public key that will be used + to authenticate. [TODO: should this be an identifier for the public + key instead?] Signature is the signature, using Ed25519, of: + + "hidserv-userauth-ed25519" + Nonce (same as above) + Pubkey (same as above) + AUTH_KEY (As in the INTRODUCE1 cell) + + The hidden service host checks this by seeing whether it recognizes + and would accept a signature from the provided public key. If it + would, then it checks whether the signature is correct. If it is, + then the correct user has authenticated. + + Replay prevention on the whole cell is sufficient to prevent replays + on the authentication. + + Users SHOULD NOT use the same public key with multiple hidden + services. + +4. The rendezvous protocol + + Before connecting to a hidden service, the client first builds a + circuit to an arbitrarily chosen Tor node (known as the rendezvous + point), and sends an ESTABLISH_RENDEZVOUS cell. The hidden service + later connects to the same node and sends a RENDEZVOUS cell. Once + this has occurred, the relay forwards the contents of the RENDEZVOUS + cell to the client, and joins the two circuits together. + +4.1. Establishing a rendezvous point [EST_REND_POINT] + + The client sends the rendezvous point a RELAY_COMMAND_ESTABLISH_RENDEZVOUS + cell containing a 20-byte value. + + RENDEZVOUS_COOKIE [20 bytes] + + Rendezvous points MUST ignore any extra bytes in an + ESTABLISH_RENDEZVOUS cell. (Older versions of Tor did not.) + + The rendezvous cookie is an arbitrary 20-byte value, chosen randomly + by the client. The client SHOULD choose a new rendezvous cookie for + each new connection attempt. If the rendezvous cookie is already in + use on an existing circuit, the rendezvous point should reject it and + destroy the circuit. + + Upon receiving an ESTABLISH_RENDEZVOUS cell, the rendezvous point associates + the cookie with the circuit on which it was sent. It replies to the client + with an empty RENDEZVOUS_ESTABLISHED cell to indicate success. Clients MUST + ignore any extra bytes in a RENDEZVOUS_ESTABLISHED cell. + + The client MUST NOT use the circuit which sent the cell for any + purpose other than rendezvous with the given location-hidden service. + + The client should establish a rendezvous point BEFORE trying to + connect to a hidden service. + +4.2. Joining to a rendezvous point [JOIN_REND] + + To complete a rendezvous, the hidden service host builds a circuit to + the rendezvous point and sends a RENDEZVOUS1 cell containing: + + RENDEZVOUS_COOKIE [20 bytes] + HANDSHAKE_INFO [variable; depends on handshake type + used.] + + where RENDEZVOUS_COOKIE is the cookie suggested by the client during the + introduction (see [PROCESS_INTRO2]) and HANDSHAKE_INFO is defined in + [NTOR-WITH-EXTRA-DATA]. + + If the cookie matches the rendezvous cookie set on any + not-yet-connected circuit on the rendezvous point, the rendezvous + point connects the two circuits, and sends a RENDEZVOUS2 cell to the + client containing the HANDSHAKE_INFO field of the RENDEZVOUS1 cell. + + Upon receiving the RENDEZVOUS2 cell, the client verifies that HANDSHAKE_INFO + correctly completes a handshake. To do so, the client parses SERVER_PK from + HANDSHAKE_INFO and reverses the final operations of section + [NTOR-WITH-EXTRA-DATA] as shown here: + + rend_secret_hs_input = EXP(Y,x) | EXP(B,x) | AUTH_KEY | B | X | Y | PROTOID + NTOR_KEY_SEED = MAC(ntor_secret_input, t_hsenc) + verify = MAC(ntor_secret_input, t_hsverify) + auth_input = verify | AUTH_KEY | B | Y | X | PROTOID | "Server" + AUTH_INPUT_MAC = MAC(auth_input, t_hsmac) + + Finally the client verifies that the received AUTH field of HANDSHAKE_INFO + is equal to the computed AUTH_INPUT_MAC. + + Now both parties use the handshake output to derive shared keys for use on + the circuit as specified in the section below: + +4.2.1. Key expansion + + The hidden service and its client need to derive crypto keys from the + NTOR_KEY_SEED part of the handshake output. To do so, they use the KDF + construction as follows: + + K = KDF(NTOR_KEY_SEED | m_hsexpand, HASH_LEN * 2 + S_KEY_LEN * 2) + + The first HASH_LEN bytes of K form the forward digest Df; the next HASH_LEN + bytes form the backward digest Db; the next S_KEY_LEN bytes form Kf, and the + final S_KEY_LEN bytes form Kb. Excess bytes from K are discarded. + + Subsequently, the rendezvous point passes relay cells, unchanged, from each + of the two circuits to the other. When Alice's OP sends RELAY cells along + the circuit, it authenticates with Df, and encrypts them with the Kf, then + with all of the keys for the ORs in Alice's side of the circuit; and when + Alice's OP receives RELAY cells from the circuit, it decrypts them with the + keys for the ORs in Alice's side of the circuit, then decrypts them with Kb, + and checks integrity with Db. Bob's OP does the same, with Kf and Kb + interchanged. + + [TODO: Should we encrypt HANDSHAKE_INFO as we did INTRODUCE2 + contents? It's not necessary, but it could be wise. Similarly, we + should make it extensible.] + +4.3. Using legacy hosts as rendezvous points + + The behavior of ESTABLISH_RENDEZVOUS is unchanged from older versions + of this protocol, except that relays should now ignore unexpected + bytes at the end. + + Old versions of Tor required that RENDEZVOUS cell payloads be exactly + 168 bytes long. All shorter rendezvous payloads should be padded to + this length with random bytes, to make them difficult to distinguish from + older protocols at the rendezvous point. + + Relays older than 0.2.9.1 should not be used for rendezvous points by next + generation onion services because they enforce too-strict length checks to + rendezvous cells. Hence the "HSRend" protocol from proposal#264 should be + used to select relays for rendezvous points. + +5. Encrypting data between client and host + + A successfully completed handshake, as embedded in the + INTRODUCE/RENDEZVOUS cells, gives the client and hidden service host + a shared set of keys Kf, Kb, Df, Db, which they use for sending + end-to-end traffic encryption and authentication as in the regular + Tor relay encryption protocol, applying encryption with these keys + before other encryption, and decrypting with these keys before other + decryption. The client encrypts with Kf and decrypts with Kb; the + service host does the opposite. + +6. Encoding onion addresses [ONIONADDRESS] + + The onion address of a hidden service includes its identity public key, a + version field and a basic checksum. All this information is then base32 + encoded as shown below: + + onion_address = base32(PUBKEY | CHECKSUM | VERSION) + ".onion" + CHECKSUM = H(".onion checksum" | PUBKEY | VERSION)[:2] + + where: + - PUBKEY is the 32 bytes ed25519 master pubkey of the hidden service. + - VERSION is an one byte version field (default value '\x03') + - ".onion checksum" is a constant string + - CHECKSUM is truncated to two bytes before inserting it in onion_address + + Here are a few example addresses: + + pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd.onion + sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd.onion + xa4r2iadxm55fbnqgwwi5mymqdcofiu3w6rpbtqn7b2dyn7mgwj64jyd.onion + + For more information about this encoding, please see our discussion thread + at [ONIONADDRESS-REFS]. + +7. Open Questions: + + Scaling hidden services is hard. There are on-going discussions that + you might be able to help with. See [SCALING-REFS]. + + How can we improve the HSDir unpredictability design proposed in + [SHAREDRANDOM]? See [SHAREDRANDOM-REFS] for discussion. + + How can hidden service addresses become memorable while retaining + their self-authenticating and decentralized nature? See + [HUMANE-HSADDRESSES-REFS] for some proposals; many more are possible. + + Hidden Services are pretty slow. Both because of the lengthy setup + procedure and because the final circuit has 6 hops. How can we make + the Hidden Service protocol faster? See [PERFORMANCE-REFS] for some + suggestions. + +References: + +[KEYBLIND-REFS]: + https://trac.torproject.org/projects/tor/ticket/8106 + https://lists.torproject.org/pipermail/tor-dev/2012-September/004026.html + +[KEYBLIND-PROOF]: + https://lists.torproject.org/pipermail/tor-dev/2013-December/005943.html + +[SHAREDRANDOM-REFS]: + https://gitweb.torproject.org/torspec.git/tree/proposals/250-commit-reveal-c... + https://trac.torproject.org/projects/tor/ticket/8244 + +[SCALING-REFS]: + https://lists.torproject.org/pipermail/tor-dev/2013-October/005556.html + +[HUMANE-HSADDRESSES-REFS]: + https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/ideas/xxx-oni... + http://archives.seul.org/or/dev/Dec-2011/msg00034.html + +[PERFORMANCE-REFS]: + "Improving Efficiency and Simplicity of Tor circuit + establishment and hidden services" by Overlier, L., and + P. Syverson + + [TODO: Need more here! Do we have any? :( ] + +[ATTACK-REFS]: + "Trawling for Tor Hidden Services: Detection, Measurement, + Deanonymization" by Alex Biryukov, Ivan Pustogarov, + Ralf-Philipp Weinmann + + "Locating Hidden Servers" by Lasse Øverlier and Paul + Syverson + +[ED25519-REFS]: + "High-speed high-security signatures" by Daniel + J. Bernstein, Niels Duif, Tanja Lange, Peter Schwabe, and + Bo-Yin Yang. http://cr.yp.to/papers.html#ed25519 + +[ED25519-B-REF]: + https://tools.ietf.org/html/draft-josefsson-eddsa-ed25519-03#section-5: + +[PRNG-REFS]: + http://projectbullrun.org/dual-ec/ext-rand.html + https://lists.torproject.org/pipermail/tor-dev/2015-November/009954.html + +[SRV-TP-REFS]: + https://lists.torproject.org/pipermail/tor-dev/2016-April/010759.html + +[VANITY-REFS]: + https://github.com/Yawning/horse25519 + +[ONIONADDRESS-REFS]: + https://lists.torproject.org/pipermail/tor-dev/2017-January/011816.html + +[TORSION-REFS]: + https://lists.torproject.org/pipermail/tor-dev/2017-April/012164.html + https://getmonero.org/2017/05/17/disclosure-of-a-major-bug-in-cryptonote-bas... + +Appendix A. Signature scheme with key blinding [KEYBLIND] + +A.1. Key derivation overview + + As described in [IMD:DIST] and [SUBCRED] above, we require a "key + blinding" system that works (roughly) as follows: + + There is a master keypair (sk, pk). + + Given the keypair and a nonce n, there is a derivation function + that gives a new blinded keypair (sk_n, pk_n). This keypair can + be used for signing. + + Given only the public key and the nonce, there is a function + that gives pk_n. + + Without knowing pk, it is not possible to derive pk_n; without + knowing sk, it is not possible to derive sk_n. + + It's possible to check that a signature was made with sk_n while + knowing only pk_n. + + Someone who sees a large number of blinded public keys and + signatures made using those public keys can't tell which + signatures and which blinded keys were derived from the same + master keypair. + + You can't forge signatures. + + [TODO: Insert a more rigorous definition and better references.] + +A.2. Tor's key derivation scheme + + We propose the following scheme for key blinding, based on Ed25519. + + (This is an ECC group, so remember that scalar multiplication is the + trapdoor function, and it's defined in terms of iterated point + addition. See the Ed25519 paper [Reference ED25519-REFS] for a fairly + clear writeup.) + + Let B be the ed25519 basepoint as found in section 5 of [ED25519-B-REF]: + B = (15112221349535400772501151409588531511454012693041857206046113283949847762202, + 46316835694926478169428394003475163141307993866256225615783033603165251855960) + + Assume B has prime order l, so lB=0. Let a master keypair be written as + (a,A), where a is the private key and A is the public key (A=aB). + + To derive the key for a nonce N and an optional secret s, compute the + blinding factor like this: + + h = H(BLIND_STRING | A | s | B | N) + BLIND_STRING = "Derive temporary signing key" + N = "key-blind" | INT_8(period-number) | INT_8(period_length) + + then clamp the blinding factor 'h' according to the ed25519 spec: + + h[0] &= 248; + h[31] &= 127; + h[31] |= 64; + + and do the key derivation as follows: + + private key for the period: a' = h a + public key for the period: A' = h A = (ha)B + + Generating a signature of M: given a deterministic random-looking r + (see EdDSA paper), take R=rB, S=r+hash(R,A',M)ah mod l. Send signature + (R,S) and public key A'. + + Verifying the signature: Check whether SB = R+hash(R,A',M)A'. + + (If the signature is valid, + SB = (r + hash(R,A',M)ah)B + = rB + (hash(R,A',M)ah)B + = R + hash(R,A',M)A' ) + + See [KEYBLIND-REFS] for an extensive discussion on this scheme and + possible alternatives. Also, see [KEYBLIND-PROOF] for a security + proof of this scheme. + +Appendix B. Selecting nodes [PICKNODES] + + Picking introduction points + Picking rendezvous points + Building paths + Reusing circuits + + (TODO: This needs a writeup) + +Appendix C. Recommendations for searching for vanity .onions [VANITY] + + EDITORIAL NOTE: The author thinks that it's silly to brute-force the + keyspace for a key that, when base-32 encoded, spells out the name of + your website. It also feels a bit dangerous to me. If you train your + users to connect to + + llamanymityx4fi3l6x2gyzmtmgxjyqyorj9qsb5r543izcwymle.onion + + I worry that you're making it easier for somebody to trick them into + connecting to + + llamanymityb4sqi0ta0tsw6uovyhwlezkcrmczeuzdvfauuemle.onion + + Nevertheless, people are probably going to try to do this, so here's a + decent algorithm to use. + + To search for a public key with some criterion X: + + Generate a random (sk,pk) pair. + + While pk does not satisfy X: + + Add the number 8 to sk + Add the scalar 8*B to pk + + Return sk, pk. + + We add 8 and 8*B, rather than 1 and B, so that sk is always a valid + Curve25519 private key, with the lowest 3 bits equal to 0. + + This algorithm is safe [source: djb, personal communication] [TODO: + Make sure I understood correctly!] so long as only the final (sk,pk) + pair is used, and all previous values are discarded. + + To parallelize this algorithm, start with an independent (sk,pk) pair + generated for each independent thread, and let each search proceed + independently. + + See [VANITY-REFS] for a reference implementation of this vanity .onion + search scheme. + +Appendix D. Numeric values reserved in this document + + [TODO: collect all the lists of commands and values mentioned above] + +Appendix E. Reserved numbers + + We reserve these certificate type values for Ed25519 certificates: + + [08] short-term descriptor signing key, signed with blinded + public key. (Section 2.4) + [09] intro point authentication key, cross-certifying the descriptor + signing key. (Section 2.5) + [0B] ed25519 key derived from the curve25519 intro point encryption key, + cross-certifying the descriptor signing key. (Section 2.5) + + Note: The value "0A" is skipped because it's reserved for the onion key + cross-certifying ntor identity key from proposal 228. + +Appendix F. Hidden service directory format [HIDSERVDIR-FORMAT] + + This appendix section specifies the contents of the HiddenServiceDir directory: + + - "hostname" [FILE] + + This file contains the onion address of the onion service. + + - "private_key_ed25519" [FILE] + + This file contains the private master ed25519 key of the onion service. + [TODO: Offline keys] + + - "client_authorized_pubkeys" [FILE] + + If client authorization is _enabled_, this is a newline-separated file of + "<client name> <pubkeys> entries for authorized clients. You can think of it + as the ~/.ssh/authorized_keys of onion services. See [CLIENT-AUTH-MGMT] for + more details. + + - "./client_authorized_privkeys/" [DIRECTORY] + "./client_authorized_privkeys/alice.privkey" [FILE] + "./client_authorized_privkeys/bob.privkey" [FILE] + "./client_authorized_privkeys/charlie.privkey" [FILE] + + If client authorization is _enabled_ _AND_ if the hidden service is + responsible for generating and distributing private keys for its clients, + then this directory contains files with client's private keys. See + [CLIENT-AUTH-MGMT] for more details. + +Appendix E. Managing authorized client data [CLIENT-AUTH-MGMT] + + Hidden services and clients can configure their authorized client data either + using the torrc, or using the control port. This section presents a suggested + scheme for configuring client authorization. Please see appendix + [HIDSERVDIR-FORMAT] for more information about relevant hidden service files. + + E.1. Configuring client authorization using torrc + + E.1.1. Hidden Service side + + A hidden service that wants to perform client authorization, adds a new + option HiddenServiceAuthorizeClient to its torrc file: + + HiddenServiceAuthorizeClient auth-type client-name,client-name,... + + The only recognized auth-type value is "basic" which describes the scheme in + section [CLIENT-AUTH]. The rest of the line is a comma-separated list of + human-readable authorized client names. + + Let's consider that one of the listed client names is "alice". In this + case, Tor checks the "client_authorized_pubkeys" file for any entries + with client_name being "alice". If an "alice" entry is found, we use the + relevant pubkeys to authenticate Alice. + + If no "alice" entry is found in the "client_authorized_pubkeys" file, Tor + is tasked with generating public/private keys for Alice. To do so, Tor + generates x25519 and ed25519 keypairs for Alice, then makes a + "client_authorized_privkeys/alice.privkey" file and writes the private + keys inside; it also adds an entry for alice to the + "client_authorized_pubkeys" file. + + In this last case, the hidden service operator has the responsibility to + pass the .key file to Alice in a secure out-of-band way. After the file + is passed to Alice, it can be shredded from the filesystem, as only the + public keys are required for the hidden service to function. + + E.1.2. Client side + + A client who wants to register client authorization data for a hidden service + needs to add the following line to their torrc: + + HidServAuth onion-address x25519-private-key ed25519-private-key + + The keys above are either generated by Alice using a key generation utility, + or they are extracted from a .key file provided by the hidden service. + + In the former case, the client is also tasked with transfering the public + keys to the hidden service in a secure out-of-band way. + + E.2. Configuring client authorization using the control port + + E.2.1. Service side + + A hidden service also has the option to configure authorized clients + using the control port. The idea is that hidden service operators can use + controller utilities that manage their access control instead of using + the filesystem to register client keys. + + Specifically, we require a new control port command ADD_ONION_CLIENT_AUTH + which is able to register x25519/ed25519 public keys tied to a specific + authorized client. + [XXX figure out control port command format] + + Hidden services who use the control port interface for client auth need + to perform their own key management. + + E.2.2. Client side + + There should also be a control port interface for clients to register + authorization data for hidden services without having to use the + torrc. It should allow both generation of client authorization private + keys, and also to import client authorization data provided by a hidden + service + + This way, Tor Browser can present "Generate client auth keys" and "Import + client auth keys" dialogs to users when they try to visit a hidden service + that is protected by client authorization. + + Specifically, we require two new control port commands: + IMPORT_ONION_CLIENT_AUTH_DATA + GENERATE_ONION_CLIENT_AUTH_DATA + which import and generate client authorization data respectively. + + [XXX how does key management work here?] + [XXX what happens when people use both the control port interface and the + filesystem interface?] +