Hi, all!
I've been trying to fill in all the cracks and corners for a revamp of
the hidden services protocol, based on earlier writings by George
Kadianakis and other discussions on the mailing list. (See draft
acknowledgments section below.)
After a bunch of comments, I'm ready to give this a number and call it
(draft) proposal 224. I'd like to know what doesn't make sense, what
I need to explain better, and what I need to design better. I'd like
to fill in the gaps and turn this into a …
[View More]more full document. I'd like
to answer the open questions. Comments are most welcome, especially if
they grow into improvements.
FWIW, I am likely to be offline for most of the current weekend,
because of Thanksgiving, so please be patient with my reply speed; I
hope to catch up with emails next week.
Filename: 224-rend-spec-ng.txt
Title: Next-Generation Hidden Services in Tor
Author: Nick Mathewson
Created: 2013-11-29
Status: Draft
-1. Draft notes
This document describes a proposed design and specification for
hidden services in Tor version 0.2.5.x or later. It's a replacement
for the current rend-spec.txt, rewritten for clarity and for improved
design.
Look for the string "TODO" below: it describes gaps or uncertainties
in the design.
Change history:
2013-11-29: Proposal first numbered. Some TODO and XXX items remain.
0. Hidden services: overview and preliminaries.
Hidden services aim to provide responder anonymity for bidirectional
stream-based communication on the Tor network. Unlike regular Tor
connections, where the connection initiator receives anonymity but
the responder does not, hidden services attempt to provide
bidirectional anonymity.
Other features include:
* [TODO: WRITE ME once there have been some more drafts and we know
what the summary should say.]
Participants:
Operator -- A person running a hidden service
Host, "Server" -- The Tor software run by the operator to provide
a hidden service.
User -- A person contacting a hidden service.
Client -- The Tor software running on the User's computer
Hidden Service Directory (HSDir) -- A Tor node that hosts signed
statements from hidden service hosts so that users can make
contact with them.
Introduction Point -- A Tor node that accepts connection requests
for hidden services and anonymously relays those requests to the
hidden service.
Rendezvous Point -- A Tor node to which clients and servers
connect and which relays traffic between them.
0.1. Improvements over previous versions.
[TODO write me once there have been more drafts and we know what the
summary should say.]
0.2. Notation and vocabulary
Unless specified otherwise, all multi-octet integers are big-endian.
We write sequences of bytes in two ways:
1. A sequence of two-digit hexadecimal values in square brackets,
as in [AB AD 1D EA].
2. A string of characters enclosed in quotes, as in "Hello". These
characters in these string are encoded in their ascii
representations; strings are NOT nul-terminated unless
explicitly described as NUL terminated.
We use the words "byte" and "octet" interchangeably.
We use the vertical bar | to denote concatenation.
We use INT_N(val) to denote the network (big-endian) encoding of the
unsigned integer "val" in N bytes. For example, INT_4(1337) is [00 00
05 39].
0.3. Cryptographic building blocks
This specification uses the following cryptographic building blocks:
* A stream cipher STREAM(iv, k) where iv is a nonce of length
S_IV_LEN bytes and k is a key of length S_KEY_LEN bytes.
* A public key signature system SIGN_KEYGEN()->seckey, pubkey;
SIGN_SIGN(seckey,msg)->sig; and SIGN_CHECK(pubkey, sig, msg) ->
{ "OK", "BAD" }; where secret keys are of length SIGN_SECKEY_LEN
bytes, public keys are of length SIGN_PUBKEY_LEN bytes, and
signatures are of length SIGN_SIG_LEN bytes.
This signature system must also support key blinding operations
as discussed in appendix [KEYBLIND] and in section [SUBCRED]:
SIGN_BLIND_SECKEY(seckey, blind)->seckey2 and
SIGN_BLIND_PUBKEY(pubkey, blind)->pubkey2 .
* A public key agreement system "PK", providing
PK_KEYGEN()->seckey, pubkey; PK_VALID(pubkey) -> {"OK", "BAD"};
and PK_HANDHAKE(seckey, pubkey)->output; where secret keys are
of length PK_SECKEY_LEN bytes, public keys are of length
PK_PUBKEY_LEN bytes, and the handshake produces outputs of
length PK_OUTPUT_LEN bytes.
* A cryptographic hash function H(d), which should be preimage and
collision resistant. It produces hashes of length HASH_LEN
bytes.
* A cryptographic message authentication code MAC(key,msg) that
produces outputs of length MAC_LEN bytes.
* A key derivation function KDF(key data, salt, personalization,
n) that outputs n bytes.
As a first pass, I suggest:
* Instantiate STREAM with AES128-CTR. [TODO: or ChaCha20?]
* Instantiate SIGN with Ed25519 and the blinding protocol in
[KEYBLIND].
* Instantiate PK with Curve25519.
* Instantiate H with SHA256. [TODO: really?]
* Instantiate MAC with HMAC using H.
* Instantiate KDF with HKDF using H.
For legacy purposes, we specify compatibility with older versions of
the Tor introduction point and rendezvous point protocols. These used
RSA1024, DH1024, AES128, and SHA1, as discussed in
rend-spec.txt. Except as noted, all RSA keys MUST have exponent
values of 65537.
As in [proposal 220], all signatures are generated not over strings
themselves, but over those strings prefixed with a distinguishing
value.
0.4. Protocol building blocks [BUILDING-BLOCKS]
In sections below, we need to transmit the locations and identities
of Tor nodes. We do so in the link identification format used by
EXTEND2 cells in the Tor protocol.
NSPEC (Number of link specifiers) [1 byte]
NSPEC times:
LSTYPE (Link specifier type) [1 byte]
LSLEN (Link specifier length) [1 byte]
LSPEC (Link specifier) [LSLEN bytes]
Link specifier types are as described in tor-spec.txt. Every set of
link specifiers MUST include at minimum specifiers of type [00]
(TLS-over-TCP, IPv4) and [02] (legacy node identity).
We also incorporate Tor's circuit extension handshakes, as used in
the CREATE2 and CREATED2 cells described in tor-spec.txt. In these
handshakes, a client who knows a public key for a server sends a
message and receives a message from that server. Once the exchange is
done, the two parties have a shared set of forward-secure key
material, and the client knows that nobody else shares that key
material unless they control the secret key corresponding to the
server's public key.
0.5. Assigned relay cell types
These relay cell types are reserved for use in the hidden service
protocol.
32 -- RELAY_COMMAND_ESTABLISH_INTRO
Sent from hidden service host to introduction point;
establishes introduction point. Discussed in
[REG_INTRO_POINT].
33 -- RELAY_COMMAND_ESTABLISH_RENDEZVOUS
Sent from client to rendezvous point; creates rendezvous
point. Discussed in [EST_REND_POINT].
34 -- RELAY_COMMAND_INTRODUCE1
Sent from client to introduction point; requests
introduction. Discussed in [SEND_INTRO1]
35 -- RELAY_COMMAND_INTRODUCE2
Sent from client to introduction point; requests
introduction. Same format as INTRODUCE1. Discussed in
[FMT_INTRO1] and [PROCESS_INTRO2]
36 -- RELAY_COMMAND_RENDEZVOUS1
Sent from introduction point to rendezvous point;
attempts to join introduction point's circuit to
client's circuit. Discussed in [JOIN_REND]
37 -- RELAY_COMMAND_RENDEZVOUS2
Sent from introduction point to rendezvous point;
reports join of introduction point's circuit to
client's circuit. Discussed in [JOIN_REND]
38 -- RELAY_COMMAND_INTRO_ESTABLISHED
Sent from introduction point to hidden service host;
reports status of attempt to establish introduction
point. Discussed in [INTRO_ESTABLISHED]
39 -- RELAY_COMMAND_RENDEZVOUS_ESTABLISHED
Sent from rendezvous point to client; acknowledges
receipt of ESTABLISH_RENDEZVOUS cell. Discussed in
[EST_REND_POINT]
40 -- RELAY_COMMAND_INTRODUCE_ACK
Sent form introduction point to client; acknowledges
receipt of INTRODUCE1 cell and reports success/failure.
Discussed in [INTRO_ACK]
0.5. Acknowledgments
[TODO reformat these once the lists are more complete.]
This design includes ideas from many people, including
Christopher Baines,
Daniel J. Bernstein,
Matthew Finkel,
Ian Goldberg,
George Kadianakis,
Aniket Kate,
Tanja Lange,
Robert Ransom,
It's based on Tor's original hidden service design by Roger
Dingledine, Nick Mathewson, and Paul Syverson, and on improvements to
that design over the years by people including
Tobias Kamm,
Thomas Lauterbach,
Karsten Loesing,
Alessandro Preite Martinez,
Robert Ransom,
Ferdinand Rieger,
Christoph Weingarten,
Christian Wilms,
We wouldn't be able to do any of this work without good attack
designs from researchers including
Alex Biryukov,
Lasse Øverlier,
Ivan Pustogarov,
Paul Syverson
Ralf-Philipp Weinmann,
See [ATTACK-REFS] for their papers.
Several of these ideas have come from conversations with
Christian Grothoff,
Brian Warner,
Zooko Wilcox-O'Hearn,
And if this document makes any sense at all, it's thanks to
editing help from
Matthew Finkel
George Kadianakis,
Peter Palfrader,
[XXX Acknowledge the huge bunch of people working on 8106.]
[XXX Acknowledge the huge bunch of people working on 8244.]
Please forgive me if I've missed you; please forgive me if I've
misunderstood your best ideas here too.
1. Protocol overview
In this section, we outline the hidden service protocol. This section
omits some details in the name of simplicity; those are given more
fully below, when we specify the protocol in more detail.
1.1. View from 10,000 feet
A hidden service host prepares to offer a hidden service by choosing
several Tor nodes to serve as its introduction points. It builds
circuits to those nodes, and tells them to forward introduction
requests to it using those circuits.
Once introduction points have been picked, the host builds a set of
documents called "hidden service descriptors" (or just "descriptors"
for short) and uploads them to a set of HSDir nodes. These documents
list the hidden service's current introduction points and describe
how to make contact with the hidden service.
When a client wants to connect to a hidden service, it first chooses
a Tor node at random to be its "rendezvous point" and builds a
circuit to that rendezvous point. If the client does not have an
up-to-date descriptor for the service, it contacts an appropriate
HSDir and requests such a descriptor.
The client then builds an anonymous circuit to one of the hidden
service's introduction points listed in its descriptor, and gives the
introduction point an introduction request to pass to the hidden
service. This introduction request includes the target rendezvous
point and the first part of a cryptographic handshake.
Upon receiving the introduction request, the hidden service host
makes an anonymous circuit to the rendezvous point and completes the
cryptographic handshake. The rendezvous point connects the two
circuits, and the cryptographic handshake gives the two parties a
shared key and proves to the client that it is indeed talking to the
hidden service.
Once the two circuits are joined, the client can send Tor RELAY cells
to the server. RELAY_BEGIN cells open streams to an external process
or processes configured by the server; RELAY_DATA cells are used to
communicate data on those streams, and so forth.
1.2. In more detail: naming hidden services [NAMING]
A hidden service's name is its long term master identity key. This
is encoded as a hostname by encoding the entire key in Base 32, and
adding the string ".onion" at the end.
(This is a change from older versions of the hidden service protocol,
where we used an 80-bit truncated SHA1 hash of a 1024 bit RSA key.)
The names in this format are distinct from earlier names because of
their length. An older name might look like:
unlikelynamefora.onion
yyhws9optuwiwsns.onion
And a new name following this specification might look like:
a1uik0w1gmfq3i5ievxdm9ceu27e88g6o7pe0rffdw9jmntwkdsd.onion
Note that since master keys are 32 bytes long, and 52 bytes of base
32 encoding can hold 260 bits of information, we have four unused
bits in each of these names.
[TODO: Alternatively, we could require that the first bit of the
master key always be zero, and use a 51-byte encoding. Or we could
require that the first two bits be zero, and use a 51-byte encoding
and reserve the first bit. Or we could require that the first nine
bits, or ten bits be zero, etc.]
1.3. In more detail: Access control [IMD:AC]
Access control for a hidden service is imposed at multiple points
through the process above.
In order to download a descriptor, clients must know which blinded
signing key was used to sign it. (See the next section for more info
on key blinding.) This blinded signing key is derived from the
service's public key and, optionally, an additional secret that is
not part of the hidden service's onion address. The public key and
this secret together constitute the service's "credential".
When the secret is in use, the hidden service gains protections
equivalent to the "stealth mode" in previous designs.
To learn the introduction points, the clients must decrypt the body
of the hidden service descriptor. The encryption key for these is
derived from the service's credential.
In order to make an introduction point send a request to the server,
the client must know the introduction point and know the service's
per-introduction-point authentication key from the hidden service
descriptor.
The final level of access control happens at the server itself, which
may decide to respond or not respond to the client's request
depending on the contents of the request. The protocol is extensible
at this point: at a minimum, the server requires that the client
demonstrate knowledge od the contents of the encrypted portion of the
hidden service descriptor. The service may additionally require a
user- or group-specific access token before it responds to requests.
1.4. In more detail: Distributing hidden service descriptors. [IMD:DIST]
Periodically, hidden service descriptors become stored at different
locations to prevent a single directory or small set of directories
from becoming a good DoS target for removing a hidden service.
For each period, the Tor directory authorities agree upon a
collaboratively generated random value. (See section 2.3 for a
description of how to incorporate this value into the voting
practice; generating the value is described in other proposals,
including [TODO: add a reference]) That value, combined with hidden service
directories' public identity keys, determines each HSDirs' position
in the hash ring for descriptors made in that period.
Each hidden service's descriptors are placed into the ring in
positions based on the key that was used to sign them. Note that
hidden service descriptors are not signed with the services' public
keys directly. Instead, we use a key-blinding system [KEYBLIND] to
create a new key-of-the-day for each hidden service. Any client that
knows the hidden service's credential can derive these blinded
signing keys for a given period. It should be impossible to derive
the blinded signing key lacking that credential.
The body of each descriptor is also encrypted with a key derived from
the credential.
To avoid a "thundering herd" problem where every service generates
and uploads a new descriptor at the start of each period, each
descriptor comes online at a time during the period that depends on
its blinded signing key. The keys for the last period remain valid
until the new keys come online.
1.5. In more detail: Scaling to multiple hosts
[THIS SECTION IS UNFINISHED]
In order to allow multiple hosts to provide a single hidden service,
I'm considering two options.
* We can have each server build an introduction circuit to each
introduction point, and have the introduction points responsible
for round-robining between these circuits. One service host is
responsible for picking the introduction points and publishing
the descriptors.
* We can have servers choose their introduction points
independently, and build circuits to them. One service host is
responsible for combining these introduction points into a
single descriptor.
If we want to avoid having a single "master" host without which the
whole service goes down (the "one service host" in the description
above), we need a way to fail over from one host to another. We also
need a way to coordinate between the hosts. This is as yet
undesigned. Maybe it should use a hidden service?
See [SCALING-REFS] for discussion on this topic.
[TODO: Finalize this design.]
1.6. In more detail: Backward compatibility with older hidden service
protocols
This design is incompatible with the clients, server, and hsdir node
protocols from older versions of the hidden service protocol as
described in rend-spec.txt. On the other hand, it is designed to
enable the use of older Tor nodes as rendezvous points and
introduction points.
1.7. In more detail: Offline operation
In this design, a hidden service's secret identity key may be stored
offline. It's used only to generate blinded identity keys, which are
used to sign descriptor signing keys. In order to operate a hidden
service, the operator can generate a number of descriptor signing
keys and their certifications (see [DESC-OUTER] and [ENCRYPTED-DATA]
below), and their corresponding descriptor encryption keys, and
export those to the hidden service hosts.
1.8. In more detail: Encryption Keys And Replay Resistance
To avoid replays of an introduction request by an introduction point,
a hidden service host must never accept the same request
twice. Earlier versions of the hidden service design used a
authenticated timestamp here, but including a view of the current
time can create a problematic fingerprint. (See proposal 222 for more
discussion.)
1.9. In more detail: A menagerie of keys
[In the text below, an "encryption keypair" is roughly "a keypair you
can do Diffie-Hellman with" and a "signing keypair" is roughly "a
keypair you can do ECDSA with."]
Public/private keypairs defined in this document:
Master (hidden service) identity key -- A master signing keypair
used as the identity for a hidden service. This key is not used
on its own to sign anything; it is only used to generate blinded
signing keys as described in [KEYBLIND] and [SUBCRED].
Blinded signing key -- A keypair derived from the identity key,
used to sign descriptor signing keys. Changes periodically for
each service. Clients who know a 'credential' consisting of the
service's public identity key and an optional secret can derive
the public blinded identity key for a service. This key is used
as an index in the DHT-like structure of the directory system.
Descriptor signing key -- A key used to sign hidden service
descriptors. This is signed by blinded signing keys. Unlike
blinded signing keys and master identity keys, the secret part
of this key must be stored online by hidden service hosts.
Introduction point authentication key -- A short-term signing
keypair used to identify a hidden service to a given
introduction point. A fresh keypair is made for each
introduction point; these are used to sign the request that a
hidden service host makes when establishing an introduction
point, so that clients who know the public component of this key
can get their introduction requests sent to the right
service. No keypair is ever used with more than one introduction
point. (previously called a "service key" in rend-spec.txt)
Introduction point encryption key -- A short-term encryption
keypair used when establishing connections via an introduction
point. Plays a role analogous to Tor nodes' onion keys. A fresh
keypair is made for each introduction point.
Symmetric keys defined in this document:
Descriptor encryption keys -- A symmetric encryption key used to
encrypt the body of hidden service descriptors. Derived from the
current period and the hidden service credential.
Public/private keypairs defined elsewhere:
Onion key -- Short-term encryption keypair
(Node) identity key
Symmetric key-like things defined elsewhere:
KH from circuit handshake -- An unpredictable value derived as
part of the Tor circuit extension handshake, used to tie a request
to a particular circuit.
2. Generating and publishing hidden service descriptors [HSDIR]
Hidden service descriptors follow the same metaformat as other Tor
directory objects. They are published anonymously to Tor servers with
the HSDir3 flag.
(Authorities should assign this flag as they currently assign the
HSDir flag, except that they should restrict it to Tor versions
implementing the HSDir parts of this specification.)
2.1. Deriving blinded keys and subcredentials [SUBCRED]
In each time period (see [TIME-PERIOD] for a definition of time
periods), a hidden service host uses a different blinded private key
to sign its directory information, and clients use a different
blinded public key as the index for fetching that information.
For a candidate for a key derivation method, see Appendix [KEYBLIND].
Additionally, clients and hosts derive a subcredential for each
period. Knowledge of the subcredential is needed to decrypt hidden
service descriptors for each period and to authenticate with the
hidden service host in the introduction process. Unlike the
credential, it changes each period. Knowing the subcredential, even
in combination with the blinded private key, does not enable the
hidden service host to derive the main credential--therefore, it is
safe to put the subcredential on the hidden service host while
leaving the hidden service's private key offline.
The subcredential for a period is derived as:
H("subcredential" |
credential |
blinded-public-key).
2.2. Locating, uploading, and downloading hidden service descriptors
[HASHRING]
To avoid attacks where a hidden service's descriptor is easily
targeted for censorship, we store them at different directories over
time, and use shared random values to prevent those directories from
being predictable far in advance.
Which Tor servers hosts a hidden service depends on:
* the current time period,
* the daily subcredential,
* the hidden service directories' public keys,
* a shared random value that changes in each time period,
* a set of network-wide networkstatus consensus parameters.
Below we explain in more detail.
2.2.1. Dividing time into periods [TIME-PERIODS]
To prevent a single set of hidden service directory from becoming a
target by adversaries looking to permanently censor a hidden service,
hidden service descriptors are uploaded to different locations that
change over time.
The length of a "time period" is controlled by the consensus
parameter 'hsdir-interval', and is a number of minutes between 30 and
14400 (10 days). The default time period length is 1500 (one day plus
one hour).
Time periods start with the Unix epoch (Jan 1, 1970), and are
computed by taking the number of whole minutes since the epoch and
dividing by the time period. So if the current time is 2013-11-12
13:44:32 UTC, making the seconds since the epoch 1384281872, the
number of minutes since the epoch is 23071364. If the current time
period length is 1500 (the default), then the current time period
number is 15380. It began 15380*1500*60 seconds after the epoch at
2013-11-11 20:00:00 UTC, and will end at (15380+1)*1500*60 seconds
after the epoch at 2013-11-12 21:00:00 UTC.
2.2.2. Overlapping time periods to avoid thundering herds [TIME-OVERLAP]
If every hidden service host were to generate a new set of keys and
upload a new descriptor at exactly the start of each time period, the
directories would be overwhelmed by every host uploading at the same
time. Instead, each public key becomes valid at its new location at a
deterministic time somewhat _before_ the period begins, depending on
the public key and the period.
The time at which a key might first become valid is determined by the
consensus parameter "hsdir-overlap-begins", which is an integer in
range [1,100] with default value 80. This parameter denotes a
percentage of the interval for which no overlap occurs. So for the
default interval (1500 minutes) and default overlap-begins value
(80%), new keys do not become valid for the first 1200 minutes of the
interval.
The new shared random value must be published *before* the start of
the next overlap interval by at least enough time to ensure that
clients all get it. [TODO: how much earlier?]
The time at which a key from the next interval becomes valid is
determined by taking the first two bytes of
OFFSET = H(Key | INT_8(Next_Period_Num))
as a big-endian integer, dividing by 65536, and treating that as a
fraction of the overlap interval.
For example, if the period is 1500 minutes long, and overlap interval
is 300 minutes long, and OFFSET begins with [90 50], then the next
key becomes valid at 1200 + 300 * (0x9050 / 65536) minutes, or
approximately 22 hours and 49 minutes after the beginning of the
period.
Hidden service directories should accept descriptors at least [TODO:
how much?] minutes before they would become valid, and retain them
for at least [TODO: how much?] minutes after the end of the period.
When a client is looking for a service, it must calculate its key
both for the current and for the subsequent period, to decide whether
the next period's key is valid yet.
2.2.3. Where to publish a service descriptor
The following consensus parameters control where a hidden service
descriptor is stored;
hsdir_n_replicas = an integer in range [1,16]
with default value 2.
hsdir_spread_fetch = an integer in range [1,128]
with default value 3.
hsdir_spread_store = an integer in range [1,128]
with default value 3.
hsdir_spread_accept = an integer in range [1,128]
with default value 8.
To determine where a given hidden service descriptor will be stored
in a given period, after the blinded public key for that period is
derived, the uploading or downloading party calculate
for replicanum in 1...hsdir_n_replicas:
hs_index(replicanum) = H("store-at-idx" |
blinded_public_key | replicanum |
periodnum)
where blinded_public_key is specified in section KEYBLIND, and
periodnum is defined in section TIME-PERIODS.
where n_replicas is determined by the consensus parameter
"hsdir_n_replicas".
Then, for each node listed in the current consensus with the HSDir3
flag, we compute a directory index for that node as:
hsdir_index(node) = H(node_identity_digest |
shared_random |
INT_8(period_num) )
where shared_random is the shared value generated by the authorities
in section PUB-SHAREDRANDOM.
Finally, for replicanum in 1...hsdir_n_replicas, the hidden service
host uploads descriptors to the first hsdir_spread_store nodes whose
indices immediately follow hs_index(replicanum).
When choosing an HSDir to download from, clients choose randomly from
among the first hsdir_spread_fetch nodes after the indices. (Note
that, in order to make the system better tolerate disappearing
HSDirs, hsdir_spread_fetch may be less than hsdir_spread_store.)
An HSDir should rejects a descriptor if that HSDir is not one of the
first hsdir_spread_accept HSDirs for that node.
[TODO: Incorporate the findings from proposal 143 here. But watch
out: proposal 143 did not analyze how much the set of nodes changes
over time, or how much client and host knowledge might diverge.]
2.2.4. URLs for anonymous uploading and downloading
Hidden service descriptors conforming to this specification are
uploaded with an HTTP POST request to the URL
/tor/rendezvous3/publish relative to the hidden service directory's
root, and downloaded with an HTTP GET request for the URL
/tor/rendezvous3/<z> where z is a base-64 encoding of the hidden
service's blinded public key.
[TODO: raw base64 is not super-nice for URLs, since it can have
slashes. We already use it for microdescriptor URLs, though. Do we
care here?]
These requests must be made anonymously, on circuits not used for
anything else.
2.3. Publishing shared random values [PUB-SHAREDRANDOM]
Our design for limiting the predictability of HSDir upload locations
relies on a shared random value that isn't predictable in advance or
too influenceable by an attacker. The authorities must run a protocol
to generate such a value at least once per hsdir period. Here we
describe how they publish these values; the procedure they use to
generate them can change independently of the rest of this
specification. For one possible (somewhat broken) protocol, see
Appendix [SHAREDRANDOM].
We add a new line in votes and consensus documents:
"hsdir-shared-random" PERIOD-START VALUE
PERIOD-START = YYYY-MM-DD HH:MM:SS
VALUE = A base-64 encoded 256-bit value.
To decide which hsdir-shared-random line to include in a consensus
for a given PERIOD-START, we choose whichever line appears verbatim
in the most votes, so long as it is listed by at least three
authorities. Ties are broken in favor of the lower value. More than
one PERIOD-START is allowed per vote, and per consensus. The same
PERIOD-START must not appear twice in a vote or in a consensus.
[TODO: Need to define a more robust algorithm. Need to cover cases
where multiple cluster of authorities publish a different value,
etc.]
The hs-dir-shared-random lines appear, sorted by PERIOD-START, in the
consensus immediately after the "params" line.
The authorities should publish the shared random value for the
current period, and, at a time at least three voting periods before
the overlap interval begins, the shared random value for the next
period.
[TODO: find out what weasel doesn't like here.]
2.4. Hidden service descriptors: outer wrapper [DESC-OUTER]
The format for a hidden service descriptor is as follows, using the
meta-format from dir-spec.txt.
"hs-descriptor" SP "3" SP public-key SP certification NL
[At start, exactly once.]
public-key is the blinded public key for the service, encoded in
base 64. Certification is a certification of a short-term ed25519
descriptor signing key using the public key, in the format of
proposal 220.
"time-period" SP YYYY-MM-DD HH:MM:SS NUM NL
[Exactly once.]
The time period for which this descriptor is relevant, including
its starting time and its period number.
"revision-counter" SP Integer NL
[Exactly once.]
The revision number of the descriptor. If an HSDir receives a
second descriptor for a key that it already has a descriptor for,
it should retain and serve the descriptor with the higher
revision-counter.
(Checking for monotonically increasing revision-counter values
prevents an attacker from replacing a newer descriptor signed by
a given key with a copy of an older version.)
"encrypted" NL encrypted-string
[Exactly once.]
An encrypted blob, whose format is discussed in [ENCRYPTED-DATA]
below. The blob is base-64 encoded and enclosed in -----BEGIN
MESSAGE---- and ----END MESSAGE---- wrappers.
"signature" SP signature NL
[exactly once, at end.]
A signature of all previous fields, using the signing key in the
hs-descriptor line. We use a separate key for signing, so that
the hidden service host does not need to have its private blinded
key online.
2.5. Hidden service descriptors: encryption format [ENCRYPTED-DATA]
The encrypted part of the hidden service descriptor is encrypted and
authenticated with symmetric keys generated as follows:
salt = 16 random bytes
secret_input = nonce | blinded_public_key | subcredential |
INT_4(revision_counter)
keys = KDF(secret_input, salt, "hsdir-encrypted-data",
S_KEY_LEN + S_IV_LEN + MAC_KEY_LEN)
SECRET_KEY = first S_KEY_LEN bytes of keys
SECRET_IV = next S_IV_LEN bytes of keys
MAC_KEY = last MAC_KEY_LEN bytes of keys
The encrypted data has the format:
SALT (random bytes from above) [16 bytes]
ENCRYPTED The plaintext encrypted with S [variable]
MAC MAC of both above fields [32 bytes]
The encryption format is ENCRYPTED =
STREAM(SECRET_IV,SECRET_KEY) xor Plaintext
Before encryption, the plaintext must be padded to a multiple of ???
bytes with NUL bytes. The plaintext must not be longer than ???
bytes. [TODO: how much? Should this be a parameter? What values in
practice is needed to hide how many intro points we have, and how
many might be legacy ones?]
The plaintext format is:
"create2-formats" SP formats NL
[Exactly once]
A space-separated list of integers denoting CREATE2 cell format
numbers that the server recognizes. Must include at least TAP and
ntor as described in tor-spec.txt. See tor-spec section 5.1 for a
list of recognized handshake types.
"authentication-required" SP types NL
[At most once]
A space-separated list of authentication types. A client that does
not support at least one of these authentication types will not be
able to contact the host. Recognized types are: 'password' and
'ed25519'. See [INTRO-AUTH] below.
At least once:
"introduction-point" SP link-specifiers NL
[Exactly once per introduction point at start of introduction
point section]
The link-specifiers is a base64 encoding of a link specifier
block in the format described in BUILDING-BLOCKS.
"auth-key" SP "ed25519" SP key SP certification NL
[Exactly once per introduction point]
Base-64 encoded introduction point authentication key that was
used to establish introduction point circuit, cross-certifying
the blinded public key key using the certification format of
proposal 220.
"enc-key" SP "ntor" SP key NL
[At most once per introduction point]
Base64-encoded curve25519 key used to encrypt request to
hidden service.
[TODO: I'd like to have a cross-certification here too.]
"enc-key" SP "legacy" NL key NL
[At most once per introduction point]
Base64-encoded RSA key, wrapped in "----BEGIN RSA PUBLIC
KEY-----" armor, for use with a legacy introduction point as
described in [LEGACY_EST_INTRO] and [LEGACY-INTRODUCE1] below.
Exactly one of the "enc-key ntor" and "enc-key legacy"
elements must be present for each introduction point.
[TODO: I'd like to have a cross-certification here too.]
Other encryption and authentication key formats are allowed; clients
should ignore ones they do not recognize.
3. The introduction protocol
The introduction protocol proceeds in three steps.
First, a hidden service host builds an anonymous circuit to a Tor
node and registers that circuit as an introduction point.
[Between these steps, the hidden service publishes its
introduction points and associated keys, and the client fetches
them as described in section [HSDIR] above.]
Second, a client builds an anonymous circuit to the introduction
point, and sends an introduction request.
Third, the introduction point relays the introduction request along
the introduction circuit to the hidden service host, and acknowledges
the introduction request to the client.
3.1. Registering an introduction point [REG_INTRO_POINT]
3.1.1. Extensible ESTABLISH_INTRO protocol. [EST_INTRO]
When a hidden service is establishing a new introduction point, it
sends a ESTABLISH_INTRO cell with the following contents:
AUTH_KEY_TYPE [1 byte]
AUTH_KEY_LEN [1 byte]
AUTH_KEY [AUTH_KEY_LEN bytes]
Any number of times:
EXT_FIELD_TYPE [1 byte]
EXT_FIELD_LEN [1 byte]
EXT_FIELD [EXTRA_FIELD_LEN bytes]
ZERO [1 byte]
HANDSHAKE_AUTH [MAC_LEN bytes]
SIGLEN [1 byte]
SIG [SIGLEN bytes]
The AUTH_KEY_TYPE field indicates the type of the introduction point
authentication key and the type of the MAC to use in for
HANDSHAKE_AUTH. Recognized types are:
[00, 01] -- Reserved for legacy introduction cells; see
[LEGACY_EST_INTRO below]
[02] -- Ed25519; HMAC-SHA256.
[FF] -- Reserved for maintenance messages on existing
circuits; see MAINT_INTRO below.
[TODO: Should this just be a new relay cell type?
Matthew and George think so.]
The AUTH_KEY_LEN field determines the length of the AUTH_KEY
field. The AUTH_KEY field contains the public introduction point
authentication key.
The EXT_FIELD_TYPE, EXT_FIELD_LEN, EXT_FIELD entries are reserved for
future extensions to the introduction protocol. Extensions with
unrecognized EXT_FIELD_TYPE values must be ignored.
The ZERO field contains the byte zero; it marks the end of the
extension fields.
The HANDSHAKE_AUTH field contains the MAC of all earlier fields in
the cell using as its key the shared per-circuit material ("KH")
generated during the circuit extension protocol; see tor-spec.txt
section 5.2, "Setting circuit keys". It prevents replays of
ESTABLISH_INTRO cells.
SIGLEN is the length of the signature.
SIG is a signature, using AUTH_KEY, of all contents of the cell, up
to but not including SIG. These contents are prefixed with the string
"Tor establish-intro cell v1".
Upon receiving an ESTABLISH_INTRO cell, a Tor node first decodes the
key and the signature, and checks the signature. The node must reject
the ESTABLISH_INTRO cell and destroy the circuit in these cases:
* If the key type is unrecognized
* If the key is ill-formatted
* If the signature is incorrect
* If the HANDSHAKE_AUTH value is incorrect
* If the circuit is already a rendezvous circuit.
* If the circuit is already an introduction circuit.
[TODO: some scalability designs fail there.]
* If the key is already in use by another circuit.
Otherwise, the node must associate the key with the circuit, for use
later in INTRODUCE1 cells.
[TODO: The above will work fine with what we do today, but it will do
quite badly if we ever freak out and want to go back to RSA2048 or
bigger. Do we care?]
3.1.2. Registering an introduction point on a legacy Tor node [LEGACY_EST_INTRO]
Tor nodes should also support an older version of the ESTABLISH_INTRO
cell, first documented in rend-spec.txt. New hidden service hosts
must use this format when establishing introduction points at older
Tor nodes that do not support the format above in [EST_INTRO].
In this older protocol, an ESTABLISH_INTRO cell contains:
KEY_LENGTH [2 bytes]
KEY [KEY_LENGTH bytes]
HANDSHAKE_AUTH [20 bytes]
SIG [variable, up to end of relay payload]
The KEY_LENGTH variable determines the length of the KEY field.
The KEY field is a ASN1-encoded RSA public key.
The HANDSHAKE_AUTH field contains the SHA1 digest of (KH |
"INTRODUCE").
The SIG field contains an RSA signature, using PKCS1 padding, of all
earlier fields.
Note that since the relay payload itself may be no more than 498
bytes long, the KEY_LENGTH field can never have a first byte other
than [00] or [01]. These values are used to distinguish legacy
ESTABLISH_INTRO cells from newer ones.
Older versions of Tor always use a 1024-bit RSA key for these
introduction authentication keys.
Newer hidden services MAY use RSA keys up 1904 bits. Any more than
that will not fit in a RELAY cell payload.
3.1.3. Managing introduction circuits [MAINT_INTRO]
If the first byte of an ESTABLISH_INTRO cell is [FF], the cell's body
contains an administrative command for the circuit. The format of
such a command is:
Any number of times:
SUBCOMMAND_TYPE [2 bytes]
SUBCOMMAND_LEN [2 bytes]
SUBCOMMAND [COMMAND_LEN bytes]
Recognized SUBCOMMAND_TYPE values are:
[00 01] -- update encryption keys
[TODO: Matthew says, "This can be used to fork an intro point to
balance traffic over multiple hidden service servers while
maintaining the criteria for a valid ESTABLISH_INTRO
cell. -MF". Investigate.]
Unrecognized SUBCOMMAND_TYPE values should be ignored.
3.1.3.1. Updating encryption keys (subcommand 0001) [UPDATE-KEYS-SUBCMD]
Hidden service hosts send this subcommand to set their initial
encryption keys or update the configured public encryption keys
associated with this circuit. This message must be sent after
establishing an introduction point, before the circuit can be
advertised. These keys are given in the form:
NUMKEYS [1 byte]
NUMKEYS times:
KEYTYPE [1 byte]
KEYLEN [1 byte]
KEY [KEYLEN bytes]
COUNTER [4 bytes]
SIGLEN [1 byte]
SIGNATURE [SIGLEN bytes.]
The KEYTYPE value [01] is for Curve25519 keys.
The COUNTER field is a monotonically increasing value across a given
introduction point authentication key.
The SIGNATURE must be generated with the introduction point
authentication key, and must cover the entire subcommand body,
prefixed with the string "Tor hidden service introduction encryption
keys v1".
[TODO: Nothing is done here to prove ownership of the encryption
keys. Does that matter?]
[TODO: The point here is to allow encryption keys to change while
maintaining an introduction point and not forcing a client to
download a new descriptor. I'm not sure if that's worth it. It makes
clients who have seen a key before distinguishable from ones who have
not.]
[Matthew says: "Repeat-client over long periods of time will always
be distinguishable. It may be better to simply expire intro points
than try to preserve forward-secrecy, though". Must find out what he
meant.]
Setting the encryption keys for a given circuit replaces the previous
keys for that circuit. Clients who attempt to connect using the old
key receive an INTRO_ACK cell with error code [00 02] as described in
section [INTRO_ACK] below.
3.1.4. Acknowledging establishment of introduction point [INTRO_ESTABLISHED]
After setting up an introduction circuit, the introduction point
reports its status back to the hidden service host with an empty
INTRO_ESTABLISHED cell.
[TODO: make this cell type extensible. It should be able to include
data if that turns out to be needed.]
3.2. Sending an INTRODUCE1 cell to the introduction point. [SEND_INTRO1]
In order to participate in the introduction protocol, a client must
know the following:
* An introduction point for a service.
* The introduction authentication key for that introduction point.
* The introduction encryption key for that introduction point.
The client sends an INTRODUCE1 cell to the introduction point,
containing an identifier for the service, an identifier for the
encryption key that the client intends to use, and an opaque blob to
be relayed to the hidden service host.
In reply, the introduction point sends an INTRODUCE_ACK cell back to
the client, either informing it that its request has been delivered,
or that its request will not succeed.
3.2.1. INTRODUCE1 cell format [FMT_INTRO1]
An INTRODUCE1 cell has the following contents:
AUTH_KEYID [32 bytes]
ENC_KEYID [8 bytes]
Any number of times:
EXT_FIELD_TYPE [1 byte]
EXT_FIELD_LEN [1 byte]
EXT_FIELD [EXTRA_FIELD_LEN bytes]
ZERO [1 byte]
ENCRYPTED [Up to end of relay payload]
[TODO: Should we have a field to determine the type of ENCRYPTED, or
should we instead assume that there is exactly one encryption key per
encryption method? The latter is probably safer.]
Upon receiving an INTRODUCE1 cell, the introduction point checks
whether AUTH_KEYID and ENC_KEYID match a configured introduction
point authentication key and introduction point encryption key. If
they do, the cell is relayed; if not, it is not.
The AUTH_KEYID for an Ed25519 public key is the public key itself.
The ENC_KEYID for a Curve25519 public key is the first 8 bytes of the
public key. (This key ID is safe to truncate, since all the keys are
generated by the hidden service host, and the ID is only valid
relative to a single AUTH_KEYID.) The ENCRYPTED field is as
described in 3.3 below.
To relay an INTRODUCE1 cell, the introduction point sends an
INTRODUCE2 cell with exactly the same contents.
3.2.2. INTRODUCE_ACK cell format. [INTRO_ACK]
An INTRODUCE_ACK cell has the following fields:
STATUS [2 bytes]
Any number of times:
EXT_FIELD_TYPE [1 byte]
EXT_FIELD_LEN [1 byte]
EXT_FIELD [EXTRA_FIELD_LEN bytes]
Recognized status values are:
[00 00] -- Success: cell relayed to hidden service host.
[00 01] -- Failure: service ID not recognzied
[00 02] -- Failure: key ID not recognized
[00 03] -- Bad message format
Recognized extension field types:
[00 01] -- signed set of encryption keys
The extension field type 0001 is a signed set of encryption keys; its
body matches the body of the key update command in
[UPDATE-KEYS-CMD]. Whenever sending status [00 02], the introduction
point MUST send this extension field.
3.2.3. Legacy formats [LEGACY-INTRODUCE1]
When the ESTABLISH_INTRO cell format of [LEGACY_EST_INTRO] is used,
INTRODUCE1 cells are of the form:
AUTH_KEYID_HASH [20 bytes]
ENC_KEYID [8 bytes]
Any number of times:
EXT_FIELD_TYPE [1 byte]
EXT_FIELD_LEN [1 byte]
EXT_FIELD [EXTRA_FIELD_LEN bytes]
ZERO [1 byte]
ENCRYPTED [Up to end of relay payload]
Here, AUTH_KEYID_HASH is the hash of the introduction point
authentication key used to establish the introduction.
Because of limitations in older versions of Tor, the relay payload
size for these INTRODUCE1 cells must always be at least 246 bytes, or
they will be rejected as invalid.
3.3. Processing an INTRODUCE2 cell at the hidden service. [PROCESS_INTRO2]
Upon receiving an INTRODUCE2 cell, the hidden service host checks
whether the AUTH_KEYID/AUTH_KEYID_HASH field and the ENC_KEYID fields
are as expected, and match the configured authentication and
encryption key(s) on that circuit.
The service host then checks whether it has received a cell with
these contents before. If it has, it silently drops it as a
replay. (It must maintain a replay cache for as long as it accepts
cells with the same encryption key.)
If the cell is not a replay, it decrypts the ENCRYPTED field,
establishes a shared key with the client, and authenticates the whole
contents of the cell as having been unmodified since they left the
client. There may be multiple ways of decrypting the ENCRYTPED field,
depending on the chosen type of the encryption key. Requirements for
an introduction handshake protocol are described in
[INTRO-HANDSHAKE-REQS]. We specify one below in section
[NTOR-WITH-EXTRA-DATA].
The decrypted plaintext must have the form:
REND_TOKEN [20 bytes]
Any number of times:
EXT_FIELD_TYPE [1 byte]
EXT_FIELD_LEN [1 byte]
EXT_FIELD [EXTRA_FIELD_LEN bytes]
ZERO [1 byte]
ONION_KEY_TYPE [2 bytes]
ONION_KEY [depends on ONION_KEY_TYPE]
NSPEC (Number of link specifiers) [1 byte]
NSPEC times:
LSTYPE (Link specifier type) [1 byte]
LSLEN (Link specifier length) [1 byte]
LSPEC (Link specifier) [LSLEN bytes]
PAD (optional padding) [up to end of plaintext]
Upon processing this plaintext, the hidden service makes sure that
any required authentication is present in the extension fields, and
then extends a rendezvous circuit to the node described in the LSPEC
fields, using the ONION_KEY to complete the extension. As mentioned
in [BUILDING-BLOCKS], the "TLS-over-TCP, IPv4" and "Legacy node
identity" specifiers must be present.
The hidden service SHOULD NOT reject any LSTYPE fields which it
doesn't recognize; instead, it should use them verbatim in its EXTEND
request to the rendezvous point.
The ONION_KEY_TYPE field is one of:
[01] TAP-RSA-1024: ONION_KEY is 128 bytes long.
[02] NTOR: ONION_KEY is 32 bytes long.
The ONION_KEY field describes the onion key that must be used when
extending to the rendezvous point. It must be of a type listed as
supported in the hidden service descriptor.
Upon receiving a well-formed INTRODUCE2 cell, the hidden service host
will have:
* The information needed to connect to the client's chosen
rendezvous point.
* The second half of a handshake to authenticate and establish a
shared key with the hidden service client.
* A set of shared keys to use for end-to-end encryption.
3.3.1. Introduction handshake encryption requirements [INTRO-HANDSHAKE-REQS]
When decoding the encrypted information in an INTRODUCE2 cell, a
hidden service host must be able to:
* Decrypt additional information included in the INTRODUCE2 cell,
to include the rendezvous token and the information needed to
extend to the rendezvous point.
* Establish a set of shared keys for use with the client.
* Authenticate that the cell has not been modified since the client
generated it.
Note that the old TAP-derived protocol of the previous hidden service
design achieved the first two requirements, but not the third.
3.3.2. Example encryption handshake: ntor with extra data [NTOR-WITH-EXTRA-DATA]
This is a variant of the ntor handshake (see tor-spec.txt, section
5.1.4; see proposal 216; and see "Anonymity and one-way
authentication in key-exchange protocols" by Goldberg, Stebila, and
Ustaoglu).
It behaves the same as the ntor handshake, except that, in addition
to negotiating forward secure keys, it also provides a means for
encrypting non-forward-secure data to the server (in this case, to
the hidden service host) as part of the handshake.
Notation here is as in section 5.1.4 of tor-spec.txt, which defines
the ntor handshake.
The PROTOID for this variant is
"hidden-service-ntor-curve25519-sha256-1". Define the tweak value
t_hsenc, and the tag value m_hsexpand as:
t_hsenc = PROTOID | ":hs_key_extract"
m_hsexpand = PROTOID | ":hs_key_expand"
To make an INTRODUCE cell, the client must know a public encryption
key B for the hidden service on this introduction circuit. The client
generates a single-use keypair:
x,X = KEYGEN()
and computes:
secret_hs_input = EXP(B,x) | AUTH_KEYID | X | B | PROTOID
info = m_hsexpand | subcredential
hs_keys = HKDF(secret_hs_input, t_hsenc, info,
S_KEY_LEN+MAC_LEN)
ENC_KEY = hs_keys[0:S_KEY_LEN]
MAC_KEY = hs_keys[S_KEY_LEN:S_KEY_LEN+MAC_KEY_LEN]
and sends, as the ENCRYPTED part of the INTRODUCE1 cell:
CLIENT_PK [G_LENGTH bytes]
ENCRYPTED_DATA [Padded to length of plaintext]
MAC [MAC_LEN bytes]
Substituting those fields into the INTRODUCE1 cell body format
described in [FMT_INTRO1] above, we have
AUTH_KEYID [32 bytes]
ENC_KEYID [8 bytes]
Any number of times:
EXT_FIELD_TYPE [1 byte]
EXT_FIELD_LEN [1 byte]
EXT_FIELD [EXTRA_FIELD_LEN bytes]
ZERO [1 byte]
ENCRYPTED:
CLIENT_PK [G_LENGTH bytes]
ENCRYPTED_DATA [Padded to length of plaintext]
MAC [MAC_LEN bytes]
(This format is as documented in [FMT_INTRO1] above, except that here
we describe how to build the ENCRYPTED portion. If the introduction
point is running an older Tor that does not support this protocol,
the first field is replaced by a 20-byte AUTH_KEYID_HASH field as
described in [LEGACY-INTRODUCE1].)
Here, the encryption key plays the role of B in the regular ntor
handshake, and the AUTH_KEYID field plays the role of the node ID.
The CLIENT_PK field is the public key X. The ENCRYPTED_DATA field is
the message plaintext, encrypted with the symmetric key ENC_KEY. The
MAC field is a MAC of all of the cell from the AUTH_KEYID through the
end of ENCRYPTED_DATA, using the MAC_KEY value as its key.
To process this format, the hidden service checks PK_VALID(CLIENT_PK)
as necessary, and then computes ENC_KEY and MAC_KEY as the client did
above, except using EXP(CLIENT_PK,b) in the calculation of
secret_hs_input. The service host then checks whether the MAC is
correct. If it is invalid, it drops the cell. Otherwise, it computes
the plaintext by decrypting ENCRYPTED_DATA.
The hidden service host now completes the service side of the
extended ntor handshake, as described in tor-spec.txt section 5.1.4,
with the modified PROTOID as given above. To be explicit, the hidden
service host generates a keypair of y,Y = KEYGEN(), and uses its
introduction point encryption key 'b' to computes:
xb = EXP(X,b)
secret_hs_input = xb | AUTH_KEYID | X | B | PROTOID
info = m_hsexpand | subcredential
hs_keys = HKDF(secret_hs_input, t_hsenc, info,
S_KEY_LEN+MAC_LEN)
HS_DEC_KEY = hs_keys[0:S_KEY_LEN]
HS_MAC_KEY = hs_keys[S_KEY_LEN:S_KEY_LEN+MAC_KEY_LEN]
(The above are used to check the MAC and then decrypt the
encrypted data.)
ntor_secret_input = EXP(X,y) | xb | ID | B | X | Y | PROTOID
NTOR_KEY_SEED = H(secret_input, t_key)
verify = H(secret_input, t_verify)
auth_input = verify | ID | B | Y | X | PROTOID | "Server"
(The above are used to finish the ntor handshake.)
The server's handshake reply is:
SERVER_PK Y [G_LENGTH bytes]
AUTH H(auth_input, t_mac) [H_LENGTH bytes]
These faileds can be send to the client in a RENDEZVOUS1 cell.
(See [JOIN_REND] below.)
The hidden service host now also knows the keys generated by the
handshake, which it will use to encrypt and authenticate data
end-to-end between the client and the server. These keys are as
computed in tor-spec.txt section 5.1.4.
3.4. Authentication during the introduction phase. [INTRO-AUTH]
Hidden services may restrict access only to authorized users. One
mechanism to do so is the credential mechanism, where only users who
know the credential for a hidden service may connect at all. For more
fine-grained conntrol, a hidden service can be configured with
password-based or public-key-based authentication.
3.4.1. Password-based authentication.
To authenticate with a password, the user must include an extension
field in the encrypted part of the INTRODUCE cell with an
EXT_FIELD_TYPE type of [01] and the contents:
Username [00] Password.
The username may not include any [00] bytes. The password may.
On the server side, the password MUST be stored hashed and salted,
ideally with scrypt or something better.
3.4.2. Ed25519-based authentication.
To authenticate with an Ed25519 private key, the user must include an
extension field in the encrypted part of the INTRODUCE cell with an
EXT_FIELD_TYPE type of [02] and the contents:
Nonce [16 bytes]
Pubkey [32 bytes]
Signature [64 bytes]
Nonce is a random value. Pubkey is the public key that will be used
to authenticate. [TODO: should this be an identifier for the public
key instead?] Signature is the signature, using Ed25519, of:
"Hidserv-userauth-ed25519"
Nonce (same as above)
Pubkey (same as above)
AUTH_KEYID (As in the INTRODUCE1 cell)
ENC_KEYID (As in the INTRODUCE1 cell)
The hidden service host checks this by seeing whether it recognizes
and would accept a signature from the provided public key. If it
would, then it checks whether the signature is correct. If it is,
then the correct user has authenticated.
Replay prevention on the whole cell is sufficient to prevent replays
on the authentication.
Users SHOULD NOT use the same public key with multiple hidden
services.
4. The rendezvous protocol
Before connecting to a hidden service, the client first builds a
circuit to an arbitrarily chosen Tor node (known as the rendezvous
point), and sends an ESTABLISH_RENDEZVOUS cell. The hidden service
later connects to the same node and sends a RENDEZVOUS cell. Once
this has occurred, the relay forwards the contents of the RENDEZVOUS
cell to the client, and joins the two circuits together.
4.1. Establishing a rendezvous point [EST_REND_POINT]
The client sends the rendezvous point a
RELAY_COMMAND_ESTABLISH_RENDEZVOUS cell containing a 20-byte value.
RENDEZVOUS_COOKIE [20 bytes]
Rendezvous points MUST ignore any extra bytes in an
ESTABLISH_RENDEZVOUS message. (Older versions of Tor did not.)
The rendezvous cookie is an arbitrary 20-byte value, chosen randomly
by the client. The client SHOULD choose a new rendezvous cookie for
each new connection attempt. If the rendezvous cookie is already in
use on an existing circuit, the rendezvous point should reject it and
destroy the circuit.
Upon receiving a ESTABLISH_RENDEZVOUS cell, the rendezvous point
associates the cookie with the circuit on which it was sent. It
replies to the client with an empty RENDEZVOUS_ESTABLISHED cell to
indicate success. [TODO: make this extensible]
The client MUST NOT use the circuit which sent the cell for any
purpose other than rendezvous with the given location-hidden service.
The client should establish a rendezvous point BEFORE trying to
connect to a hidden service.
4.2. Joining to a rendezvous point [JOIN_REND]
To complete a rendezvous, the hidden service host builds a circuit to
the rendezvous point and sends a RENDEZVOUS1 cell containing:
RENDEZVOUS_COOKIE [20 bytes]
HANDSHAKE_INFO [variable; depends on handshake type
used.]
If the cookie matches the rendezvous cookie set on any
not-yet-connected circuit on the rendezvous point, the rendezvous
point connects the two circuits, and sends a RENDEZVOUS2 cell to the
client containing the contents of the RENDEZVOUS1 cell.
Upon receiving the RENDEZVOUS2 cell, the client verifies that the
HANDSHAKE_INFO correctly completes a handshake, and uses the
handshake output to derive shared keys for use on the circuit.
[TODO: Should we encrypt HANDSHAKE_INFO as we did INTRODUCE2
contents? It's not necessary, but it could be wise. Similarly, we
should make it extensible.]
4.3. Using legacy hosts as rendezvous points
The behavior of ESTABLISH_RENDEZVOUS is unchanged from older versions
of this protocol, except that relays should now ignore unexpected
bytes at the end.
Old versions of Tor required that RENDEZVOUS cell payloads be exactly
168 bytes long. All shorter rendezvous payloads should be padded to
this length with [00] bytes.
5. Encrypting data between client and host
A successfully completed handshake, as embedded in the
INTRODUCE/RENDEZVOUS cells, gives the client and hidden service host
a shared set of keys Kf, Kb, Df, Db, which they use for sending
end-to-end traffic encryption and authentication as in the regular
Tor relay encryption protocol, applying encryption with these keys
before other encryption, and decrypting with these keys before other
encryption. The client encrypts with Kf and decrypts with Kb; the
service host does the opposite.
6. Open Questions:
Scaling hidden services is hard. There are on-going discussions that
you might be able to help with. See [SCALING-REFS].
How can we improve the HSDir unpredictability design proposed in
[SHAREDRANDOM]? See [SHAREDRANDOM-REFS] for discussion.
How can hidden service addresses become memorable while retaining
their self-authenticating and decentralized nature? See
[HUMANE-HSADDRESSES-REFS] for some proposals; many more are possible.
Hidden Services are pretty slow. Both because of the lengthy setup
procedure and because the final circuit has 6 hops. How can we make
the Hidden Service protocol faster? See [PERFORMANCE-REFS] for some
suggestions.
References:
[KEYBLIND-REFS]:
https://trac.torproject.org/projects/tor/ticket/8106https://lists.torproject.org/pipermail/tor-dev/2012-September/004026.html
[SHAREDRANDOM-REFS]:
https://trac.torproject.org/projects/tor/ticket/8244https://lists.torproject.org/pipermail/tor-dev/2013-November/005847.htmlhttps://lists.torproject.org/pipermail/tor-talk/2013-November/031230.html
[SCALING-REFS]:
https://lists.torproject.org/pipermail/tor-dev/2013-October/005556.html
[HUMANE-HSADDRESSES-REFS]:
https://gitweb.torproject.org/torspec.git/blob/HEAD:/proposals/ideas/xxx-on…http://archives.seul.org/or/dev/Dec-2011/msg00034.html
[PERFORMANCE-REFS]:
"Improving Efficiency and Simplicity of Tor circuit
establishment and hidden services" by Overlier, L., and
P. Syverson
[TODO: Need more here! Do we have any? :( ]
[ATTACK-REFS]:
"Trawling for Tor Hidden Services: Detection, Measurement,
Deanonymization" by Alex Biryukov, Ivan Pustogarov,
Ralf-Philipp Weinmann
"Locating Hidden Servers" by Lasse Øverlier and Paul
Syverson
[ED25519-REFS]:
"High-speed high-security signatures" by Daniel
J. Bernstein, Niels Duif, Tanja Lange, Peter Schwabe, and
Bo-Yin Yang. http://cr.yp.to/papers.html#ed25519
Appendix A. Signature scheme with key blinding [KEYBLIND]
As described in [IMD:DIST] and [SUBCRED] above, we require a "key
blinding" system that works (roughly) as follows:
There is a master keypair (sk, pk).
Given the keypair and a nonce n, there is a derivation function
that gives a new blinded keypair (sk_n, pk_n). This keypair can
be used for signing.
Given only the public key and the nonce, there is a function
that gives pk_n.
Without knowing pk, it is not possible to derive pk_n; without
knowing sk, it is not possible to derive sk_n.
It's possible to check that a signature make with sk_n while
knowing only pk_n.
Someone who sees a large number of blinded public keys and
signatures made using those public keys can't tell which
signatures and which blinded keys were derived from the same
master keypair.
You can't forge signatures.
[TODO: Insert a more rigorous definition and better references.]
We propose the following scheme for key blinding, based on Ed25519.
(This is an ECC group, so remember that scalar multiplication is the
trapdoor function, and it's defined in terms of iterated point
addition. See the Ed25519 paper [Reference ED25519-REFS] for a fairly
clear writeup.)
Let the basepoint be written as B. Assume B has prime order l, so
lB=0. Let a master keypair be written as (a,A), where a is the private
key and A is the public key (A=aB).
To derive the key for a nonce N and an optional secret s, compute the
blinding factor h as H(A | s, B, N), and let:
private key for the period: a' = h a
public key for the period: A' = h' A = (ha)B
Generating a signature of M: given a deterministic random-looking r
(see EdDSA paper), take R=rB, S=r+hash(R,A',M)ah mod l. Send signature
(R,S) and public key A'.
Verifying the signature: Check whether SB = R+hash(R,A',M)A'.
(If the signature is valid,
SB = (r + hash(R,A',M)ah)B
= rB + (hash(R,A',M)ah)B
= R + hash(R,A',M)A' )
See [KEYBLIND-REFS] for an extensive discussion on this scheme and
possible alternatives. I've transcribed this from a description by
Tanja Lange at the end of the thread. [TODO: We'll want a proof for
this.]
(To use this with Tor, set N = INT_8(period-number) | INT_8(Start of
period in seconds since epoch).)
Appendix B. Selecting nodes [PICKNODES]
Picking introduction points
Picking rendezvous points
Building paths
Reusing circuits
(TODO: This needs a writeup)
Appendix C. Recommendations for searching for vanity .onions [VANITY]
EDITORIAL NOTE: The author thinks that it's silly to brute-force the
keyspace for a key that, when base-32 encoded, spells out the name of
your website. It also feels a bit dangerous to me. If you train your
users to connect to
llamanymityx4fi3l6x2gyzmtmgxjyqyorj9qsb5r543izcwymle.onion
I worry that you're making it easier for somebody to trick them into
connecting to
llamanymityb4sqi0ta0tsw6uovyhwlezkcrmczeuzdvfauuemle.onion
Nevertheless, people are probably going to try to do this, so here's a
decent algorithm to use.
To search for a public key with some criterion X:
Generate a random (sk,pk) pair.
While pk does not satisfy X:
Add the number 1 to sk
Add the scalar B to pk
Return sk, pk.
This algorithm is safe [source: djb, personal communication] [TODO:
Make sure I understood correctly!] so long as only the final (sk,pk)
pair is used, and all previous values are discarded.
To parallelize this algorithm, start with an independent (sk,pk) pair
generated for each independent thread, and let each search proceed
independently.
Appendix D. Numeric values reserved in this document
[TODO: collect all the lists of commands and values mentioned above]
[View Less]
Sitting with David Goulet, it's problematic that when I search for
"torsocks" I get the old Google Code page instead of the up-to-date
page on http://gitweb.torproject.org
This is problematic because people report bugs to the Google Code
repository instead of the torproject repository.
Jake/Robert: You are the project owners on Google Code. Could you
please remove the project?
Much love,
-Virgil
> On Mon, Apr 7, 2014 at 11:34 AM, George Kadianakis > So, based on your
> response, IIUC, the idea is that because young
> > guards are underutilized, we want to increase the probability of them
> > being chosen in non-guard positions, so that they become more utilized
> > till more people pick them as guards?
>
> Right.
>
> > Some questions on the terminology you used:
> >
> > a) What do you mean by 'last rotation period'? When you say "…
[View More]for
> > fraction k of the last rotation period", you mean that if the
> > authorities read consensuses for the past 12 months, and the relay R
> > was up as a guard for 6 months, then k would be 6/12 == 0.5?
>
> The "rotation period" is the average length of time between choosing
> guards. For example, with the current parameters (IIRC), clients
> discard a guard and choose a new one at a time uniformly chosen
> between 30 and 60 days. So the rotation period is 45 days = 45*24
> hours = 1080 consensus votes, and essentially in each of those hours,
> 1/1080 of the clients choose a new guard. So the last rotation
> period's worth of status documents should be the only ones that matter
> for figuring out how heavily a node is used as a guard. For example,
> if a relay has had the guard flag for 15 days (=360 consensus
> documents) out of the last 45, then that relay has about k=360/1080 of
> the number of clients using it as a guard as it would have, if it had
> been a guard for the full 1080 hours.
>
Ah, I see. I originally misinterpreted what you meant by 'last
rotation period'.
> (N.B. I'm making an assumption that guard rotations will be uniformly
> distributed over the period. This should be safe in equilibrium, but
> might get kind of shaky when the period first increases to 9 months or
> whatever the final decision is)
>
Yep.
We also don't consider new releases of TBB/Tails (many users pick new
entry guards), or sudden bulk arrival of users (botnets, real life
events, etc.).
> > b) By (weight with the guard flag) you mean the result of:
> > <consensus BW> * <consensus weight> ?
> >
> > Is that right, or did I misunderstand you?
>
> Yes.
>
> > Assuming the above terminology assumptions, I began trying to
> > understand your formula. First of all, I was wondering how you ended
> > up with it? Is this some standard form? I'm not very familiar with
> > these things.
>
> Well, the way the weights work is that they compute what fraction of
> guard bandwidth should be used for middle and exit traffic, assuming
> that the rest will be all used up by client-guard connections. That
> is, the weights are telling us to use (1-(<consensus middle weight> +
> <consensus exit weight>)) fraction of the guard's bandwidth for these
> connections.
>
> But if a relay has only fraction k of its "guard bandwidth" used up
> this way, then (1-k) of this bandwidth should go back into the pool
> for other positions. I think that's what the formula does, but I
> could have it wrong. Does that make sense?
>
I see. That makes sense, I think.
I will ponder on this a bit more, and then edit the proposal.
(I would like to think a bit more about how accurate the assumption
"relay has been a guard for 0.25 of the rotation period =>
relay has 0.25 of its bandwidth occupied for guard purposes")
Thanks!
[View Less]
Hello all,
I just tagged the obfsclient v0.0.1 (Release)
Download at: https://github.com/Yawning/obfsclient/releases/tag/v0.0.1
Major changes since 0.0.1-rc2:
* Invalid padding lengths in obfs3/ScrambleSuit handshakes are now
correctly handled.
* Handshake timeout is now set to a random interval between 60 and 120
seconds.
* The ScrambleSuit probability distributions are now closer to what
obfsproxy generates.
* "--version" will now cause obfsclient to print the version and …
[View More]exit.
* Various code cleanups.
Some notes about ScrambleSuit:
* I reverted the "workaround" I had for bug #11100, so Session Ticket
handshakes to bridges running obfsproxy 0.2.6 will fail if they have
been running long enough to trigger the bug condition.
* Actual protocol behavior is closer to obfsproxy than my previous
releases, but still not identical to 0.2.7. the next obfsproxy
release will close the difference further.
Hopefully it is of some use to people.
Special thanks to Fabian Keil for help testing and fixing the various
release candidates.
Questions, comments, feedback appreciated as always,
--
Yawning Angel
[View Less]
I've revised proposal 220 based on commentary from Roger. The biggest
changes is tweaking all of the things called "certificates" to make
them actually follow the same format to greatest the extent possible.
To see diffs, you can use git, or browse the gitweb site at
https://gitweb.torproject.org/torspec.git/history/HEAD:/proposals/220-ecc-i…
Filename: 220-ecc-id-keys.txt
Title: Migrate server identity keys to Ed25519
Authors: Nick Mathewson
Created: 12 August 2013
Target: 0.2.x.x
Status: …
[View More]Draft
[Note: This is a draft proposal; I've probably made some important
mistakes, and there are parts that need more thinking. I'm
publishing it now so that we can do the thinking together.]
0. Introduction
In current Tor designs, router identity keys are limited to
1024-bit RSA keys.
Clearly, that should change, because RSA doesn't represent a good
performance-security tradeoff nowadays, and because 1024-bit RSA is
just plain too short.
We've already got an improved circuit extension handshake protocol
that uses curve25519 in place of RSA1024, and we're using (where
supported) P256 ECDHE in our TLS handshakes, but there are more uses
of RSA1024 to replace, including:
* Router identity keys
* TLS link keys
* Hidden service keys
This proposal describes how we'll migrate away from using 1024-bit
RSA in the first two, since they're tightly coupled. Hidden service
crypto changes will be complex, and will merit their own proposal.
In this proposal, we'll also (incidentally) be extirpating a number
of SHA1 usages.
1. Overview
When this proposal is implemented, every router will have an Ed25519
identity key in addition to its current RSA1024 public key.
Ed25519 (specifically, Ed25519-SHA-512 as described and specified at
http://ed25519.cr.yp.to/) is a desirable choice here: it's secure,
fast, has small keys and small signatures, is bulletproof in several
important ways, and supports fast batch verification. (It isn't quite
as fast as RSA1024 when it comes to public key operations, since RSA
gets to take advantage of small exponents when generating public
keys.)
(For reference: In Ed25519 public keys are 32 bytes long, private keys
are 64 bytes long, and signatures are 64 bytes long.)
To mirror the way that authority identity keys work, we'll fully
support keeping Ed25519 identity keys offline; they'll be used to
sign long-ish term signing keys, which in turn will do all of the
heavy lifting. A signing key will get used to sign the things that
RSA1024 identity keys currently sign.
1.1. 'Personalized' signatures
Each of the keys introduced here is used to sign more than one kind
of document. While these documents should be unambiguous, I'm going
to forward-proof the signatures by specifying each signature to be
generated, not on the document itself, but on the document prefixed
with some distinguishing string.
2. Certificates and Router descriptors.
2.1. Certificates
When generating a signing key, we also generate a certificate for it.
Unlike the certificates for authorities' signing keys, these
certificates need to be sent around frequently, in significant
numbers. So we'll choose a compact representation.
VERSION [1 Byte]
CERT_TYPE [1 Byte]
EXPIRATION_DATE [3 Bytes]
CERT_KEY_TYPE [1 byte]
CERTIFIED_KEY [32 Bytes]
EXTENSIONS [variable length, up to length of certificate
minus 64 bytes.]
SIGNATURE [64 Bytes]
The "VERSION" field holds the value [01]. The "CERT_TYPE" field
holds a value depending on the type of certificate. (See appendix
A.1.) The CERTIFIED_KEY field is an Ed25519 public key if
CERT_KEY_TYPE is [01], or a SHA256 hash of some other key type
depending on the value of CERT_KEY_TYPE. The EXPIRATION_DATE is a
date, given in HOURS since the epoch, after which this
certificate isn't valid. (A three-byte field here will work fine
until 5797 A.D.)
The EXTENSIONS field contains zero or more extensions, each of
the format:
ExtLength [1 or 2 bytes]
ExtType [1 or 2 bytes]
ExtData [Length bytes]
The ExtLength and ExtType fields can represent values between 0
and 2^15-1, representing values under 128 as "0xxxxxxx" and
values over 128 as "1xxxxxxx yyyyyyyy". The meaning of the
ExtData field in an extension is type-dependent.
It is an error for an extension to be truncated; such a
certificate is invalid.
Before processing any certificate, parties MUST know which
identity key it is supposed to be signed by, and then check the
signature. The signature is formed by signing the first N-64
bytes of the certificate prefixed with the string "Tor node
signing key certificate v1".
2.2. Basic extensions
2.2.1. Signed-with-ed25519-key extension [type 04]
In several places, it's desirable to bundle the key signing a
certificate along with the certificate. We do so with this
extension.
ExtLength = 32
ExtData =
An ed25519 key [32 bytes]
When this extension is present, it MUST match the key used to
sign the certificate.
This
2.3. Revoking keys.
We also specify a revocation document for revoking a signing key or an
identity key. Its format is:
FIXED_PREFIX [8 Bytes]
VERSION [1 Byte]
KEYTYPE [1 Byte]
IDENTITY_KEY [32 Bytes]
REVOKED_KEY [32 Bytes]
PUBLISHED [8 Bytes]
REV_EXTENSIONS [variable length, up to length of revocation
document minus 64 bytes]
SIGNATURE [64 Bytes]
FIXED_PREFIX is "REVOKEID" or "REVOKESK". VERSION is [01]. KEYTYPE is
[01] for revoking a signing key or [02] for revoking an identity key.
REVOKED_KEY is the key being revoked; IDENTITY_KEY is the node's
Ed25519 identity key. PUBLISHED is the time that the document was
generated, in seconds since the epoch. REV_EXTENSIONS is left for a
future version of this document. The SIGNATURE is generated with
the same key as in IDENTITY_KEY, and covers the entire revocation,
prefixed with "Tor key revocation v1".
Using these revocation documents is left for a later specification.
2.4. Managing keys
By default, we can keep the easy-to-setup key management properties
that Tor has now, so that node operators aren't required to have
offline public keys:
* When a Tor node starts up with no Ed25519 identity keys, it
generates a new identity keypair.
* When a Tor node has an Ed25519 identity keypair, and it has
no signing key, or its signing key is going to expire within
the next 48 hours, it generates a new signing key to last
30 days.
But we also support offline identity keys:
* When a Tor node starts with an Ed25519 public identity key
but no private identity key, it checks whether it has
a currently valid certified signing keypair. If it does,
it starts. Otherwise, it refuses to start.
* If a Tor node's signing key is going to expire soon, it starts
warning the user. If it is expired, then the node shuts down.
2.5. Router descriptors
We specify the following element that may appear at most once in
each router descriptor:
"identity-ed25519" SP certificate NL
The identity-key and certificate are base64-encoded with
terminating =s removed. When this element is present, it MUST appear
as the first or second element in the router descriptor.
[XXX The rationale here is to allow extracting the identity key and
signing key and checking the signature before fully parsing the rest
of the document. -NM]
The certificate has CERT_TYPE of [04]. It must include a
signed-with-ed25519-key extension (see section 2.2.1), so that we
can extract the identity key.
When an identity-ed25519 element is present, there must also be a
"router-signature-ed25519" element. It MUST be the next-to-last
element in the descriptor, appearing immediately before the RSA
signature. It MUST contain an ed25519 signature of the entire
document, from the first character up to but not including the
"router-signature-ed25519" element, prefixed with the string "Tor
router descriptor signature v1". Its format is:
"router-signature-ed25519" SP signature NL
Where 'signature' is encoded in base64 with terminating =s removed.
The signing key in the certificate MUST
be the one used to sign the document.
Note that these keys cross-certify as follows: the ed25519 identity
key signs the ed25519 signing key in the certificate. The ed25519
signing key signs itself and the ed25519 identity key and the RSA
identity key as part of signing the descriptor. And the RSA identity
key also signs all three keys as part of signing the descriptor.
2.5.1. Checking descriptor signatures.
Current versions of Tor will handle these new formats by ignoring the
new fields, and not checking any ed25519 information.
New versions of Tor will have a flag that tells them whether to check
ed25519 information. When it is set, they must check:
* All RSA information and signatures that Tor implementations
currently check.
* If the identity-ed25519 line is present, it must be well-formed,
and the certificate must be well-formed and correctly signed,
and there must be a valid router-signature-ed25519 signature.
* If we require an ed25519 key for this node (see 3.1 below), the
ed25519 key must be present.
Authorities and directory caches will have this flag always-on. For
clients, it will be controlled by a torrc option and consensus
option, to be set to "always-on" in the future once enough clients
support it.
2.5.2. Extra-info documents
Extra-info documents now include "identity-ed25519" and
"router-signature-ed25519" fields in the same positions in which they
appear in router descriptors.
Additionally, we add the base64-encoded, =-stripped SHA256 digest of
a node's extra-info document field to the extra-info-digest line in
the router descriptor. (All versions of Tor that recognize this line
allow an extra field there.)
2.5.3. A note on signature verification
Here and elsewhere, we're receiving a certificate and a document
signed with the key certified by that certificate in the same step.
This is a fine time to use the batch signature checking capability of
Ed25519, so that we can check both signatures at once without (much)
additional overhead over checking a single signature.
3. Consensus documents and authority operation
3.1. Handling router identity at the authority
When receiving router descriptors, authorities must track mappings
between RSA and Ed25519 keys.
Rule 1: Once an authority has seen an Ed25519 identity key and an RSA
identity key together on the same (valid) descriptor, it should no
longer accept any descriptor signed by that RSA key with a different
Ed25519 key, or that Ed25519 key with a different RSA key.
Rule 2: Once an authority has seen an Ed25519 identity key and an RSA
identity key on the same descriptor, it should no longer accept any
descriptor signed by that RSA key unless it also has that Ed25519
key.
These rules together should enforce the property that, even if an
attacker manages to steal or factor a node's RSA identity key, the
attacker can't impersonate that node to the authorities, even when
that node is identified by its RSA key.
Enforcement of Rule 1 should be advisory-only for a little while (a
release or two) while node operators get experience having Ed25519
keys, in case there are any bugs that cause or force identity key
replacement. Enforcement of Rule 2 should be advisory-only for
little while, so that node operators can try 0.2.5 but downgrade to
0.2.4 without being de-listed from the consensus.
[XXX I could specify a way to do a signed "I'm downgrading for a
while!" statement, and kludge some code back into 0.2.4.x to better
support that?]
3.2. Formats
Vote and microdescriptor documents now contain an optional "id"
field for each routerstatus section. Its format is:
"id" SP "ed25519" SP ed25519-identity NL
where ed25519-identity is base64-encoded, with trailing = characters
omitted. In vote documents, it may be replaced by the format:
"id" SP "ed25519" SP "none" NL
which indicates that the node does not have an ed25519 identity. (In
a microdescriptor, a lack of "id" line means that the node has no ed25519
identity.)
[XXXX Should the id entries in consensuses go into microdescriptors
instead? I think perhaps so. -NM]
A vote or consensus document is ill-formed if it includes the same
ed25519 identity key twice.
3.3. Generating votes
An authority should pick which descriptor to choose for a node as
before, and include the ed25519 identity key for the descriptor if
it's present.
As a transition, before Rule 1 and Rule 2 in 3.1 are fully enforced,
authorities need a way to deal with the possibility that there might
be two nodes with the same ed25519 key but different RSA keys. In
that case, it votes for the one with the most recent publication
date.
(The existing rules already prevent an authority from voting for two
servers with the same RSA identity key.)
3.4. Generating a consensus from votes
This proposal requires a new consensus vote method. When we deploy
it, we'll pick the next available vote method in sequence to use for
this.
When the new consensus method is in use, we must choose nodes first
by ECC key, then by RSA key.
First, for every {ECC identity key, RSA identity key} pair listed by
over half of the voting authorities, list it, unless some other RSA
identity key digest is listed more popularly for the ECC key. Break
ties in favor of low RSA digests. Treat all routerstatus entries that
mention this <ECC,RSA> pair as being for the same router, and all
routerstatus entries that mention the same RSA key with an
unspecified ECC key as being for the same router.
Then, for every node that has previously not been listed, perform the
current routerstatus algorithm: listing a node if it has been listed
by at least N/2 voting authorities, and treating all routerstatuses
containing the same identity as the same router.
In other words:
Let Entries = []
for each ECC ID listed by any voter:
Find the RSA key associated with that ECC ID by the most voters,
breaking ties in favor of low RSA keys.
If that ECC ID and RSA key ID are listed by > N/2 voting authorities:
Add the consensus of the routerstatus entries for those
voters, along with the routerstatus entry for every voter
that included that RSA key with no ECC key, to Entries.
Include the ECC ID in the consensus.
For each RSA key listed by any voter:
If that RSA key is already in Entries, skip it.
If the RSA key is listed by > N/2 voting authorities:
Add the consensus of the routerstatus entries for those
voters to Entries. Do not include an ECC key in the
consensus.
[XXX Think about this even more.]
4. The link protocol
4.1. Overview of the status quo
This section won't make much sense unless you grok the v3
link protocol as described in tor-spec.txt, first proposed in
proposal 195. So let's review.
In the v3 link protocol, the client completes a TLS handshake
with the server, in which the server uses an arbitrary
certificate signed with an RSA key. The client then sends a
VERSIONS cell. The server replies with a VERSIONS cell to
negotiate version 3 or higher. The server also sends a CERTS
cell and an AUTH_CHALLENGE cell and a NETINFO cell.
The CERTS cell from the server contains a set of one or more
certificates that authenticate the RSA key used in the TLS
handshake. (Right now there's one self-signed RSA identity key
certificate, and one certificate signing the RSA link key with
the identity key. These certificates are X509.)
Having received a CERTS cell, the client has enough information
to authenticate the server. At this point, the client may send a
NETINFO cell to finish the handshake. But if the client wants to
authenticate as well, it can send a CERTS cell and an AUTENTICATE
cell.
The client's CERTS cell also contains certs of the same general
kinds as the server's key file: a self-signed identity
certificate, and an authentication certificate signed with the
identity key. The AUTHENTICATE cell contains a signature of
various fields, including the contents of the AUTH_CHALLENGE
which the server sent cell, using the client's authentication
key. These cells allow the client to authenticate to the server.
4.2. Link protocol changes for ECC ID keys
We add four new CertType values for use in CERTS cells:
4: Ed25519 signing key
5: Link key certificate certified by Ed25519 signing key
6: Ed25519 TLS authentication key certified by Ed25519 signing key
7: RSA cross-certificate for Ed25519 identity key
These correspond to types used in the CERT_TYPE field of
the certificates.
The content of certificate type [04] (Ed25519 signing key)
is as in section 2.5 above, containing an identity key and the
signing key, both signed by the identity key.
Certificate type [05] (Link certificate signed with Ed25519
signing key) contains a SHA256 digest of the X.509 link
certificate used on the TLS connection in its key field; it is
signed with the signing key.
Certificate type [06] (Ed25519 TLS authentication signed with
Ed25519 signing key) has the signing key used to sign the
AUTHENTICATE cell described later in this section.
Certificate type [07] (Cross-certification of Ed25519 identity
with RSA key) contains the following data:
ED25519_KEY [32 bytes]
EXPIRATION_DATE [4 bytes]
SIGNATURE [128 bytes]
Here, the Ed25519 identity key is signed with router's RSA
identity key, to indicate that authenticating with a key
certified by the Ed25519 key counts as certifying with RSA
identity key. (The signature is computed on the SHA256 hash of
the non-signature parts of the certificate, prefixed with the
string "Tor TLS RSA/Ed25519 cross-certificate".)
(There's no reason to have a corresponding Ed25519-signed-RSA-key
certificate here, since we do not treat authenticating with an RSA
key as proving ownership of the Ed25519 identity.)
Relays with Ed25519 keys should always send these certificate types
in addition to their other certificate types.
Non-bridge relays with Ed25519 keys should generate TLS link keys of
appropriate strength, so that the certificate chain from the Ed25519
key to the link key is strong enough.
We add a new authentication type for AUTHENTICATE cells:
"Ed25519-TLSSecret", with AuthType value 2. Its format is the same as
"RSA-SHA256-TLSSecret", except that the CID and SID fields support
more key types; some strings are different, and the signature is
performed with Ed25519 using the authentication key from a type-6
cert. Clients can send this AUTHENTICATE type if the server
lists it in its AUTH_CHALLENGE cell.
Modified values and new fields below are marked with asterisks.
TYPE: The characters "AUTH0002"* [8 octets]
CID: A SHA256 hash of the initiator's RSA1024 identity key [32 octets]
SID: A SHA256 hash of the responder's RSA1024 identity key [32 octets]
*CID_ED: The initiator's Ed25519 identity key [32 octets]
*SID_ED: The responder's Ed25519 identity key, or all-zero. [32 octets]
SLOG: A SHA256 hash of all bytes sent from the responder to the
initiator as part of the negotiation up to and including the
AUTH_CHALLENGE cell; that is, the VERSIONS cell, the CERTS cell,
the AUTH_CHALLENGE cell, and any padding cells. [32 octets]
CLOG: A SHA256 hash of all bytes sent from the initiator to the
responder as part of the negotiation so far; that is, the
VERSIONS cell and the CERTS cell and any padding cells. [32
octets]
SCERT: A SHA256 hash of the responder's TLS link certificate. [32
octets]
TLSSECRETS: A SHA256 HMAC, using the TLS master secret as the
secret key, of the following:
- client_random, as sent in the TLS Client Hello
- server_random, as sent in the TLS Server Hello
- the NUL terminated ASCII string:
"Tor V3 handshake TLS cross-certification with Ed25519"*
[32 octets]
RAND: A 24 byte value, randomly chosen by the initiator. [24 octets]
*SIG: A signature of all previous fields using the initiator's
Ed25519 authentication flags.
[variable length]
If you've got a consensus that lists an ECC key for a node, but the
node doesn't give you an ECC key, then refuse this connection.
5. The extend protocol
We add a new NSPEC node specifier for use in EXTEND2 cells, with
LSTYPE value [03]. Its length must be 32 bytes; its content is the
Ed25519 identity key of the target node.
Clients should use this type only when:
* They know an Ed25519 identity key for the destination node.
* The source node supports EXTEND2 cells
* A torrc option is set, _or_ a consensus value is set.
We'll leave the consensus value off for a while until more clients
support this, and then turn it on.
When picking a channel for a circuit, if this NSPEC value is
provided, then the RSA identity *and* the Ed25519 identity must
match.
If we have a channel with a given Ed25519 ID and RSA identity, and we
have a request for that Ed25519 ID and a different RSA identity, we
do not attempt to make another connection: we just fail and DESTROY
the circuit.
If we receive an EXTEND or EXTEND2 request for a node listed in the
consensus, but that EXTEND/EXTEND2 request does not include an
Ed25519 identity key, the node SHOULD treat the connection as failed
if the Ed25519 identity key it receives does not match the one in the
consensus.
6. Naming nodes in the interface
Anywhere in the interface that takes an $identity should be able to
take an ECC identity too. ECC identities are case-sensitive base64
encodings of Ed25519 identity keys. You can use $ to indicate them as
well; we distinguish RSA identity digests by length.
When we need to indicate an Ed25519 identity key in a hostname
format (as in a .exit address), we use the lowercased version of the
name, and perform a case-insensitive match. (This loses us a little
less than one bit per byte of name, leaving plenty of bits to make
sure we choose the right node.)
Nodes must not list Ed25519 identities in their family lines; clients and
authorities must not honor them there. (Doing so would make different
clients change paths differently in a possibly manipulatable way.)
Clients shouldn't accept .exit addresses with Ed25519 names on SOCKS
or DNS ports by default, even when AllowDotExit is set. We can add
another option for them later if there's a good reason to have this.
We need an identity-to-node map for ECC identity and for RSA
identity.
The controller interface will need to accept and report Ed25519
identity keys as well as (or instead of) RSA identity keys. That's a
separate proposal, though.
7. Hidden service changes out of scope
Hidden services need to be able to identify nodes by ECC keys, just as
they will need to include ntor keys as well as TAP keys. Not just
yet though. This needs to be part of a bigger hidden service
revamping strategy.
8. Proposed migration steps
Once a few versions have shipped with Ed25519 key support, turn on
"Rule 1" on the authorities. (Don't allow an Ed25519<->RSA pairing
to change.)
Once the release with these changes is in beta or rc, turn on the
consensus option for everyone who receives descriptors with
Ed25519 identity keys to check them.
Once the release with these changes is in beta or rc, turn on the
consensus option for clients to generate EXTEND2 requests with
Ed25519 identity keys.
Once the release with these changes has been stable for a month
or two, turn on "Rule 2" on authorities. (Don't allow nodes that
have advertised an Ed25519 key to stop.)
9. Future proposals
* Ed25519 identity support on the controller interface
* Supporting nodes without RSA keys
* Remove support for nodes without Ed25519 keys
* Ed25519 support for hidden services
* Bridge identity support.
* Ed25519-aware family support
A.1. List of certificate types
The values marked with asterisks are not types corresponding to
the certificate format of section 2.1. Instead, they are
reserved for RSA-signed certificates to avoid conflicts between
the certificate type enumeration of the CERTS cell and the
certificate type enumeration of in our Ed25519 certificates.
**[00],[01],[02],[03] - Reserved to avoid conflict with types used
in CERTS cells.
[04] - signing a signing key with an identity key (Section 2.5)
[05] - TLS link certificate signed with ed25519 signing key
(Section 4.2)
[06] - Ed25519 authentication key signed with ed25519 signing key
(Section 4.2)
**[07] - reserved for RSA identity cross-certification (Section 4.2)
A.2. List of extension types
[01] - signed-with-ed25519-key (section 2.2.1)
A.3. List of signature prefixes
We describe various documents as being signed with a prefix. Here
are those prefixes:
"Tor router descriptor signature v1" (section 2.5)
"Tor node signing key certificate v1" (section 2.1)
A.4. List of certified key types
[01] ed25519 key
[02] SHA256 hash of an RSA key
[03] SHA256 hash of an X.509 certificate
A.5. Reserved numbers
We need a new consensus algorithm number to encompass checking
ed25519 keys and putting them in microdescriptors.
We need new CertType values for use in CERTS cells. We reserved
in section 4.2.
4: Ed25519 signing key
5: Link key certificate certified by Ed25519 signing key
6: TLS authentication key certified by Ed25519 signing key
7: RSA cross-certificate for Ed25519 identity key
[View Less]
Filename: xxx-kill-named-flag.txt
Title: Stop assigning (and eventually supporting) the Named flag
Authors: Sebastian Hahnn
Created: 10 April 2014
Target: 0.2.5
Status: Draft
1. Intro and motivation
Currently, Tor supports the concept of linking a Tor relay's nickname
to its identity key. This happens automatically as a new relay joins
the network with a unique nickname, and keeps it for a while. To
indicate that a …
[View More]nickname is linked to the presented identity, the
directory authorities vote on a Named flag for all relays where they
have such a link. Not all directory authorities are currently doing
this - in fact, there are only two, gabelmoo and tor26.
For a long time, we've been telling everyone to not rely on relay
nicknames, even if the Named flag is assigned. This has two reasons:
First off, it adds another trust requirement on the directory
authorities, and secondly naming may change over time as relays go
offline for substantial amounts of time.
Now that a significant portion of the network is required to rotate
their identity keys, few relays will keep their Named flag. We should
use this chance to stop assigning Named flags.
2. Design
None. Tor clients already support consensuses without Named flags,
and testing in private Tor networks has never revealed any issues in
this regard.
3. Implementation
The gabelmoo and tor26 directory authorities can simply remove the
NamingAuthoritativeDirectory configuration option to stop giving out
Named flags. This will mean the consensus won't include Named and
Unnamed flags any longer. The code collecting naming statistics is
independent of Tor, so it can run a while longer to ensure Naming can
be switched on if unforeseen issues arise.
Once this has been shown to not cause any issues, support for the
Named flag can be removed from the Tor client implementation, and
support for the NamingAuthoritativeDirectory can be removed from the
Tor directory authority implementation.
4. Open questions
None.
[View Less]
I have been looking at doing some work on Tor as part of my degree, and
more specifically, looking at Hidden Services. One of the issues where I
believe I might be able to make some progress, is the Hidden Service
Scaling issue as described here [1].
So, before I start trying to implement a prototype, I thought I would
set out my ideas here to check they are reasonable (I have also been
discussing this a bit on #tor-dev). The goal of this is two fold, to
reduce the probability of failure of a …
[View More]hidden service and to increase
hidden service scalability.
I think what I am planning distils down to two main changes. Firstly,
when a OP initialises a hidden service, currently if you start a hidden
service using an existing keypair and address, the new OP's introduction
points replace the existing introduction points [2]. This does provide
some redundancy (if slow), but no load balancing.
My current plan is to change this such that if the OP has an existing
public/private keypair and address, it would attempt to lookup the
existing introduction points (probably over a Tor circuit). If found, it
then establishes introduction circuits to those Tor servers.
Then comes the second problem, following the above, the introduction
point would then disconnect from any other connected OP using the same
public key (unsure why as a reason is not given in the rend-spec). This
would need to change such that an introduction point can talk to more
than one instance of the hidden service.
These two changes combined should help with the two goals. Reliability
is improved by having multiple OP's providing the service, and having
all of these accessible from the introduction points. Scalability is
also improved, as you are not limited to one OP (as described above,
currently you can also have +1 but only one will receive most of the
traffic, and fail over is slow).
I am aware that there are several undefined parts of the above
description, e.g. how does a introduction point choose what circuit to
use? but at the moment I am more interested in the wider picture. It
would be good to get some feedback on this.
1: https://blog.torproject.org/blog/hidden-services-need-some-love
2:
http://tor.stackexchange.com/questions/13/can-a-hidden-service-be-hosted-by…
[View Less]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
I'm a student who is starting to work with ahmia.fi search engine as a
part of Google Summer of Code. :)
The proposal is online here https://ahmia.fi/gsoc/
In practise, I have now time and funding to develop my search engine.
George is my primary mentor and Moritz the backup mentor.
Today, I will submit all the required documents (the tax forms etc.)
to Google.
After that, I think I will speed up with code base in the GitHub :)
Cheers,
…
[View More]Juha
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJTViWUAAoJELGTs54GL8vAuC8H/jSdgBCRQM/3l3mX5Uig9fgM
wacPsxm6RJd3Sw+JJpYgoRP1nDqI513haP4Z6s//tR3Vn5RyQ/u7ik3QdFEVKbJD
KqnQ4Eaf5hT4xsJwBXZIjzW6uhbYaq1GmUJi4eaglwUrgIgJrHzDbOz/p8q71O1z
rLnrS1vrsvMzY4rU0dRe1/S9LyPWTUAfpVMINa54RPmNjMzrTT/WUnlcQWo9cY3a
SRrT2MVz5nwBEXJuhZUmC3L6XLL8RX2TgzGwVyYOUfMlNuZdcSaOOTvF7gKVZVZQ
hGhr/V40iNm5BOAcQ2TVaxuR5HjxSFWUp15T8ux+xxyN/Yp9EeaDjsAsTVegq0w=
=QMfR
-----END PGP SIGNATURE-----
[View Less]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Sorry everyone for the long pause.
I wrote down a proposal (and some code) to address issues raised by
Mike and George:
https://securehomes.esat.kuleuven.be/~gacar/summer_2014.pdf
Looking for your comments and critics...
On 03/21/2014 11:39 PM, Peter Eckersley wrote:
> I think we're fine with open sourcing under the Affero GPLv3.
>
>
> On Tue, Mar 18, 2014 at 12:28:12PM -0700, Mike Perry wrote:
>> Yan Zhu:
>>> On 03/…
[View More]17/2014 04:41 AM, Gunes Acar wrote:
>>>> Hi Yan,
>>>>
>>>> Glad that you're interested in the project. It'd be very nice
>>>> collaborate with you on this.
>>>>
>>>> Indeed, we've been corresponding with Peter for a related
>>>> project and I mentioned my intention to work as a middleman
>>>> between EFF and Tor.
>>>>
>>>
>>> Great, it seems that Peter and I are both interested and
>>> willing to help.
>>>
>>> Regarding
>>> https://trac.torproject.org/projects/tor/ticket/6119#comment:10,
>>> Peter says he has some reluctance to open source the project
>>> (not the data) because it might make it easier for some
>>> websites to track visitors without their consent.
>>
>> This might have been a valid concern 5 years ago, but now it's
>> just a joke. The tests on Panopticlick are ancient, widely known,
>> easy to reproduce, and since then much more severe and invasive
>> mechanisms of fingerprinting have since been developed/deployed
>> in modern browsers.
>>
>> Moreover, only 2 of the tests it performs actually apply to Tor
>> Browser users.
>>
>> Banks in particular have already deployed some of the techniques
>> we've fixed that the EFF study entirely predates. And these
>> techniques are far higher entropy than browser resolution (such
>> as localhost open port enumeration, OS theme fingerprinting, and
>> HTML5+WebGL canvas rendering+extraction+hashing).
>>
>> Not only should we (as Tor) publicly provide tests and
>> easy-to-deploy working PoC code for all of these vectors, we
>> should also endeavor to detail cases where major browser vendors
>> are ignoring or exacerbating this problem, and make it easy for
>> everyone to test and observe this behavior themselves.
>>
>> Not sure if that means the EFF now has a conflict of interest
>> with this project for some ridiculous reason, but frankly any
>> attempt at trying to "hide" these techniques is downright silly.
>> They are too well known (most are publicly documented elsewhere,
>> or at least on our bugtracker), and there's waaay too much money
>> on the other side of the fence in terms of incentives to develop
>> and deploy working attacks.
>>
>> Further, starting the from EFF codebase might also be a hindrance
>> to us. It is not designed for measuring the effects of defenses.
>> In fact, its measurement mechanisms actively penalize any attempt
>> at defense development (because any approach to alter browser
>> behavior instantly makes you more unique than the previous
>> userbase).
>>
>> I actually think Panopticlick has of late done more to prevent
>> browser fingerprinting defense development than to encourage it.
>> I would really like to see it DIAF.
>>
>> Here's hoping we can make something better!
>>
>> -- Mike Perry
>
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJTUg4mAAoJEPb7JcMmVt4gQjwIALCsTOxvUFP3HY0N8Ap9fpKW
GD193EW32X80iH6VY54AIWA29wxKaKEM4vBJBVLkhYt8s68OAqaV1vMNtmEev26h
2yg9us2HNYjBzaxFQpX7qhmDCiucpe3zVZZXq9T34OhjxscWc90JdvWA5D8Eiqto
exJzqi3k5djFU66apzfFAwYpk8E0Og582XFg5TOQFYGo6LvNyT69LH7+jlNsHL65
atspVO47wKH4+0nhoG22tsMdZRzhmgSbSB1gx2a7Esf3dOBf+R0BBw+qcZptqq8Z
NPyobAecQzmV/SXgjDF4PaE12A4IkK4nCWUax7ksE6YbCNhAKBlcAt+/q2JeOt8=
=i+/g
-----END PGP SIGNATURE-----
[View Less]
Hello everyone,
My name is Daniel and this summer I'll be working on consensus diffs
[0], heavily based on proposal 140 [1]. This should allow for quicker
and scalable consensus udpates, which will have more weight as the
consensus grows larger. I've never gotten involved in Tor before, so I'm
really looking forward to this summer.
My intention is to use a simplified ed format as described on proposal
140, but I am open to alternatives and suggestions. As far as the diff
creation algorithm, I …
[View More]plan on using a dynamic programming algorithm for
the Longest Common Substring problem. Like before, comments are very
welcome.
I will spend the following couple of weeks looking at alternative diff
formats and algorithms, rather than start coding this early. It would be
appreciated if any ideas regarding any of the two aspects were posed
during this time, so that afterward I can start their implementation.
I will always be lurking on #tor-dev, #tor-project and #tor under the
nick 'mvdan'. I am also subscribed to the tor-dev and tor-talk mailing
lists. And lastly, my PGP fingerprint is below - encrypted mail is
welcome :)
Regards.
[0] https://www.torproject.org/getinvolved/volunteer.html.en#consensusDiffs
[1] https://gitweb.torproject.org/torspec.git/blob_plain/refs/heads/master:/pro…
--
Daniel Martí - mvdan(a)mvdan.cc - http://mvdan.cc/
PGP: A9DA 13CD F7A1 4ACD D3DE E530 F4CA FFDB 4348 041C
[View Less]