[or-cvs] r9411: Move specification documents into new doc/spec subdirectory. (in tor/trunk: . doc doc/spec)
nickm at seul.org
nickm at seul.org
Fri Jan 26 01:59:53 UTC 2007
Author: nickm
Date: 2007-01-25 20:59:50 -0500 (Thu, 25 Jan 2007)
New Revision: 9411
Added:
tor/trunk/doc/spec/address-spec.txt
tor/trunk/doc/spec/control-spec-v0.txt
tor/trunk/doc/spec/control-spec.txt
tor/trunk/doc/spec/dir-spec.txt
tor/trunk/doc/spec/path-spec.txt
tor/trunk/doc/spec/rend-spec.txt
tor/trunk/doc/spec/socks-extensions.txt
tor/trunk/doc/spec/tor-spec.txt
tor/trunk/doc/spec/version-spec.txt
Removed:
tor/trunk/doc/address-spec.txt
tor/trunk/doc/control-spec-v0.txt
tor/trunk/doc/control-spec.txt
tor/trunk/doc/dir-spec.txt
tor/trunk/doc/path-spec.txt
tor/trunk/doc/rend-spec.txt
tor/trunk/doc/socks-extensions.txt
tor/trunk/doc/tor-spec.txt
tor/trunk/doc/version-spec.txt
Modified:
tor/trunk/configure.in
tor/trunk/doc/Makefile.am
Log:
Move specification documents into new doc/spec subdirectory. (Proposals, drafts, and bad ideas still remain in doc.)
Modified: tor/trunk/configure.in
===================================================================
--- tor/trunk/configure.in 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/configure.in 2007-01-26 01:59:50 UTC (rev 9411)
@@ -858,7 +858,7 @@
fi
-AC_OUTPUT(Makefile tor.spec contrib/tor.sh contrib/torctl contrib/torify contrib/tor.logrotate contrib/Makefile contrib/osx/Makefile contrib/osx/TorBundleDesc.plist contrib/osx/TorBundleInfo.plist contrib/osx/TorDesc.plist contrib/osx/TorInfo.plist contrib/osx/TorStartupDesc.plist src/config/torrc.sample doc/tor.1 src/Makefile doc/Makefile doc/design-paper/Makefile src/config/Makefile src/common/Makefile src/or/Makefile src/win32/Makefile src/tools/Makefile contrib/suse/Makefile contrib/suse/tor.sh)
+AC_OUTPUT(Makefile tor.spec contrib/tor.sh contrib/torctl contrib/torify contrib/tor.logrotate contrib/Makefile contrib/osx/Makefile contrib/osx/TorBundleDesc.plist contrib/osx/TorBundleInfo.plist contrib/osx/TorDesc.plist contrib/osx/TorInfo.plist contrib/osx/TorStartupDesc.plist src/config/torrc.sample doc/tor.1 src/Makefile doc/Makefile doc/design-paper/Makefile doc/spec/Makefile src/config/Makefile src/common/Makefile src/or/Makefile src/win32/Makefile src/tools/Makefile contrib/suse/Makefile contrib/suse/tor.sh)
if test -x /usr/bin/perl && test -x ./contrib/updateVersions.pl ; then
./contrib/updateVersions.pl
Modified: tor/trunk/doc/Makefile.am
===================================================================
--- tor/trunk/doc/Makefile.am 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/Makefile.am 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,15 +1,12 @@
-EXTRA_DIST = tor-spec.txt rend-spec.txt control-spec.txt \
- dir-spec.txt socks-extensions.txt path-spec.txt \
- version-spec.txt address-spec.txt \
- website img HACKING \
+EXTRA_DIST = website img HACKING \
tor-resolve.1 \
tor-osx-dmg-creation.txt tor-rpm-creation.txt \
tor-win32-mingw-creation.txt
man_MANS = tor.1 tor-resolve.1
-SUBDIRS = design-paper
+SUBDIRS = design-paper spec
DIST_SUBDIRS = design-paper
Deleted: tor/trunk/doc/address-spec.txt
===================================================================
--- tor/trunk/doc/address-spec.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/address-spec.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,64 +0,0 @@
-$Id$
-
- Special Hostnames in Tor
- Nick Mathewson
-
-1. Overview
-
- Most of the time, Tor treats user-specified hostnames as opaque: When the
- user connects to tor.eff.org, Tor picks an exit node and uses that node to
- connect to "tor.eff.org". Some hostnames, however, can be used to override
- Tor's default behavior and circuit-building rules.
-
- These hostnames can be passed to Tor as the address part of a SOCKS4a or
- SOCKS5 request. If the application is connected to Tor using an IP-only
- method (such as SOCKS4, TransPort, or NatdPort), these hostnames can be
- substituted for certain IP addresses using the MapAddress configuration
- option or the MAPADDRESS control command.
-
-2. .exit
-
- SYNTAX: [hostname].[name-or-digest].exit
- [name-or-digest].exit
-
- Hostname is a valid hostname; [name-or-digest] is either the nickname of a
- Tor node or the hex-encoded digest of that node's public key.
-
- When Tor sees an address in this format, it uses the specified hostname as
- the exit node. If no "hostname" component is given, Tor defaults to the
- published IPv4 address of the exit node.
-
- It is valid to try to resolve hostnames, and in fact upon success Tor
- will cache an internal mapaddress of the form
- "www.google.com.foo.exit=64.233.161.99.foo.exit" to speed subsequent
- lookups.
-
- EXAMPLES:
- www.example.com.exampletornode.exit
-
- Connect to www.example.com from the node called "exampletornode."
-
- exampletornode.exit
-
- Connect to the published IP address of "exampletornode" using
- "exampletornode" as the exit.
-
-3. .onion
-
- SYNTAX: [digest].onion
-
- The digest is the first eighty bits of a SHA1 hash of the identity key for
- a hidden service, encoded in base32.
-
- When Tor sees an address in this format, it tries to look up and connect to
- the specified hidden service. See rend-spec.txt for full details.
-
-4. .noconnect
-
- SYNTAX: [string].noconnect
-
- When Tor sees an address in this format, it immediately closes the
- connection without attaching it to any circuit. This is useful for
- controllers that want to test whether a given application is indeed using
- the same instance of Tor that they're controlling.
-
Deleted: tor/trunk/doc/control-spec-v0.txt
===================================================================
--- tor/trunk/doc/control-spec-v0.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/control-spec-v0.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,499 +0,0 @@
-$Id$
-
- TC: A Tor control protocol (Version 0)
-
--1. Deprecation
-
-THIS PROTOCOL IS DEPRECATED. It is still documented here because Tor
-0.1.1.x happens to support much of it; but the support for v0 is not
-maintained, so you should expect it to rot in unpredictable ways. Support
-for v0 will be removed some time after Tor 0.1.2.
-
-0. Scope
-
-This document describes an implementation-specific protocol that is used
-for other programs (such as frontend user-interfaces) to communicate
-with a locally running Tor process. It is not part of the Tor onion
-routing protocol.
-
-We're trying to be pretty extensible here, but not infinitely
-forward-compatible.
-
-1. Protocol outline
-
-TC is a bidirectional message-based protocol. It assumes an underlying
-stream for communication between a controlling process (the "client") and
-a Tor process (the "server"). The stream may be implemented via TCP,
-TLS-over-TCP, a Unix-domain socket, or so on, but it must provide
-reliable in-order delivery. For security, the stream should not be
-accessible by untrusted parties.
-
-In TC, the client and server send typed variable-length messages to each
-other over the underlying stream. By default, all messages from the server
-are in response to messages from the client. Some client requests, however,
-will cause the server to send messages to the client indefinitely far into
-the future.
-
-Servers respond to messages in the order they're received.
-
-2. Message format
-
-The messages take the following format:
-
- Length [2 octets; big-endian]
- Type [2 octets; big-endian]
- Body [Length octets]
-
-Upon encountering a recognized Type, implementations behave as described in
-section 3 below. If the type is not recognized, servers respond with an
-"ERROR" message (code UNRECOGNIZED; see 3.1 below), and clients simply ignore
-the message.
-
-2.1. Types and encodings
-
- All numbers are given in big-endian (network) order.
-
- OR identities are given in hexadecimal, in the same format as identity key
- fingerprints, but without spaces; see tor-spec.txt for more information.
-
-3. Message types
-
- Message types are drawn from the following ranges:
-
- 0x0000-0xEFFF : Reserved for use by official versions of this spec.
- 0xF000-0xFFFF : Unallocated; usable by unofficial extensions.
-
-3.1. ERROR (Type 0x0000)
-
- Sent in response to a message that could not be processed as requested.
-
- The body of the message begins with a 2-byte error code. The following
- values are defined:
-
- 0x0000 Unspecified error
- []
-
- 0x0001 Internal error
- [Something went wrong inside Tor, so that the client's
- request couldn't be fulfilled.]
-
- 0x0002 Unrecognized message type
- [The client sent a message type we don't understand.]
-
- 0x0003 Syntax error
- [The client sent a message body in a format we can't parse.]
-
- 0x0004 Unrecognized configuration key
- [The client tried to get or set a configuration option we don't
- recognize.]
-
- 0x0005 Invalid configuration value
- [The client tried to set a configuration option to an
- incorrect, ill-formed, or impossible value.]
-
- 0x0006 Unrecognized byte code
- [The client tried to set a byte code (in the body) that
- we don't recognize.]
-
- 0x0007 Unauthorized.
- [The client tried to send a command that requires
- authorization, but it hasn't sent a valid AUTHENTICATE
- message.]
-
- 0x0008 Failed authentication attempt
- [The client sent a well-formed authorization message.]
-
- 0x0009 Resource exhausted
- [The server didn't have enough of a given resource to
- fulfill a given request.]
-
- 0x000A No such stream
-
- 0x000B No such circuit
-
- 0x000C No such OR
-
- The rest of the body should be a human-readable description of the error.
-
- In general, new error codes should only be added when they don't fall under
- one of the existing error codes.
-
-3.2. DONE (Type 0x0001)
-
- Sent from server to client in response to a request that was successfully
- completed, with no more information needed. The body is usually empty but
- may contain a message.
-
-3.3. SETCONF (Type 0x0002)
-
- Change the value of a configuration variable. The body contains a list of
- newline-terminated key-value configuration lines. An individual key-value
- configuration line consists of the key, followed by a space, followed by
- the value. The server behaves as though it had just read the key-value pair
- in its configuration file.
-
- The server responds with a DONE message on success, or an ERROR message on
- failure.
-
- When a configuration options takes multiple values, or when multiple
- configuration keys form a context-sensitive group (see below), then
- setting _any_ of the options in a SETCONF command is taken to reset all of
- the others. For example, if two ORBindAddress values are configured,
- and a SETCONF command arrives containing a single ORBindAddress value, the
- new command's value replaces the two old values.
-
- To _remove_ all settings for a given option entirely (and go back to its
- default value), send a single line containing the key and no value.
-
-3.4. GETCONF (Type 0x0003)
-
- Request the value of a configuration variable. The body contains one or
- more NL-terminated strings for configuration keys. The server replies
- with a CONFVALUE message.
-
- If an option appears multiple times in the configuration, all of its
- key-value pairs are returned in order.
-
- Some options are context-sensitive, and depend on other options with
- different keywords. These cannot be fetched directly. Currently there
- is only one such option: clients should use the "HiddenServiceOptions"
- virtual keyword to get all HiddenServiceDir, HiddenServicePort,
- HiddenServiceNodes, and HiddenServiceExcludeNodes option settings.
-
-3.5. CONFVALUE (Type 0x0004)
-
- Sent in response to a GETCONF message; contains a list of "Key Value\n"
- (A non-whitespace keyword, a single space, a non-NL value, a NL)
- strings.
-
-3.6. SETEVENTS (Type 0x0005)
-
- Request the server to inform the client about interesting events.
- The body contains a list of 2-byte event codes (see "event" below).
- Any events *not* listed in the SETEVENTS body are turned off; thus, sending
- SETEVENTS with an empty body turns off all event reporting.
-
- The server responds with a DONE message on success, and an ERROR message
- if one of the event codes isn't recognized. (On error, the list of active
- event codes isn't changed.)
-
-3.7. EVENT (Type 0x0006)
-
- Sent from the server to the client when an event has occurred and the
- client has requested that kind of event. The body contains a 2-byte
- event code followed by additional event-dependent information. Event
- codes are:
- 0x0001 -- Circuit status changed
-
- Status [1 octet]
- 0x00 Launched - circuit ID assigned to new circuit
- 0x01 Built - all hops finished, can now accept streams
- 0x02 Extended - one more hop has been completed
- 0x03 Failed - circuit closed (was not built)
- 0x04 Closed - circuit closed (was built)
- Circuit ID [4 octets]
- (Must be unique to Tor process/time)
- Path [NUL-terminated comma-separated string]
- (For extended/failed, is the portion of the path that is
- built)
-
- 0x0002 -- Stream status changed
-
- Status [1 octet]
- (Sent connect=0,sent resolve=1,succeeded=2,failed=3,
- closed=4, new connection=5, new resolve request=6,
- stream detached from circuit and still retriable=7)
- Stream ID [4 octets]
- (Must be unique to Tor process/time)
- Target (NUL-terminated address-port string]
-
- 0x0003 -- OR Connection status changed
-
- Status [1 octet]
- (Launched=0,connected=1,failed=2,closed=3)
- OR nickname/identity [NUL-terminated]
-
- 0x0004 -- Bandwidth used in the last second
-
- Bytes read [4 octets]
- Bytes written [4 octets]
-
- 0x0005 -- Notice/warning/error occurred
-
- Message [NUL-terminated]
-
- <obsolete: use 0x0007-0x000B instead.>
-
- 0x0006 -- New descriptors available
-
- OR List [NUL-terminated, comma-delimited list of
- OR identity]
-
- 0x0007 -- Debug message occurred
- 0x0008 -- Info message occurred
- 0x0009 -- Notice message occurred
- 0x000A -- Warning message occurred
- 0x000B -- Error message occurred
-
- Message [NUL-terminated]
-
-3.8. AUTHENTICATE (Type 0x0007)
-
- Sent from the client to the server. Contains a 'magic cookie' to prove
- that client is really allowed to control this Tor process. The server
- responds with DONE or ERROR.
-
- The format of the 'cookie' is implementation-dependent; see 4.1 below for
- information on how the standard Tor implementation handles it.
-
-3.9. SAVECONF (Type 0x0008)
-
- Sent from the client to the server. Instructs the server to write out
- its config options into its torrc. Server returns DONE if successful, or
- ERROR if it can't write the file or some other error occurs.
-
-3.10. SIGNAL (Type 0x0009)
-
- Sent from the client to the server. The body contains one byte that
- indicates the action the client wishes the server to take.
-
- 1 (0x01) -- Reload: reload config items, refetch directory.
- 2 (0x02) -- Controlled shutdown: if server is an OP, exit immediately.
- If it's an OR, close listeners and exit after 30 seconds.
- 10 (0x0A) -- Dump stats: log information about open connections and
- circuits.
- 12 (0x0C) -- Debug: switch all open logs to loglevel debug.
- 15 (0x0F) -- Immediate shutdown: clean up and exit now.
-
- The server responds with DONE if the signal is recognized (or simply
- closes the socket if it was asked to close immediately), else ERROR.
-
-3.11. MAPADDRESS (Type 0x000A)
-
- Sent from the client to the server. The body contains a sequence of
- address mappings, each consisting of the address to be mapped, a single
- space, the replacement address, and a NL character.
-
- Addresses may be IPv4 addresses, IPv6 addresses, or hostnames.
-
- The client sends this message to the server in order to tell it that future
- SOCKS requests for connections to the original address should be replaced
- with connections to the specified replacement address. If the addresses
- are well-formed, and the server is able to fulfill the request, the server
- replies with a single DONE message containing the source and destination
- addresses. If request is malformed, the server replies with a syntax error
- message. The server can't fulfill the request, it replies with an internal
- ERROR message.
-
- The client may decline to provide a body for the original address, and
- instead send a special null address ("0.0.0.0" for IPv4, "::0" for IPv6, or
- "." for hostname), signifying that the server should choose the original
- address itself, and return that address in the DONE message. The server
- should ensure that it returns an element of address space that is unlikely
- to be in actual use. If there is already an address mapped to the
- destination address, the server may reuse that mapping.
-
- If the original address is already mapped to a different address, the old
- mapping is removed. If the original address and the destination address
- are the same, the server removes any mapping in place for the original
- address.
-
- {Note: This feature is designed to be used to help Tor-ify applications
- that need to use SOCKS4 or hostname-less SOCKS5. There are three
- approaches to doing this:
- 1. Somehow make them use SOCKS4a or SOCKS5-with-hostnames instead.
- 2. Use tor-resolve (or another interface to Tor's resolve-over-SOCKS
- feature) to resolve the hostname remotely. This doesn't work
- with special addresses like x.onion or x.y.exit.
- 3. Use MAPADDRESS to map an IP address to the desired hostname, and then
- arrange to fool the application into thinking that the hostname
- has resolved to that IP.
- This functionality is designed to help implement the 3rd approach.}
-
- [XXXX When, if ever, can mappings expire? Should they expire?]
- [XXXX What addresses, if any, are safe to use?]
-
-3.12 GETINFO (Type 0x000B)
-
- Sent from the client to the server. The message body is as for GETCONF:
- one or more NL-terminated strings. The server replies with an INFOVALUE
- message.
-
- Unlike GETCONF, this message is used for data that are not stored in the
- Tor configuration file, but instead.
-
- Recognized key and their values include:
-
- "version" -- The version of the server's software, including the name
- of the software. (example: "Tor 0.0.9.4")
-
- "desc/id/<OR identity>" or "desc/name/<OR nickname>" -- the latest server
- descriptor for a given OR, NUL-terminated. If no such OR is known, the
- corresponding value is an empty string.
-
- "network-status" -- a space-separated list of all known OR identities.
- This is in the same format as the router-status line in directories;
- see tor-spec.txt for details.
-
- "addr-mappings/all"
- "addr-mappings/config"
- "addr-mappings/cache"
- "addr-mappings/control" -- a NL-terminated list of address mappings, each
- in the form of "from-address" SP "to-address". The 'config' key
- returns those address mappings set in the configuration; the 'cache'
- key returns the mappings in the client-side DNS cache; the 'control'
- key returns the mappings set via the control interface; the 'all'
- target returns the mappings set through any mechanism.
-
-3.13 INFOVALUE (Type 0x000C)
-
- Sent from the server to the client in response to a GETINFO message.
- Contains one or more items of the format:
-
- Key [(NUL-terminated string)]
- Value [(NUL-terminated string)]
-
- The keys match those given in the GETINFO message.
-
-3.14 EXTENDCIRCUIT (Type 0x000D)
-
- Sent from the client to the server. The message body contains two fields:
- Circuit ID [4 octets]
- Path [NUL-terminated, comma-delimited string of OR nickname/identity]
-
- This request takes one of two forms: either the Circuit ID is zero, in
- which case it is a request for the server to build a new circuit according
- to the specified path, or the Circuit ID is nonzero, in which case it is a
- request for the server to extend an existing circuit with that ID according
- to the specified path.
-
- If the request is successful, the server sends a DONE message containing
- a message body consisting of the four-octet Circuit ID of the newly created
- circuit.
-
-3.15 ATTACHSTREAM (Type 0x000E)
-
- Sent from the client to the server. The message body contains two fields:
- Stream ID [4 octets]
- Circuit ID [4 octets]
-
- This message informs the server that the specified stream should be
- associated with the specified circuit. Each stream may be associated with
- at most one circuit, and multiple streams may share the same circuit.
- Streams can only be attached to completed circuits (that is, circuits that
- have sent a circuit status 'built' event).
-
- If the circuit ID is 0, responsibility for attaching the given stream is
- returned to Tor.
-
- {Implementation note: By default, Tor automatically attaches streams to
- circuits itself, unless the configuration variable
- "__LeaveStreamsUnattached" is set to "1". Attempting to attach streams
- via TC when "__LeaveStreamsUnattached" is false may cause a race between
- Tor and the controller, as both attempt to attach streams to circuits.}
-
-3.16 POSTDESCRIPTOR (Type 0x000F)
-
- Sent from the client to the server. The message body contains one field:
- Descriptor [NUL-terminated string]
-
- This message informs the server about a new descriptor.
-
- The descriptor, when parsed, must contain a number of well-specified
- fields, including fields for its nickname and identity.
-
- If there is an error in parsing the descriptor, the server must send an
- appropriate error message. If the descriptor is well-formed but the server
- chooses not to add it, it must reply with a DONE message whose body
- explains why the server was not added.
-
-3.17 FRAGMENTHEADER (Type 0x0010)
-
- Sent in either direction. Used to encapsulate messages longer than 65535
- bytes in length.
-
- Underlying type [2 bytes]
- Total Length [4 bytes]
- Data [Rest of message]
-
- A FRAGMENTHEADER message MUST be followed immediately by a number of
- FRAGMENT messages, such that lengths of the "Data" fields of the
- FRAGMENTHEADER and FRAGMENT messages add to the "Total Length" field of the
- FRAGMENTHEADER message.
-
- Implementations MUST NOT fragment messages of length less than 65536 bytes.
- Implementations MUST be able to process fragmented messages that not
- optimally packed.
-
-3.18 FRAGMENT (Type 0x0011)
-
- Data [Entire message]
-
- See FRAGMENTHEADER for more information
-
-3.19 REDIRECTSTREAM (Type 0x0012)
-
- Sent from the client to the server. The message body contains two fields:
- Stream ID [4 octets]
- Address [variable-length, NUL-terminated.]
-
- Tells the server to change the exit address on the specified stream. No
- remapping is performed on the new provided address.
-
- To be sure that the modified address will be used, this event must be sent
- after a new stream event is received, and before attaching this stream to
- a circuit.
-
-3.20 CLOSESTREAM (Type 0x0013)
-
- Sent from the client to the server. The message body contains three
- fields:
- Stream ID [4 octets]
- Reason [1 octet]
- Flags [1 octet]
-
- Tells the server to close the specified stream. The reason should be
- one of the Tor RELAY_END reasons given in tor-spec.txt. Flags is not
- used currently. Tor may hold the stream open for a while to flush
- any data that is pending.
-
-3.21 CLOSECIRCUIT (Type 0x0014)
-
- Sent from the client to the server. The message body contains two
- fields:
- Circuit ID [4 octets]
- Flags [1 octet]
-
- Tells the server to close the specified circuit. If the LSB of the flags
- field is nonzero, do not close the circuit unless it is unused.
-
-4. Implementation notes
-
-4.1. Authentication
-
- By default, the current Tor implementation trusts all local users.
-
- If the 'CookieAuthentication' option is true, Tor writes a "magic cookie"
- file named "control_auth_cookie" into its data directory. To authenticate,
- the controller must send the contents of this file.
-
- If the 'HashedControlPassword' option is set, it must contain the salted
- hash of a secret password. The salted hash is computed according to the
- S2K algorithm in RFC 2440 (OpenPGP), and prefixed with the s2k specifier.
- This is then encoded in hexadecimal, prefixed by the indicator sequence
- "16:". Thus, for example, the password 'foo' could encode to:
- 16:660537E3E1CD49996044A3BF558097A981F539FEA2F9DA662B4626C1C2
- ++++++++++++++++**^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- salt hashed value
- indicator
- You can generate the salt of a password by calling
- 'tor --hash-password <password>'
- or by using the example code in the Python and Java controller libraries.
- To authenticate under this scheme, the controller sends Tor the original
- secret that was used to generate the password.
-
-4.2. Don't let the buffer get too big.
-
- If you ask for lots of events, and 16MB of them queue up on the buffer,
- the Tor process will close the socket.
-
Deleted: tor/trunk/doc/control-spec.txt
===================================================================
--- tor/trunk/doc/control-spec.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/control-spec.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,1313 +0,0 @@
-$Id$
-
- TC: A Tor control protocol (Version 1)
-
-0. Scope
-
- This document describes an implementation-specific protocol that is used
- for other programs (such as frontend user-interfaces) to communicate with a
- locally running Tor process. It is not part of the Tor onion routing
- protocol.
-
- This protocol replaces version 0 of TC, which is now deprecated. For
- reference, TC is described in "control-spec-v0.txt". Implementors are
- recommended to avoid using TC directly, but instead to use a library that
- can easily be updated to use the newer protocol. (Version 0 is used by Tor
- versions 0.1.0.x; the protocol in this document only works with Tor
- versions in the 0.1.1.x series and later.)
-
-1. Protocol outline
-
- TC is a bidirectional message-based protocol. It assumes an underlying
- stream for communication between a controlling process (the "client"
- or "controller") and a Tor process (or "server"). The stream may be
- implemented via TCP, TLS-over-TCP, a Unix-domain socket, or so on,
- but it must provide reliable in-order delivery. For security, the
- stream should not be accessible by untrusted parties.
-
- In TC, the client and server send typed messages to each other over the
- underlying stream. The client sends "commands" and the server sends
- "replies".
-
- By default, all messages from the server are in response to messages from
- the client. Some client requests, however, will cause the server to send
- messages to the client indefinitely far into the future. Such
- "asynchronous" replies are marked as such.
-
- Servers respond to messages in the order messages are received.
-
-2. Message format
-
-2.1. Description format
-
- The message formats listed below use ABNF as described in RFC 2234.
- The protocol itself is loosely based on SMTP (see RFC 2821).
-
- We use the following nonterminals from RFC 2822: atom, qcontent
-
- We define the following general-use nonterminals:
-
- String = DQUOTE *qcontent DQUOTE
-
- There are explicitly no limits on line length. All 8-bit characters are
- permitted unless explicitly disallowed.
-
-2.2. Commands from controller to Tor
-
- Command = Keyword Arguments CRLF / "+" Keyword Arguments CRLF Data
- Keyword = 1*ALPHA
- Arguments = *(SP / VCHAR)
-
- Specific commands and their arguments are described below in section 3.
-
-2.3. Replies from Tor to the controller
-
- Reply = *(MidReplyLine / DataReplyLine) EndReplyLine
-
- MidReplyLine = "-" ReplyLine
- DataReplyLine = "+" ReplyLine Data
- EndReplyLine = SP ReplyLine
- ReplyLine = StatusCode [ SP ReplyText ] CRLF
- ReplyText = XXXX
- StatusCode = XXXX
-
- Specific replies are mentioned below in section 3, and described more fully
- in section 4.
-
-2.4. General-use tokens
-
- ; Identifiers for servers.
- ServerID = Nickname / Fingerprint
-
- Nickname = 1*19 NicknameChar
- NicknameChar = "a"-"z" / "A"-"Z" / "0" - "9"
- Fingerprint = "$" 40*HEXDIG
-
- ; A "=" indicates that the given nickname is canonical; a "~" indicates
- ; that the given nickname is not canonical.
- LongName = Fingerprint [ ( "=" / "~" ) Nickname ]
-
- ; How a controller tells Tor about a particular OR. There are four
- ; possible formats:
- ; $Digest -- The router whose identity key hashes to the given digest.
- ; This is the preferred way to refer to an OR.
- ; $Digest~Name -- The router whose identity key hashes to the given
- ; digest, but only if the router has the given nickname.
- ; $Digest=Name -- The router whose identity key hashes to the given
- ; digest, but only if the router is Named and has the given
- ; nickname.
- ; Name -- The Named router with the given nickname, or, if no such
- ; router exists, any router whose nickname matches the one given.
- ; This is not a safe way to refer to routers, since Named status
- ; could under some circumstances change over time.
- ServerSpec = LongName / Nickname
-
- ; Unique identifiers for streams or circuits. Currently, Tor only
- ; uses digits, but this may change
- StreamID = 1*16 IDChar
- CircuitID = 1*16 IDChar
- IDChar = ALPHA / DIGIT
-
- Address = ip4-address / ip6-address / hostname (XXXX Define these)
-
- ; A "Data" section is a sequence of octets concluded by the terminating
- ; sequence CRLF "." CRLF. The terminating sequence may not appear in the
- ; body of the data. Leading periods on lines in the data are escaped with
- ; an additional leading period as in RFC 2821 section 4.5.2.
- Data = *DataLine "." CRLF
- DataLine = CRLF / "." 1*LineItem CRLF / NonDotItem *LineItem CRLF
- LineItem = NonCR / 1*CR NonCRLF
- NonDotItem = NonDotCR / 1*CR NonCRLF
-
-3. Commands
-
- All commands and other keywords are case-insensitive.
-
-3.1. SETCONF
-
- Change the value of one or more configuration variables. The syntax is:
-
- "SETCONF" 1*(SP keyword ["=" String]) CRLF
-
- Tor behaves as though it had just read each of the key-value pairs
- from its configuration file. Keywords with no corresponding values have
- their configuration values reset to 0 or NULL (use RESETCONF if you want
- to set it back to its default). SETCONF is all-or-nothing: if there
- is an error in any of the configuration settings, Tor sets none of them.
-
- Tor responds with a "250 configuration values set" reply on success.
- If some of the listed keywords can't be found, Tor replies with a
- "552 Unrecognized option" message. Otherwise, Tor responds with a
- "513 syntax error in configuration values" reply on syntax error, or a
- "553 impossible configuration setting" reply on a semantic error.
-
- When a configuration option takes multiple values, or when multiple
- configuration keys form a context-sensitive group (see GETCONF below), then
- setting _any_ of the options in a SETCONF command is taken to reset all of
- the others. For example, if two ORBindAddress values are configured, and a
- SETCONF command arrives containing a single ORBindAddress value, the new
- command's value replaces the two old values.
-
-3.2. RESETCONF
-
- Remove all settings for a given configuration option entirely, assign
- its default value (if any), and then assign the String provided.
- Typically the String is left empty, to simply set an option back to
- its default. The syntax is:
-
- "RESETCONF" 1*(SP keyword ["=" String]) CRLF
-
- Otherwise it behaves like SETCONF above.
-
-3.3. GETCONF
-
- Request the value of a configuration variable. The syntax is:
-
- "GETCONF" 1*(SP keyword) CRLF
-
- If all of the listed keywords exist in the Tor configuration, Tor replies
- with a series of reply lines of the form:
- 250 keyword=value
- If any option is set to a 'default' value semantically different from an
- empty string, Tor may reply with a reply line of the form:
- 250 keyword
-
- If some of the listed keywords can't be found, Tor replies with a
- "552 unknown configuration keyword" message.
-
- If an option appears multiple times in the configuration, all of its
- key-value pairs are returned in order.
-
- Some options are context-sensitive, and depend on other options with
- different keywords. These cannot be fetched directly. Currently there
- is only one such option: clients should use the "HiddenServiceOptions"
- virtual keyword to get all HiddenServiceDir, HiddenServicePort,
- HiddenServiceNodes, and HiddenServiceExcludeNodes option settings.
-
-3.4. SETEVENTS
-
- Request the server to inform the client about interesting events. The
- syntax is:
-
- "SETEVENTS" [SP "EXTENDED"] *(SP EventCode) CRLF
-
- EventCode = "CIRC" / "STREAM" / "ORCONN" / "BW" / "DEBUG" /
- "INFO" / "NOTICE" / "WARN" / "ERR" / "NEWDESC" / "ADDRMAP" /
- "AUTHDIR_NEWDESCS" / "DESCCHANGED" / "STATUS_GENERAL" /
- "STATUS_CLIENT" / "STATUS_SERVER" / "GUARDS" / "NS"
-
- Any events *not* listed in the SETEVENTS line are turned off; thus, sending
- SETEVENTS with an empty body turns off all event reporting.
-
- The server responds with a "250 OK" reply on success, and a "552
- Unrecognized event" reply if one of the event codes isn't recognized. (On
- error, the list of active event codes isn't changed.)
-
- If the flag string "EXTENDED" is provided, Tor may provide extra
- information with events for this connection; see 4.1 for more information.
- NOTE: All events on a given connection will be provided in extended format,
- or none.
- NOTE: "EXTENDED" is only supported in Tor 0.1.1.9-alpha or later.
-
- Each event is described in more detail in Section 4.1.
-
-3.5. AUTHENTICATE
-
- Sent from the client to the server. The syntax is:
- "AUTHENTICATE" [ SP 1*HEXDIG / QuotedString ] CRLF
-
- The server responds with "250 OK" on success or "515 Bad authentication" if
- the authentication cookie is incorrect.
-
- The format of the 'cookie' is implementation-dependent; see 5.1 below for
- information on how the standard Tor implementation handles it.
-
- If Tor requires authentication and the controller has not yet sent an
- AUTHENTICATE message, Tor sends a "514 authentication required" reply to
- any other kind of message.
-
-3.6. SAVECONF
-
- Sent from the client to the server. The syntax is:
- "SAVECONF" CRLF
-
- Instructs the server to write out its config options into its torrc. Server
- returns "250 OK" if successful, or "551 Unable to write configuration
- to disk" if it can't write the file or some other error occurs.
-
-3.7. SIGNAL
-
- Sent from the client to the server. The syntax is:
-
- "SIGNAL" SP Signal CRLF
-
- Signal = "RELOAD" / "SHUTDOWN" / "DUMP" / "DEBUG" / "HALT" /
- "HUP" / "INT" / "USR1" / "USR2" / "TERM" / "NEWNYM" /
- "CLEARDNSCACHE"
-
- The meaning of the signals are:
-
- RELOAD -- Reload: reload config items, refetch directory. (like HUP)
- SHUTDOWN -- Controlled shutdown: if server is an OP, exit immediately.
- If it's an OR, close listeners and exit after 30 seconds.
- (like INT)
- DUMP -- Dump stats: log information about open connections and
- circuits. (like USR1)
- DEBUG -- Debug: switch all open logs to loglevel debug. (like USR2)
- HALT -- Immediate shutdown: clean up and exit now. (like TERM)
- CLEARDNSCACHE -- Forget the client-side cached IPs for all hostnames.
- NEWNYM -- Switch to clean circuits, so new application requests
- don't share any circuits with old ones. Also clears
- the client-side DNS cache.
-
- The server responds with "250 OK" if the signal is recognized (or simply
- closes the socket if it was asked to close immediately), or "552
- Unrecognized signal" if the signal is unrecognized.
-
-3.8. MAPADDRESS
-
- Sent from the client to the server. The syntax is:
-
- "MAPADDRESS" 1*(Address "=" Address SP) CRLF
-
- The first address in each pair is an "original" address; the second is a
- "replacement" address. The client sends this message to the server in
- order to tell it that future SOCKS requests for connections to the original
- address should be replaced with connections to the specified replacement
- address. If the addresses are well-formed, and the server is able to
- fulfill the request, the server replies with a 250 message:
- 250-OldAddress1=NewAddress1
- 250 OldAddress2=NewAddress2
-
- containing the source and destination addresses. If request is
- malformed, the server replies with "512 syntax error in command
- argument". If the server can't fulfill the request, it replies with
- "451 resource exhausted".
-
- The client may decline to provide a body for the original address, and
- instead send a special null address ("0.0.0.0" for IPv4, "::0" for IPv6, or
- "." for hostname), signifying that the server should choose the original
- address itself, and return that address in the reply. The server
- should ensure that it returns an element of address space that is unlikely
- to be in actual use. If there is already an address mapped to the
- destination address, the server may reuse that mapping.
-
- If the original address is already mapped to a different address, the old
- mapping is removed. If the original address and the destination address
- are the same, the server removes any mapping in place for the original
- address.
-
- Example:
- C: MAPADDRESS 0.0.0.0=tor.eff.org 1.2.3.4=tor.freehaven.net
- S: 250-127.192.10.10=tor.eff.org
- S: 250 1.2.3.4=tor.freehaven.net
-
- {Note: This feature is designed to be used to help Tor-ify applications
- that need to use SOCKS4 or hostname-less SOCKS5. There are three
- approaches to doing this:
- 1. Somehow make them use SOCKS4a or SOCKS5-with-hostnames instead.
- 2. Use tor-resolve (or another interface to Tor's resolve-over-SOCKS
- feature) to resolve the hostname remotely. This doesn't work
- with special addresses like x.onion or x.y.exit.
- 3. Use MAPADDRESS to map an IP address to the desired hostname, and then
- arrange to fool the application into thinking that the hostname
- has resolved to that IP.
- This functionality is designed to help implement the 3rd approach.}
-
- Mappings set by the controller last until the Tor process exits:
- they never expire. If the controller wants the mapping to last only
- a certain time, then it must explicitly un-map the address when that
- time has elapsed.
-
-3.9. GETINFO
-
- Sent from the client to the server. The syntax is as for GETCONF:
- "GETINFO" 1*(SP keyword) CRLF
- one or more NL-terminated strings. The server replies with an INFOVALUE
- message, or a 551 or 552 error.
-
- Unlike GETCONF, this message is used for data that are not stored in the Tor
- configuration file, and that may be longer than a single line. On success,
- one ReplyLine is sent for each requested value, followed by a final 250 OK
- ReplyLine. If a value fits on a single line, the format is:
- 250-keyword=value
- If a value must be split over multiple lines, the format is:
- 250+keyword=
- value
- .
- Recognized keys and their values include:
-
- "version" -- The version of the server's software, including the name
- of the software. (example: "Tor 0.0.9.4")
-
- "config-file" -- The location of Tor's configuration file ("torrc").
-
- ["exit-policy/prepend" -- The default exit policy lines that Tor will
- *prepend* to the ExitPolicy config option.
- -- Never implemented. Useful?]
-
- "exit-policy/default" -- The default exit policy lines that Tor will
- *append* to the ExitPolicy config option.
-
- "desc/id/<OR identity>" or "desc/name/<OR nickname>" -- the latest
- server descriptor for a given OR, NUL-terminated.
-
- "ns/id/<OR identity>" or "ns/name/<OR nickname>" -- the latest network
- status info for a given OR. Network status info is as given in
- dir-spec.txt, and reflects the current beliefs of this Tor about the
- router in question. Like directory clients, controllers MUST
- tolerate unrecognized flags and lines. The published date and
- descriptor digest are those believed to be best by this Tor,
- not necessarily those for a descriptor that Tor currently has.
- [First implemented in 0.1.2.3-alpha.]
-
- "ns/all" -- Network status info for all ORs we have an opinion about,
- joined by newlines. [First implemented in 0.1.2.3-alpha.]
-
- "desc/all-recent" -- the latest server descriptor for every router that
- Tor knows about.
-
- "network-status" -- a space-separated list of all known OR identities.
- This is in the same format as the router-status line in directories;
- see dir-spec-v1.txt section 3 for details. (If VERBOSE_NAMES is
- enabled, the output will not conform to dir-spec-v1.txt; instead, the
- result will be a space-separated list of LongName, each preceded by a
- "!" if it is believed to be not running.)
-
- "addr-mappings/all"
- "addr-mappings/config"
- "addr-mappings/cache"
- "addr-mappings/control" -- a space-separated list of address
- mappings, each in the form of "from-address=to-address".
- The 'config' key returns those address mappings set in the
- configuration; the 'cache' key returns the mappings in the
- client-side DNS cache; the 'control' key returns the mappings set
- via the control interface; the 'all' target returns the mappings
- set through any mechanism.
-
- "address" -- the best guess at our external IP address. If we
- have no guess, return a 551 error. (Added in 0.1.2.2-alpha)
-
- "fingerprint" -- the contents of the fingerprint file that Tor
- writes as a server, or a 551 if we're not a server currently.
- (Added in 0.1.2.3-alpha)
-
- "circuit-status"
- A series of lines as for a circuit status event. Each line is of
- the form:
- CircuitID SP CircStatus [SP Path] CRLF
-
- "stream-status"
- A series of lines as for a stream status event. Each is of the form:
- StreamID SP StreamStatus SP CircID SP Target CRLF
-
- "orconn-status"
- A series of lines as for an OR connection status event. Each is of the
- form:
- ServerID SP ORStatus CRLF
-
- "entry-guards"
- A series of lines listing the currently chosen entry guards, if any.
- Each is of the form:
- ServerID SP (Status-with-time / Status) CRLF
-
- Status-with-time = ("down" / "unlisted") SP ISOTime
- Status = ("up" / "never-connected")
-
- [From 0.1.1.4-alpha to 0.1.1.10-alpha, this was called "helper-nodes".
- Tor still supports calling it that for now, but support will be
- removed in 0.1.3.x.]
-
- "accounting/enabled"
- "accounting/hibernating"
- "accounting/bytes"
- "accounting/bytes-left"
- "accounting/interval-start"
- "accounting/interval-wake"
- "accounting/interval-end"
- Information about accounting status. If accounting is enabled,
- "enabled" is 1; otherwise it is 0. The "hibernating" field is "hard"
- if we are accepting no data; "soft" if we're accepting no new
- connections, and "awake" if we're not hibernating at all. The "bytes"
- and "bytes-left" fields contain (read-bytes SP write-bytes), for the
- start and the rest of the interval respectively. The 'interval-start'
- and 'interval-end' fields are the borders of the current interval; the
- 'interval-wake' field is the time within the current interval (if any)
- where we plan[ned] to start being active.
-
- "config/names"
- A series of lines listing the available configuration options. Each is
- of the form:
- OptionName SP OptionType [ SP Documentation ] CRLF
- OptionName = Keyword
- OptionType = "Integer" / "TimeInterval" / "DataSize" / "Float" /
- "Boolean" / "Time" / "CommaList" / "Dependant" / "Virtual" /
- "String" / "LineList"
- Documentation = Text
-
- "info/names"
- A series of lines listing the available GETINFO options. Each is of
- one of these forms:
- OptionName SP Documentation CRLF
- OptionPrefix SP Documentation CRLF
- OptionPrefix = OptionName "/*"
-
- "events/names"
- A space-separated list of all the events supported by this version of
- Tor's SETEVENTS.
-
- "features/names"
- A space-separated list of all the events supported by this version of
- Tor's USEFEATURE.
-
- "next-circuit/IP:port"
- XXX todo.
-
- "dir/status/authority"
- "dir/status/fp/<F>"
- "dir/status/fp/<F1>+<F2>+<F3>"
- "dir/status/all"
- "dir/server/fp/<F>"
- "dir/server/fp/<F1>+<F2>+<F3>"
- "dir/server/d/<D>"
- "dir/server/d/<D1>+<D2>+<D3>"
- "dir/server/authority"
- "dir/server/all"
- A series of lines listing directory contents, provided according to the
- specification for the URLs listed in Section 4.4 of dir-spec.txt. Note
- that Tor MUST NOT provide private information, such as descriptors for
- routers not marked as general-purpose. When asked for 'authority'
- information for which this Tor is not authoritative, Tor replies with
- an empty string.
-
- "status/circuit-established"
- "status/..."
- These provide the current internal Tor values for various Tor
- states. See Section 4.1.10 for explanations. (Only a few of the
- status events are available as getinfo's currently. Let us know if
- you want more exposed.)
-
- Examples:
- C: GETINFO version desc/name/moria1
- S: 250+desc/name/moria=
- S: [Descriptor for moria]
- S: .
- S: 250-version=Tor 0.1.1.0-alpha-cvs
- S: 250 OK
-
-3.10. EXTENDCIRCUIT
-
- Sent from the client to the server. The format is:
- "EXTENDCIRCUIT" SP CircuitID SP
- ServerSpec *("," ServerSpec) SP
- ("purpose=" Purpose) CRLF
-
- This request takes one of two forms: either the CircuitID is zero, in
- which case it is a request for the server to build a new circuit according
- to the specified path, or the CircuitID is nonzero, in which case it is a
- request for the server to extend an existing circuit with that ID according
- to the specified path.
-
- If CircuitID is 0 and "purpose=" is specified, then the circuit's
- purpose is set. Two choices are recognized: "general" and
- "controller". If not specified, circuits are created as "general".
-
- If the request is successful, the server sends a reply containing a
- message body consisting of the CircuitID of the (maybe newly created)
- circuit. The syntax is "250" SP "EXTENDED" SP CircuitID CRLF.
-
-3.11. SETCIRCUITPURPOSE
-
- Sent from the client to the server. The format is:
- "SETCIRCUITPURPOSE" SP CircuitID SP Purpose CRLF
-
- This changes the circuit's purpose. See EXTENDCIRCUIT above for details.
-
-3.12. SETROUTERPURPOSE
-
- Sent from the client to the server. The format is:
- "SETROUTERPURPOSE" SP NicknameOrKey SP Purpose CRLF
-
- This changes the descriptor's purpose. See +POSTDESCRIPTOR below
- for details.
-
-3.13. ATTACHSTREAM
-
- Sent from the client to the server. The syntax is:
- "ATTACHSTREAM" SP StreamID SP CircuitID CRLF
-
- This message informs the server that the specified stream should be
- associated with the specified circuit. Each stream may be associated with
- at most one circuit, and multiple streams may share the same circuit.
- Streams can only be attached to completed circuits (that is, circuits that
- have sent a circuit status 'BUILT' event or are listed as built in a
- GETINFO circuit-status request).
-
- If the circuit ID is 0, responsibility for attaching the given stream is
- returned to Tor.
-
- Tor responds with "250 OK" if it can attach the stream, 552 if the circuit
- or stream didn't exist, or 551 if the stream couldn't be attached for
- another reason.
-
- {Implementation note: Tor will close unattached streams by itself,
- roughly two minutes after they are born. Let the developers know if
- that turns out to be a problem.}
-
- {Implementation note: By default, Tor automatically attaches streams to
- circuits itself, unless the configuration variable
- "__LeaveStreamsUnattached" is set to "1". Attempting to attach streams
- via TC when "__LeaveStreamsUnattached" is false may cause a race between
- Tor and the controller, as both attempt to attach streams to circuits.}
-
- {Implementation note: You can try to attachstream to a stream that
- has already sent a connect or resolve request but hasn't succeeded
- yet, in which case Tor will detach the stream from its current circuit
- before proceeding with the new attach request.}
-
-3.14. POSTDESCRIPTOR
-
- Sent from the client to the server. The syntax is:
- "+POSTDESCRIPTOR" ("purpose=" Purpose) CRLF Descriptor CRLF "." CRLF
-
- This message informs the server about a new descriptor. If Purpose is
- specified, it must be either "general" or "controller", else we
- return a 552 error.
-
- The descriptor, when parsed, must contain a number of well-specified
- fields, including fields for its nickname and identity.
-
- If there is an error in parsing the descriptor, the server must send a "554
- Invalid descriptor" reply. If the descriptor is well-formed but the server
- chooses not to add it, it must reply with a 251 message whose body explains
- why the server was not added. If the descriptor is added, Tor replies with
- "250 OK".
-
-3.15. REDIRECTSTREAM
-
- Sent from the client to the server. The syntax is:
- "REDIRECTSTREAM" SP StreamID SP Address (SP Port) CRLF
-
- Tells the server to change the exit address on the specified stream. If
- Port is specified, changes the destination port as well. No remapping
- is performed on the new provided address.
-
- To be sure that the modified address will be used, this event must be sent
- after a new stream event is received, and before attaching this stream to
- a circuit.
-
- Tor replies with "250 OK" on success.
-
-3.16. CLOSESTREAM
-
- Sent from the client to the server. The syntax is:
-
- "CLOSESTREAM" SP StreamID SP Reason *(SP Flag) CRLF
-
- Tells the server to close the specified stream. The reason should be one
- of the Tor RELAY_END reasons given in tor-spec.txt, as a decimal. Flags is
- not used currently; Tor servers SHOULD ignore unrecognized flags. Tor may
- hold the stream open for a while to flush any data that is pending.
-
- Tor replies with "250 OK" on success, or a 512 if there aren't enough
- arguments, or a 552 if it doesn't recognize the StreamID or reason.
-
-3.17. CLOSECIRCUIT
-
- The syntax is:
- CLOSECIRCUIT SP CircuitID *(SP Flag) CRLF
- Flag = "IfUnused"
-
- Tells the server to close the specified circuit. If "IfUnused" is
- provided, do not close the circuit unless it is unused.
-
- Other flags may be defined in the future; Tor SHOULD ignore unrecognized
- flags.
-
- Tor replies with "250 OK" on success, or a 512 if there aren't enough
- arguments, or a 552 if it doesn't recognize the CircuitID.
-
-3.18. QUIT
-
- Tells the server to hang up on this controller connection. This command
- can be used before authenticating.
-
-3.19. USEFEATURE
-
- The syntax is:
-
- "USEFEATURE" *(SP FeatureName) CRLF
- FeatureName = 1*(ALPHA / DIGIT / "_" / "-")
-
- Sometimes extensions to the controller protocol break compatibility with
- older controllers. In this case, whenever possible, the extensions are
- first included in Tor disabled by default, and only enabled on a given
- controller connection when the "USEFEATURE" command is given. Once a
- "USEFEATURE" command is given, it applies to all subsequent interactions on
- the same connection; to disable an enabled feature, a new controller
- connection must be opened.
-
- This is a forward-compatibility mechanism; each feature will eventually
- become a regular part of the control protocol in some future version of Tor.
- Tor will ignore a request to use any feature that is already on by default.
- Tor will give a "552" error if any requested feature is not recognized.
-
- Feature names are case-insensitive.
-
- EXTENDED_EVENTS
-
- Same as passing 'EXTENDED' to SETEVENTS; this is the preferred way to
- request the extended event syntax.
-
- This will not be always-enabled until at least XXX (or, at least two
- stable releases after XXX, the release where it was first used for
- anything.)
-
- VERBOSE_NAMES
-
- Instead of ServerID as specified above, the controller should
- identify ORs by LongName in events and GETINFO results. This format is
- strictly more informative: rather than including Nickname for
- known Named routers and Fingerprint for unknown or unNamed routers, the
- LongName format includes a Fingerprint, an indication of Named status,
- and a Nickname (if one is known).
-
- This will not be always-enabled until at least 0.1.4.x (or at least two
- stable releases after 0.1.2.2-alpha, the release where it was first
- available.)
-
-4. Replies
-
- Reply codes follow the same 3-character format as used by SMTP, with the
- first character defining a status, the second character defining a
- subsystem, and the third designating fine-grained information.
-
- The TC protocol currently uses the following first characters:
-
- 2yz Positive Completion Reply
- The command was successful; a new request can be started.
-
- 4yz Temporary Negative Completion reply
- The command was unsuccessful but might be reattempted later.
-
- 5yz Permanent Negative Completion Reply
- The command was unsuccessful; the client should not try exactly
- that sequence of commands again.
-
- 6yz Asynchronous Reply
- Sent out-of-order in response to an earlier SETEVENTS command.
-
- The following second characters are used:
-
- x0z Syntax
- Sent in response to ill-formed or nonsensical commands.
-
- x1z Protocol
- Refers to operations of the Tor Control protocol.
-
- x5z Tor
- Refers to actual operations of Tor system.
-
- The following codes are defined:
-
- 250 OK
- 251 Operation was unnecessary
- [Tor has declined to perform the operation, but no harm was done.]
-
- 451 Resource exhausted
-
- 500 Syntax error: protocol
-
- 510 Unrecognized command
- 511 Unimplemented command
- 512 Syntax error in command argument
- 513 Unrecognized command argument
- 514 Authentication required
- 515 Bad authentication
-
- 550 Unspecified Tor error
-
- 551 Internal error
- [Something went wrong inside Tor, so that the client's
- request couldn't be fulfilled.]
-
- 552 Unrecognized entity
- [A configuration key, a stream ID, circuit ID, event,
- mentioned in the command did not actually exist.]
-
- 553 Invalid configuration value
- [The client tried to set a configuration option to an
- incorrect, ill-formed, or impossible value.]
-
- 554 Invalid descriptor
-
- 555 Unmanaged entity
-
- 650 Asynchronous event notification
-
- Unless specified to have specific contents, the human-readable messages
- in error replies should not be relied upon to match those in this document.
-
-4.1. Asynchronous events
-
- These replies can be sent after a corresponding SETEVENTS command has been
- received. They will not be interleaved with other Reply elements, but they
- can appear between a command and its corresponding reply. For example,
- this sequence is possible:
-
- C: SETEVENTS CIRC
- S: 250 OK
- C: GETCONF SOCKSPORT ORPORT
- S: 650 CIRC 1000 EXTENDED moria1,moria2
- S: 250-SOCKSPORT=9050
- S: 250 ORPORT=0
-
- But this sequence is disallowed:
- C: SETEVENTS CIRC
- S: 250 OK
- C: GETCONF SOCKSPORT ORPORT
- S: 250-SOCKSPORT=9050
- S: 650 CIRC 1000 EXTENDED moria1,moria2
- S: 250 ORPORT=0
-
- Clients MUST tolerate more arguments in an asynchonous reply than
- expected, and MUST tolerate more lines in an asynchronous reply than
- expected. For instance, a client that expects a CIRC message like:
- 650 CIRC 1000 EXTENDED moria1,moria2
- must tolerate:
- 650-CIRC 1000 EXTENDED moria1,moria2 0xBEEF
- 650-EXTRAMAGIC=99
- 650 ANONYMITY=high
-
- If clients ask for extended events, then each event line as specified below
- will be followed by additional extensions. Additional lines will be of the
- form
- "650" ("-"/" ") KEYWORD ["=" ARGUMENTS] CRLF
- Additional arguments will be of the form
- SP KEYWORD ["=" ( QuotedString / * NonSpDquote ) ]
- Such clients MUST tolerate lines with keywords they do not recognize.
-
-4.1.1. Circuit status changed
-
- The syntax is:
-
- "650" SP "CIRC" SP CircuitID SP CircStatus [SP Path]
- [SP "REASON=" Reason [SP "REMOTE_REASON=" Reason]] CRLF
-
- CircStatus =
- "LAUNCHED" / ; circuit ID assigned to new circuit
- "BUILT" / ; all hops finished, can now accept streams
- "EXTENDED" / ; one more hop has been completed
- "FAILED" / ; circuit closed (was not built)
- "CLOSED" ; circuit closed (was built)
-
- Path = ServerID *("," ServerID)
-
- Reason = "NONE" / "TORPROTOCOL" / "INTERNAL" / "REQUESTED" /
- "HIBERNATING" / "RESOURCELIMIT" / "CONNECTFAILED" /
- "OR_IDENTITY" / "OR_CONN_CLOSED" / "TIMEOUT" /
- "FINISHED" / "DESTROYED" / "NOPATH" / "NOSUCHSERVICE"
-
- The path is provided only when the circuit has been extended at least one
- hop.
-
- The "REASON" field is provided only for FAILED and CLOSED events, and only
- if extended events are enabled (see 3.19). Clients MUST accept reasons
- not listed above. Reasons are as given in tor-spec.txt, except for:
-
- NOPATH (Not enough nodes to make circuit)
-
- The "REMOTE_REASON" field is provided only when we receive a DESTROY or
- TRUNCATE cell, and only if extended events are enabled. It contains the
- actual reason given by the remote OR for closing the circuit. Clients MUST
- accept reasons not listed above. Reasons are as listed in tor-spec.txt.
-
-4.1.2. Stream status changed
-
- The syntax is:
-
- "650" SP "STREAM" SP StreamID SP StreamStatus SP CircID SP Target
- [SP "REASON=" Reason [ SP "REMOTE_REASON=" Reason ]] CRLF
-
- StreamStatus =
- "NEW" / ; New request to connect
- "NEWRESOLVE" / ; New request to resolve an address
- "SENTCONNECT" / ; Sent a connect cell along a circuit
- "SENTRESOLVE" / ; Sent a resolve cell along a circuit
- "SUCCEEDED" / ; Received a reply; stream established
- "FAILED" / ; Stream failed and not retriable
- "CLOSED" / ; Stream closed
- "DETACHED" ; Detached from circuit; still retriable
-
- Target = Address ":" Port
-
- The circuit ID designates which circuit this stream is attached to. If
- the stream is unattached, the circuit ID "0" is given.
-
- Reason = "MISC" / "RESOLVEFAILED" / "CONNECTREFUSED" /
- "EXITPOLICY" / "DESTROY" / "DONE" / "TIMEOUT" /
- "HIBERNATING" / "INTERNAL"/ "RESOURCELIMIT" /
- "CONNRESET" / "TORPROTOCOL" / "NOTDIRECTORY" / "END"
-
- The "REASON" field is provided only for FAILED, CLOSED, and DETACHED
- events, and only if extended events are enabled (see 3.19). Clients MUST
- accept reasons not listed above. Reasons are as given in tor-spec.txt,
- except for:
-
- END (We received a RELAY_END cell from the other side of thise
- stream.)
-
- The "REMOTE_REASON" field is provided only when we receive a RELAY_END
- cell, and only if extended events are enabled. It contains the actual
- reason given by the remote OR for closing the stream. Clients MUST accept
- reasons not listed above. Reasons are as listed in tor-spec.txt.
-
-4.1.3. OR Connection status changed
-
- The syntax is:
- "650" SP "ORCONN" SP (ServerID / Target) SP ORStatus [ SP "REASON="
- Reason ] [ SP "NCIRCS=" NumCircuits ]
-
- ORStatus = "NEW" / "LAUNCHED" / "CONNECTED" / "FAILED" / "CLOSED"
-
- NEW is for incoming connections, and LAUNCHED is for outgoing
- connections. CONNECTED means the TLS handshake has finished (in
- either direction). FAILED means a connection is being closed that
- hasn't finished its handshake, and CLOSED is for connections that
- have handshaked.
-
- A ServerID is specified unless it's a NEW connection, in which
- case we don't know what server it is yet, so we use Address:Port.
-
- If extended events are enabled (see 3.19), optional reason and
- circuit counting information is provided for CLOSED and FAILED
- events.
-
- Reason = "MISC" / "DONE" / "CONNECTREFUSED" /
- "IDENTITY" / "CONNECTRESET" / "TIMEOUT" / "NOROUTE" /
- "IOERROR"
-
- NumCircuits counts both established and pending circuits.
-
-
-4.1.4. Bandwidth used in the last second
-
- The syntax is:
- "650" SP "BW" SP BytesRead SP BytesWritten *(SP Type "=" Num)
- BytesRead = 1*DIGIT
- BytesWritten = 1*DIGIT
- Type = "DIR" / "OR" / "EXIT" / "APP" / ...
- Num = 1*DIGIT
-
- BytesRead and BytesWritten are the totals. In Tor 0.1.x.y-alpha
- and later, we also include a breakdown of the connection types
- that used bandwidth this second (not implemented yet).
-
-4.1.5. Log messages
-
- The syntax is:
- "650" SP Severity SP ReplyText
- or
- "650+" Severity CRLF Data
-
- Severity = "DEBUG" / "INFO" / "NOTICE" / "WARN"/ "ERR"
-
-4.1.6. New descriptors available
-
- Syntax:
- "650" SP "NEWDESC" 1*(SP ServerID)
-
-4.1.7. New Address mapping
-
- Syntax:
- "650" SP "ADDRMAP" SP Address SP Address SP Expiry
- Expiry = DQUOTE ISOTime DQUOTE / "NEVER"
-
- Expiry is expressed as the local time (rather than GMT).
-
- [XXX We should rename this to ADDRESSMAP. -RD]
-
- [FFF We should add a SOURCE=%s argument for extended events,
- which specifies what exit node told us this addressmap. -RD]
-
-4.1.8. Descriptors uploaded to us in our role as authoritative dirserver
-
- Syntax:
- "650" "+" "AUTHDIR_NEWDESCS" CRLF Action CRLF Message CRLF
- Descriptor CRLF "." CRLF
- Action = "ACCEPTED" / "DROPPED" / "REJECTED"
- Message = Text
-
-4.1.9. Our descriptor changed
-
- Syntax:
- "650" SP "DESCCHANGED"
-
- [First added in 0.1.2.2-alpha.]
-
-4.1.10. Status events
-
- Status events (STATUS_GENERAL, STATUS_CLIENT, and STATUS_SERVER) are sent
- based on occurrences in the Tor process pertaining to the general state of
- the program. Generally, they correspond to log messages of severity Notice
- or higher. They differ from log messages in that their format is a
- specified interface.
-
- Syntax:
- "650" SP StatusType SP StatusSeverity SP StatusAction
- [SP StatusArguments] CRLF
-
- StatusType = "STATUS_GENERAL" / "STATUS_CLIENT" / "STATUS_SERVER"
- StatusSeverity = "NOTICE" / "WARN" / "ERR"
- StatusAction = 1*ALPHA
- StatusArguments = StatusArgument *(SP StatusArgument)
- StatusArgument = StatusKeyword '=' StatusValue
- StatusKeyword = 1*(ALNUM / "_")
- StatusValue = 1*(ALNUM / '_') / QuotedString
-
- Action is a string, and Arguments is a series of keyword=value
- pairs on the same line. Values may be space-terminated strings,
- or quoted strings.
-
- These events are always produced with EXTENDED_EVENTS and
- VERBOSE_NAMES; see the explanations in the USEFEATURE section
- for details.
-
- Controllers MUST tolerate unrecognized actions, MUST tolerate
- unrecognized arguments, MUST tolerate missing arguments, and MUST
- tolerate arguments that arrive in any order.
-
- Each event description below is accompanied by a recommendation for
- controllers. These recommendations are suggestions only; no controller
- is required to implement them.
-
- Actions for STATUS_GENERAL events can be as follows:
-
- CLOCK_JUMPED
- "TIME=NUM"
- Tor spent enough time without CPU cycles that it has closed all
- its circuits and will establish them anew. This typically
- happens when a laptop goes to sleep and then wakes up again. It
- also happens when the system is swapping so heavily that Tor is
- starving. The "time" argument specifies the number of seconds Tor
- thinks it was unconscious for (or alternatively, the number of
- seconds it went back in time).
-
- This status event is sent as NOTICE severity normally, but WARN
- severity if Tor is acting as a server currently.
-
- {Recommendation for controller: ignore it, since we don't really
- know what the user should do anyway. Hm.}
-
- DANGEROUS_VERSION
- "CURRENT=version"
- "REASON=NEW/OLD/UNRECOMMENDED"
- "RECOMMENDED=\"version, version, ...\""
- Tor has found that directory servers don't recommend its version of
- the Tor software. RECOMMENDED is a comma-and-space-separated string
- of Tor versions that are recommended. REASON is NEW if this version
- of Tor is newer than any recommended version, OLD if this version of
- Tor is older than any recommended version, and UNRECOMMENDED if
- some recommended versions of Tor are newer and some are old than this
- version.
-
- {Controllers may want to suggest that the user upgrade OLD or
- UNRECOMMENDED versions. NEW versions may be known-insecure, or may
- simply be development versions.}
-
- TOO_MANY_CONNECTIONS
- "CURRENT=NUM"
- Tor has reached its ulimit -n or whatever the native limit is on file
- descriptors or sockets. CURRENT is the number of sockets Tor
- currently has open. The user should really do something about
- this. The "current" argument shows the number of connections currently
- open.
-
- {Controllers may recommend that the user increase the limit, or
- increase it for them. Recommendations should be phrased in an
- OS-appropriate way and automated when possible.}
-
- BUG
- "REASON=STRING"
- Tor has encountered a situation that its developers never expected,
- and the developers would like to learn that it happened. Perhaps
- the controller can explain this to the user and encourage her to
- file a bug report?
-
- {Controllers should log bugs, but shouldn't annoy the user in case a
- bug appears frequently.}
-
- CLOCK_SKEWED
- SKEW="+" / "-" SECONDS
- SOURCE="DIRSERV:IP:Port" / "NETWORKSTATUS:IP:PORT"
- If "SKEW" is present, it's an estimate of how far we are from the
- time declared in the source. If the source is a DIRSERV, we got
- the current time from a connection to a dirserver. If the source is
- a NETWORKSTATUS, we decided we're skewed because we got a
- networkstatus from far in the future.
-
- {Controllers may want to warn the user if the skew is high, or if
- multiple skew messages appear at severity WARN. Controllers
- shouldn't blindly adjust the clock, since the more accurate source
- of skew info (DIRSERV) is currently unauthenticated.}
-
- BAD_LIBEVENT
- "METHOD=" libevent method
- "VERSION=" libevent version
- "BADNESS=" "BROKEN" / "BUGGY" / "SLOW"
- "RECOVERED=" "NO" / "YES"
- Tor knows about bugs in using the configured event method in this
- version of libevent. "BROKEN" libevents won't work at all;
- "BUGGY" libevents might work okay; "SLOW" libevents will work
- fine, but not quickly. If "RECOVERED" is YES, Tor managed to
- switch to a more reliable (but probably slower!) libevent method.
-
- {Controllers may want to warn the user if this event occurs, though
- generally it's the fault of whoever built the Tor binary and there's
- not much the user can do besides upgrade libevent or upgrade the
- binary.}
-
- DIR_ALL_UNREACHABLE
- Tor believes that none of the known directory servers are
- reachable -- this is most likely because the local network is
- down or otherwise not working, and might help to explain for the
- user why Tor appears to be broken.
-
- {Controllers may want to warn the user if this event occurs; further
- action is generally not possible.}
-
- Actions for STATUS_CLIENT events can be as follows:
-
- ENOUGH_DIR_INFO
- Tor now knows enough network-status documents and enough server
- descriptors that it's going to start trying to build circuits now.
-
- {Controllers may want to use this event to decide when to indicate
- progress to their users, but should not interrupt the user's browsing
- to tell them so.}
-
- NOT_ENOUGH_DIR_INFO
- We discarded expired statuses and router descriptors to fall
- below the desired threshold of directory information. We won't
- try to build any circuits until ENOUGH_DIR_INFO occurs again.
-
- {Controllers may want to use this event to decide when to indicate
- progress to their users, but should not interrupt the user's browsing
- to tell them so.}
-
- CIRCUIT_ESTABLISHED
- Tor is able to establish circuits for client use. This event will
- only be sent if we just built a circuit that changed our mind --
- that is, prior to this event we didn't know whether we could
- establish circuits.
-
- {Suggested use: controllers can notify their users that Tor is
- ready for use as a client once they see this status event. [Perhaps
- controllers should also have a timeout if too much time passes and
- this event hasn't arrived, to give tips on how to troubleshoot.
- On the other hand, hopefully Tor will send further status events
- if it can identify the problem.]}
-
- CIRCUIT_NOT_ESTABLISHED
- "REASON=" "EXTERNAL_ADDRESS" / "DIR_ALL_UNREACHABLE" / "CLOCK_JUMPED"
- We are no longer confident that we can build circuits. The "reason"
- keyword provides an explanation: which other status event type caused
- our lack of confidence.
-
- {Controllers may want to use this event to decide when to indicate
- progress to their users, but should not interrupt the user's browsing
- to do so.}
- [Note: only REASON=CLOCK_JUMPED is implemented currently.]
-
- DANGEROUS_SOCKS
- "PROTOCOL=SOCKS4/SOCKS5"
- "ADDRESS=IP:port"
- A connection was made to Tor's SOCKS port using one of the SOCKS
- approaches that doesn't support hostnames -- only raw IP addresses.
- If the client application got this address from gethostbyname(),
- it may be leaking target addresses via DNS.
-
- {Controllers should warn their users when this occurs, unless they
- happen to know that the application using Tor is in fact doing so
- correctly (e.g., because it is part of a distributed bundle).}
-
- SOCKS_UNKNOWN_PROTOCOL
- "DATA=string"
- A connection was made to Tor's SOCKS port that tried to use it
- for something other than the SOCKS protocol. Perhaps the user is
- using Tor as an HTTP proxy? The DATA is the first few characters
- sent to Tor on the SOCKS port.
-
- {Controllers may want to warn their users when this occurs: it
- indicates a misconfigured application.}
-
- SOCKS_BAD_HOSTNAME
- "HOSTNAME=QuotedString"
- Some application gave us a funny-looking hostname. Perhaps
- it is broken? In any case it won't work with Tor and the user
- should know.
-
- {Controllers may want to warn their users when this occurs: it
- usually indicates a misconfigured application.}
-
- Actions for STATUS_SERVER can be as follows:
-
- EXTERNAL_ADDRESS
- "ADDRESS=IP"
- "HOSTNAME=NAME"
- "METHOD=CONFIGURED/DIRSERV/RESOLVED/INTERFACE/GETHOSTNAME"
- Our best idea for our externally visible IP has changed to 'IP'.
- If 'HOSTNAME' is present, we got the new IP by resolving 'NAME'. If the
- method is 'CONFIGURED', the IP was given verbatim as a configuration
- option. If the method is 'RESOLVED', we resolved the Address
- configuration option to get the IP. If the method is 'GETHOSTNAME',
- we resolved our hostname to get the IP. If the method is 'INTERFACE',
- we got the address of one of our network interfaces to get the IP. If
- the method is 'DIRSERV', a directory server told us a guess for what
- our IP might be.
-
- {Controllers may want to record this info and display it to the user.}
-
- CHECKING_REACHABILITY
- "ORADDRESS=IP:port"
- "DIRADDRESS=IP:port"
- We're going to start testing the reachability of our external OR port
- or directory port.
-
- {This event could effect the controller's idea of server status, but
- the controller should not interrupt the user to tell them so.}
-
- REACHABILITY_SUCCEEDED
- "ORADDRESS=IP:port"
- "DIRADDRESS=IP:port"
- We successfully verified the reachability of our external OR port or
- directory port.
-
- {This event could effect the controller's idea of server status, but
- the controller should not interrupt the user to tell them so.}
-
- GOOD_SERVER_DESCRIPTOR
- We successfully uploaded our server descriptor to each of the
- directory authorities, with no complaints.
-
- {This event could effect the controller's idea of server status, but
- the controller should not interrupt the user to tell them so.}
-
- NAMESERVER_STATUS
- "NS=addr"
- "STATUS=" "UP" / "DOWN"
- "ERR=" message
- One of our nameservers has changed status.
- // actually notice
-
- {This event could effect the controller's idea of server status, but
- the controller should not interrupt the user to tell them so.}
-
- NAMESERVER_ALL_DOWN
- All of our nameservers have gone down.
-
- {This is a problem; if it happens often without the nameservers
- coming up again, the user needs to configure more or better
- nameservers.}
-
- DNS_HIJACKED
- Our DNS provider is providing an address when it should be saying
- "NOTFOUND"; Tor will treat the address as a synonym for "NOTFOUND".
-
- {This is an annoyance; controllers may want to tell admins that their
- DNS provider is not to be trusted.}
-
- DNS_USELESS
- Our DNS provider is giving a hijacked address instead of well-known
- websites; Tor will not try to be an exit node.
-
- {Controllers could warn the admin if the server is running as an
- exit server: the admin needs to configure a good DNS server.
- Alternatively, this happens a lot in some restrictive environments
- (hotels, universities, coffeeshops) when the user hasn't registered.}
-
- BAD_SERVER_DESCRIPTOR
- "DIRAUTH=addr:port"
- "REASON=string"
- A directory authority rejected our descriptor. Possible reasons
- include malformed descriptors, incorrect keys, highly skewed clocks,
- and so on.
-
- {Controllers should warn the admin, and try to cope if they can.}
-
- ACCEPTED_SERVER_DESCRIPTOR
- "DIRAUTH=addr:port"
- A single directory authority accepted our descriptor.
- // actually notice
-
- {This event could effect the controller's idea of server status, but
- the controller should not interrupt the user to tell them so.}
-
- REACHABILITY_FAILED
- "ORADDRESS=IP:port"
- "DIRADDRESS=IP:port"
- We failed to connect to our external OR port or directory port
- successfully.
-
- {This event could effect the controller's idea of server status. The
- controller should warn the admin and suggest reasonable steps to take.}
-
-4.1.11. Our set of guard nodes has changed
-
- Syntax:
- "650" SP "GUARDS" SP Type SP Name SP Status ... CRLF
- Type = "ENTRY"
- Name = The (possibly verbose) nickname of the guard affected.
- Status = "NEW" | "UP" | "DOWN" | "BAD" | "GOOD" | "DROPPED"
-
- [explain states. XXX]
-
-4.1.12. Network status has changed
-
- Syntax:
- "650" "+" "NS" CRLF 1*NetworkStatus "." CRLF
-
- [First added in 0.1.2.3-alpha]
-
-5. Implementation notes
-
-5.1. Authentication
-
- By default, the current Tor implementation trusts all local users.
-
- If the 'CookieAuthentication' option is true, Tor writes a "magic cookie"
- file named "control_auth_cookie" into its data directory. To authenticate,
- the controller must send the contents of this file, encoded in hexadecimal.
-
- If the 'HashedControlPassword' option is set, it must contain the salted
- hash of a secret password. The salted hash is computed according to the
- S2K algorithm in RFC 2440 (OpenPGP), and prefixed with the s2k specifier.
- This is then encoded in hexadecimal, prefixed by the indicator sequence
- "16:". Thus, for example, the password 'foo' could encode to:
- 16:660537E3E1CD49996044A3BF558097A981F539FEA2F9DA662B4626C1C2
- ++++++++++++++++**^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- salt hashed value
- indicator
- You can generate the salt of a password by calling
- 'tor --hash-password <password>'
- or by using the example code in the Python and Java controller libraries.
- To authenticate under this scheme, the controller sends Tor the original
- secret that was used to generate the password.
-
-5.2. Don't let the buffer get too big.
-
- If you ask for lots of events, and 16MB of them queue up on the buffer,
- the Tor process will close the socket.
-
-5.3. Backward compatibility with v0 control protocol.
-
- For backward compatibility with the "version 0" control protocol, Tor checks
- whether the third octet the first command is zero. If it is, Tor
- assumes that version 0 is in use. This feature is deprecated, and will be
- removed in the 0.1.3.x Tor development series.
-
- In order to detect which version of the protocol is supported controllers
- should send the sequence [00 00 0D 0A]. This is a valid and unrecognized
- command in both protocol versions, and implementations can detect which
- error they have received.
-
Deleted: tor/trunk/doc/dir-spec.txt
===================================================================
--- tor/trunk/doc/dir-spec.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/dir-spec.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,873 +0,0 @@
-$Id$
-
- Tor directory protocol, version 2
-
-0. Scope and preliminaries
-
- This directory protocol is used by Tor version 0.1.1.x and later. See
- dir-spec-v1.txt for information on earlier versions.
-
-0.1. Goals and motivation
-
- There were several problems with the way Tor handles directory information
- in version 0.1.0.x and earlier. Here are the problems we try to fix with
- this new design, already implemented in 0.1.1.x:
- 1. Directories were very large and use up a lot of bandwidth: clients
- downloaded descriptors for all router several times an hour.
- 2. Every directory authority was a trust bottleneck: if a single
- directory authority lied, it could make clients believe for a time an
- arbitrarily distorted view of the Tor network.
- 3. Our current "verified server" system is kind of nonsensical.
-
- 4. Getting more directory authorities would add more points of failure
- and worsen possible partitioning attacks.
-
- There are two problems that remain unaddressed by this design.
- 5. Requiring every client to know about every router won't scale.
- 6. Requiring every directory cache to know every router won't scale.
-
- We attempt to fix 1-4 here, and to build a solution that will work when we
- figure out an answer for 5. We haven't thought at all about what to do
- about 6.
-
-1. Outline
-
- There is a small set (say, around 10) of semi-trusted directory
- authorities. A default list of authorities is shipped with the Tor
- software. Users can change this list, but are encouraged not to do so, in
- order to avoid partitioning attacks.
-
- Routers periodically upload signed "descriptors" to the directory
- authorities describing their keys, capabilities, and other information.
- Routers may act as directory mirrors (also called "caches"), to reduce
- load on the directory authorities. They announce this in their
- descriptors.
-
- Each directory authority periodically generates and signs a compact
- "network status" document that lists that authority's view of the current
- descriptors and status for known routers, but which does not include the
- descriptors themselves.
-
- Directory mirrors download, cache, and re-serve network-status documents
- to clients.
-
- Clients, directory mirrors, and directory authorities all use
- network-status documents to find out when their list of routers is
- out-of-date. If it is, they download any missing router descriptors.
- Clients download missing descriptors from mirrors; mirrors and authorities
- download from authorities. Descriptors are downloaded by the hash of the
- descriptor, not by the server's identity key: this prevents servers from
- attacking clients by giving them descriptors nobody else uses.
-
- All directory information is uploaded and downloaded with HTTP.
-
- Coordination among directory authorities is done client-side: clients
- compute a vote-like algorithm among the network-status documents they
- have, and base their decisions on the result.
-
-1.1. What's different from 0.1.0.x?
-
- Clients used to download a signed concatenated set of router descriptors
- (called a "directory") from directory mirrors, regardless of which
- descriptors had changed.
-
- Between downloading directories, clients would download "network-status"
- documents that would list which servers were supposed to running.
-
- Clients would always believe the most recently published network-status
- document they were served.
-
- Routers used to upload fresh descriptors all the time, whether their keys
- and other information had changed or not.
-
-1.2. Document meta-format
-
- Router descriptors, directories, and running-routers documents all obey the
- following lightweight extensible information format.
-
- The highest level object is a Document, which consists of one or more
- Items. Every Item begins with a KeywordLine, followed by one or more
- Objects. A KeywordLine begins with a Keyword, optionally followed by
- whitespace and more non-newline characters, and ends with a newline. A
- Keyword is a sequence of one or more characters in the set [A-Za-z0-9-].
- An Object is a block of encoded data in pseudo-Open-PGP-style
- armor. (cf. RFC 2440)
-
- More formally:
-
- Document ::= (Item | NL)+
- Item ::= KeywordLine Object*
- KeywordLine ::= Keyword NL | Keyword WS ArgumentsChar+ NL
- Keyword = KeywordChar+
- KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-'
- ArgumentChar ::= any printing ASCII character except NL.
- WS = (SP | TAB)+
- Object ::= BeginLine Base-64-encoded-data EndLine
- BeginLine ::= "-----BEGIN " Keyword "-----" NL
- EndLine ::= "-----END " Keyword "-----" NL
-
- The BeginLine and EndLine of an Object must use the same keyword.
-
- When interpreting a Document, software MUST ignore any KeywordLine that
- starts with a keyword it doesn't recognize; future implementations MUST NOT
- require current clients to understand any KeywordLine not currently
- described.
-
- The "opt" keyword was used until Tor 0.1.2.5-alpha for non-critical future
- extensions. All implementations MUST ignore any item of the form "opt
- keyword ....." when they would not recognize "keyword ....."; and MUST
- treat "opt keyword ....." as synonymous with "keyword ......" when keyword
- is recognized.
-
- Implementations before 0.1.2.5-alpha rejected any document with a
- KeywordLine that started with a keyword that they didn't recognize.
- Implementations MUST prefix items not recognized by older versions of Tor
- with an "opt" until those versions of Tor are obsolete.
-
-2. Router operation
-
- ORs SHOULD generate a new router descriptor whenever any of the
- following events have occurred:
-
- - A period of time (18 hrs by default) has passed since the last
- time a descriptor was generated.
-
- - A descriptor field other than bandwidth or uptime has changed.
-
- - Bandwidth has changed by more than +/- 50% from the last time a
- descriptor was generated, and at least a given interval of time
- (20 mins by default) has passed since then.
-
- - Its uptime has been reset (by restarting).
-
- After generating a descriptor, ORs upload it to every directory
- authority they know, by posting it to the URL
-
- http://<hostname:port>/tor/
-
-2.1. Router descriptor format
-
- Every router descriptor MUST start with a "router" Item; MUST end with a
- "router-signature" Item and an extra NL; and MUST contain exactly one
- instance of each of the following Items: "published" "onion-key"
- "link-key" "signing-key" "bandwidth". Additionally, a router descriptor
- MAY contain any number of "accept", "reject", "fingerprint", "uptime", and
- "opt" Items. Other than "router" and "router-signature", the items may
- appear in any order.
-
- The items' formats are as follows:
- "router" nickname address ORPort SocksPort DirPort
-
- Indicates the beginning of a router descriptor. "address" must be an
- IPv4 address in dotted-quad format. The last three numbers indicate
- the TCP ports at which this OR exposes functionality. ORPort is a port
- at which this OR accepts TLS connections for the main OR protocol;
- SocksPort is deprecated and should always be 0; and DirPort is the
- port at which this OR accepts directory-related HTTP connections. If
- any port is not supported, the value 0 is given instead of a port
- number.
-
- "bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed
-
- Estimated bandwidth for this router, in bytes per second. The
- "average" bandwidth is the volume per second that the OR is willing to
- sustain over long periods; the "burst" bandwidth is the volume that
- the OR is willing to sustain in very short intervals. The "observed"
- value is an estimate of the capacity this server can handle. The
- server remembers the max bandwidth sustained output over any ten
- second period in the past day, and another sustained input. The
- "observed" value is the lesser of these two numbers.
-
- "platform" string
-
- A human-readable string describing the system on which this OR is
- running. This MAY include the operating system, and SHOULD include
- the name and version of the software implementing the Tor protocol.
-
- "published" YYYY-MM-DD HH:MM:SS
-
- The time, in GMT, when this descriptor was generated.
-
- "fingerprint"
-
- A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded in
- hex, with a single space after every 4 characters) for this router's
- identity key. A descriptor is considered invalid (and MUST be
- rejected) if the fingerprint line does not match the public key.
-
- [We didn't start parsing this line until Tor 0.1.0.6-rc; it should
- be marked with "opt" until earlier versions of Tor are obsolete.]
-
- "hibernating" 0|1
-
- If the value is 1, then the Tor server was hibernating when the
- descriptor was published, and shouldn't be used to build circuits.
-
- [We didn't start parsing this line until Tor 0.1.0.6-rc; it should be
- marked with "opt" until earlier versions of Tor are obsolete.]
-
- "uptime"
-
- The number of seconds that this OR process has been running.
-
- "onion-key" NL a public key in PEM format
-
- This key is used to encrypt EXTEND cells for this OR. The key MUST be
- accepted for at least 1 week after any new key is published in a
- subsequent descriptor.
-
- "signing-key" NL a public key in PEM format
-
- The OR's long-term identity key.
-
- "accept" exitpattern
- "reject" exitpattern
-
- These lines describe the rules that an OR follows when
- deciding whether to allow a new stream to a given address. The
- 'exitpattern' syntax is described below. The rules are considered in
- order; if no rule matches, the address will be accepted. For clarity,
- the last such entry SHOULD be accept *:* or reject *:*.
-
- "router-signature" NL Signature NL
-
- The "SIGNATURE" object contains a signature of the PKCS1-padded
- hash of the entire router descriptor, taken from the beginning of the
- "router" line, through the newline after the "router-signature" line.
- The router descriptor is invalid unless the signature is performed
- with the router's identity key.
-
- "contact" info NL
-
- Describes a way to contact the server's administrator, preferably
- including an email address and a PGP key fingerprint.
-
- "family" names NL
-
- 'Names' is a space-separated list of server nicknames or
- hexdigests. If two ORs list one another in their "family" entries,
- then OPs should treat them as a single OR for the purpose of path
- selection.
-
- For example, if node A's descriptor contains "family B", and node B's
- descriptor contains "family A", then node A and node B should never
- be used on the same circuit.
-
- "read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL
- "write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM,NUM,NUM... NL
-
- Declare how much bandwidth the OR has used recently. Usage is divided
- into intervals of NSEC seconds. The YYYY-MM-DD HH:MM:SS field
- defines the end of the most recent interval. The numbers are the
- number of bytes used in the most recent intervals, ordered from
- oldest to newest.
-
- [We didn't start parsing these lines until Tor 0.1.0.6-rc; they should
- be marked with "opt" until earlier versions of Tor are obsolete.]
-
- "eventdns" bool NL
-
- Declare whether this version of Tor is using the newer enhanced
- dns logic. Versions of Tor without eventdns SHOULD NOT be used for
- reverse hostname lookups.
-
- [All versions of Tor before 0.1.2.2-alpha should be assumed to have
- this option set to 0 if it is not present. All Tor versions at
- 0.1.2.2-alpha or later should be assumed to have this option set to
- 1 if it is not present. Until 0.1.2.1-alpha-dev, this option was
- not generated, even when eventdns was in use. Versions of Tor
- before 0.1.2.1-alpha-dev did not parse this option, so it should be
- marked "opt". With some future version, the old 'dnsworker' logic
- will be removed, rendering this option of historical interest only.]
-
-2.1. Nonterminals in router descriptors
-
- nickname ::= between 1 and 19 alphanumeric characters, case-insensitive.
- hexdigest ::= a '$', followed by 20 hexadecimal characters.
- [Represents a server by the digest of its identity key.]
-
- exitpattern ::= addrspec ":" portspec
- portspec ::= "*" | port | port "-" port
- port ::= an integer between 1 and 65535, inclusive.
- [Some implementations incorrectly generate ports with value 0.
- Implementations SHOULD accept this, and SHOULD NOT generate it.]
-
- addrspec ::= "*" | ip4spec | ip6spec
- ipv4spec ::= ip4 | ip4 "/" num_ip4_bits | ip4 "/" ip4mask
- ip4 ::= an IPv4 address in dotted-quad format
- ip4mask ::= an IPv4 mask in dotted-quad format
- num_ip4_bits ::= an integer between 0 and 32
- ip6spec ::= ip6 | ip6 "/" num_ip6_bits
- ip6 ::= an IPv6 address, surrounded by square brackets.
- num_ip6_bits ::= an integer between 0 and 128
-
- bool ::= "0" | "1"
-
- Ports are required; if they are not included in the router
- line, they must appear in the "ports" lines.
-
-3. Network status format
-
- Directory authorities generate, sign, and compress network-status
- documents. Directory servers SHOULD generate a fresh network-status
- document when the contents of such a document would be different from the
- last one generated, and some time (at least one second, possibly longer)
- has passed since the last one was generated.
-
- The network status document contains a preamble, a set of router status
- entries, and a signature, in that order.
-
- We use the same meta-format as used for directories and router descriptors
- in "tor-spec.txt". Implementations MAY insert blank lines
- for clarity between sections; these blank lines are ignored.
- Implementations MUST NOT depend on blank lines in any particular location.
-
- As used here, "whitespace" is a sequence of 1 or more tab or space
- characters.
-
- The preamble contains:
-
- "network-status-version" -- A document format version. For this
- specification, the version is "2".
- "dir-source" -- The authority's hostname, current IP address, and
- directory port, all separated by whitespace.
- "fingerprint" -- A base16-encoded hash of the signing key's
- fingerprint, with no additional spaces added.
- "contact" -- An arbitrary string describing how to contact the
- directory server's administrator. Administrators should include at
- least an email address and a PGP fingerprint.
- "dir-signing-key" -- The directory server's public signing key.
- "client-versions" -- A comma-separated list of recommended client
- versions.
- "server-versions" -- A comma-separated list of recommended server
- versions.
- "published" -- The publication time for this network-status object.
- "dir-options" -- A set of flags, in any order, separated by whitespace:
- "Names" if this directory authority performs name bindings.
- "Versions" if this directory authority recommends software versions.
- "BadExits" if the directory authority flags nodes that it believes
- are performing incorrectly as exit nodes.
- "BadDirectories" if the directory authority flags nodes that it
- believes are performing incorrectly as directory caches.
-
- The dir-options entry is optional. The "-versions" entries are required if
- the "Versions" flag is present. The other entries are required and must
- appear exactly once. The "network-status-version" entry must appear first;
- the others may appear in any order. Implementations MUST ignore
- additional arguments to the items above, and MUST ignore unrecognized
- flags.
-
- For each router, the router entry contains: (This format is designed for
- conciseness.)
-
- "r" -- followed by the following elements, in order, separated by
- whitespace:
- - The OR's nickname,
- - A hash of its identity key, encoded in base64, with trailing =
- signs removed.
- - A hash of its most recent descriptor, encoded in base64, with
- trailing = signs removed. (The hash is calculated as for
- computing the signature of a descriptor.)
- - The publication time of its most recent descriptor, in the form
- YYYY-MM-DD HH:MM:SS, in GMT.
- - An IP address
- - An OR port
- - A directory port (or "0" for none")
- "s" -- A series of whitespace-separated status flags, in any order:
- "Authority" if the router is a directory authority.
- "BadExit" if the router is believed to be useless as an exit node
- (because its ISP censors it, because it is behind a restrictive
- proxy, or for some similar reason).
- "BadDirectory" if the router is believed to be useless as a
- directory cache (because its directory port isn't working,
- its bandwidth is always throttled, or for some similar
- reason).
- "Exit" if the router is useful for building general-purpose exit
- circuits.
- "Fast" if the router is suitable for high-bandwidth circuits.
- "Guard" if the router is suitable for use as an entry guard.
- "Named" if the router's identity-nickname mapping is canonical,
- and this authority binds names.
- "Stable" if the router is suitable for long-lived circuits.
- "Running" if the router is currently usable.
- "Valid" if the router has been 'validated'.
- "V2Dir" if the router implements this protocol.
- "v" -- The version of the Tor protocol that this server is running. If
- the value begins with "Tor" SP, the rest of the string is a Tor
- version number, and the protocol is "The Tor protocol as supported
- by the given version of Tor." Otherwise, if the value begins with
- some other string, Tor has upgraded to a more sophisticated
- protocol versioning system, and the protocol is "a version of the
- Tor protocol more recent than any we recognize."
-
- The "r" entry for each router must appear first and is required. The
- "s" entry is optional (see Section 3.1 below for how the flags are
- decided). Unrecognized flags on the "s" line and extra elements
- on the "r" line must be ignored. The "v" line is optional; it was not
- supported until 0.1.2.5-alpha, and it must be preceded with an "opt"
- until all earlier versions of Tor are obsolete.
-
- The signature section contains:
-
- "directory-signature" nickname-of-dirserver NL Signature
-
- Signature is a signature of this network-status document
- (the document up until the signature, including the line
- "directory-signature <nick>\n"), using the directory authority's
- signing key.
-
- We compress the network status list with zlib before transmitting it.
-
-3.1. Establishing server status
-
- (This section describes how directory authorities choose which status
- flags to apply to routers, as of Tor 0.1.1.18-rc. Later directory
- authorities MAY do things differently, so long as clients keep working
- well. Clients MUST NOT depend on the exact behaviors in this section.)
-
- "Valid" -- a router is 'Valid' if it is running a version of Tor not
- known to be broken, and the directory authority has not blacklisted
- it as suspicious.
-
- "Named" -- Directory authority administrators may decide to support name
- binding. If they do, then they must maintain a file of
- nickname-to-identity-key mappings, and try to keep this file consistent
- with other directory authorities. If they don't, they act as clients, and
- report bindings made by other directory authorities (name X is bound to
- identity Y if at least one binding directory lists it, and no directory
- binds X to some other Y'.) A router is called 'Named' if the router
- believes the given name should be bound to the given key.
-
- "Running" -- A router is 'Running' if the authority managed to connect to
- it successfully within the last 30 minutes.
-
- "Stable" -- A router is 'Stable' if its uptime is above median for known
- running, valid routers, and it's running a version of Tor not known to
- drop circuits stupidly. (0.1.1.10-alpha through 0.1.1.16-rc are stupid
- this way.)
-
- "Fast" -- A router is 'Fast' if its bandwidth is in the top 7/8ths for
- known running, valid routers.
-
- "Guard" -- A router is a possible 'Guard' if it is 'Stable' and its
- bandwidth is above median for known running, valid routers. If the total
- bandwidth of Running Valid non-BadExit Exit servers is less than one third
- of the total bandwidth of all Running Valid servers, no Exit is listed as
- a Guard.
-
- "Authority" -- A router is called an 'Authority' if the authority
- generating the network-status document believes it is an authority.
-
- "V2Dir" -- A router supports the v2 directory protocol if it has an open
- directory port, and it is running a version of the directory protocol that
- supports the functionality clients need. (Currently, this is
- 0.1.1.9-alpha or later.)
-
- Directory server administrators may label some servers or IPs as
- blacklisted, and elect not to include them in their network-status lists.
-
- Thus, the network-status list includes all non-blacklisted,
- non-expired, non-superseded descriptors.
-
-4. Directory server operation
-
- All directory authorities and directory mirrors ("directory servers")
- implement this section, except as noted.
-
-4.1. Accepting uploads (authorities only)
-
- When a router posts a signed descriptor to a directory authority, the
- authority first checks whether it is well-formed and correctly
- self-signed. If it is, the authority next verifies that the nickname
- question is already assigned to a router with a different public key.
- Finally, the authority MAY check that the router is not blacklisted
- because of its key, IP, or another reason.
-
- If the descriptor passes these tests, and the authority does not already
- have a descriptor for a router with this public key, it accepts the
- descriptor and remembers it.
-
- If the authority _does_ have a descriptor with the same public key, the
- newly uploaded descriptor is remembered if its publication time is more
- recent than the most recent old descriptor for that router, and either:
- - There are non-cosmetic differences between the old descriptor and the
- new one.
- - Enough time has passed between the descriptors' publication times.
- (Currently, 12 hours.)
-
- Differences between router descriptors are "non-cosmetic" if they would be
- sufficient to force an upload as described in section 2 above.
-
- Note that the "cosmetic difference" test only applies to uploaded
- descriptors, not to descriptors that the authority downloads from other
- authorities.
-
-4.2. Downloading network-status documents (authorities and caches)
-
- All directory servers (authorities and mirrors) try to keep a fresh
- set of network-status documents from every authority. To do so,
- every 5 minutes, each authority asks every other authority for its
- most recent network-status document. Every 15 minutes, each mirror
- picks a random authority and asks it for the most recent network-status
- documents for all the authorities the authority knows about (including
- the chosen authority itself).
-
- Directory servers and mirrors remember and serve the most recent
- network-status document they have from each authority. Other
- network-status documents don't need to be stored. If the most recent
- network-status document is over 10 days old, it is discarded anyway.
- Mirrors SHOULD store and serve network-status documents from authorities
- they don't recognize, but SHOULD NOT use such documents for any other
- purpose. Mirrors SHOULD discard network-status documents older than 48
- hours.
-
-4.3. Downloading and storing router descriptors (authorities and caches)
-
- Periodically (currently, every 10 seconds), directory servers check
- whether there are any specific descriptors (as identified by descriptor
- hash in a network-status document) that they do not have and that they
- are not currently trying to download.
-
- If so, the directory server launches requests to the authorities for these
- descriptors, such that each authority is only asked for descriptors listed
- in its most recent network-status. When more than one authority lists the
- descriptor, we choose which to ask at random.
-
- If one of these downloads fails, we do not try to download that descriptor
- from the authority that failed to serve it again unless we receive a newer
- network-status from that authority that lists the same descriptor.
-
- Directory servers must potentially cache multiple descriptors for each
- router. Servers must not discard any descriptor listed by any current
- network-status document from any authority. If there is enough space to
- store additional descriptors, servers SHOULD try to hold those which
- clients are likely to download the most. (Currently, this is judged
- based on the interval for which each descriptor seemed newest.)
-
- Authorities SHOULD NOT download descriptors for routers that they would
- immediately reject for reasons listed in 3.1.
-
-4.4. HTTP URLs
-
- "Fingerprints" in these URLs are base-16-encoded SHA1 hashes.
-
- The authoritative network-status published by a host should be available at:
- http://<hostname>/tor/status/authority.z
-
- The network-status published by a host with fingerprint
- <F> should be available at:
- http://<hostname>/tor/status/fp/<F>.z
-
- The network-status documents published by hosts with fingerprints
- <F1>,<F2>,<F3> should be available at:
- http://<hostname>/tor/status/fp/<F1>+<F2>+<F3>.z
-
- The most recent network-status documents from all known authorities,
- concatenated, should be available at:
- http://<hostname>/tor/status/all.z
-
- The most recent descriptor for a server whose identity key has a
- fingerprint of <F> should be available at:
- http://<hostname>/tor/server/fp/<F>.z
-
- The most recent descriptors for servers with identity fingerprints
- <F1>,<F2>,<F3> should be available at:
- http://<hostname>/tor/server/fp/<F1>+<F2>+<F3>.z
-
- (NOTE: Implementations SHOULD NOT download descriptors by identity key
- fingerprint. This allows a corrupted server (in collusion with a cache) to
- provide a unique descriptor to a client, and thereby partition that client
- from the rest of the network.)
-
- The server descriptor with (descriptor) digest <D> (in hex) should be
- available at:
- http://<hostname>/tor/server/d/<D>.z
-
- The most recent descriptors with digests <D1>,<D2>,<D3> should be
- available at:
- http://<hostname>/tor/server/d/<D1>+<D2>+<D3>.z
-
- The most recent descriptor for this server should be at:
- http://<hostname>/tor/server/authority.z
- [Nothing in the Tor protocol uses this resource yet, but it is useful
- for debugging purposes. Also, the official Tor implementations
- (starting at 0.1.1.x) use this resource to test whether a server's
- own DirPort is reachable.]
-
- A concatenated set of the most recent descriptors for all known servers
- should be available at:
- http://<hostname>/tor/server/all.z
-
- For debugging, directories SHOULD expose non-compressed objects at URLs like
- the above, but without the final ".z".
- Clients MUST handle compressed concatenated information in two forms:
- - A concatenated list of zlib-compressed objects.
- - A zlib-compressed concatenated list of objects.
- Directory servers MAY generate either format: the former requires less
- CPU, but the latter requires less bandwidth.
-
- Clients SHOULD use upper case letters (A-F) when base16-encoding
- fingerprints. Servers MUST accept both upper and lower case fingerprints
- in requests.
-
-5. Client operation: downloading information
-
- Every Tor that is not a directory server (that is, those that do
- not have a DirPort set) implements this section.
-
-5.1. Downloading network-status documents
-
- Each client maintains an ordered list of directory authorities.
- Insofar as possible, clients SHOULD all use the same ordered list.
-
- For each network-status document a client has, it keeps track of its
- publication time *and* the time when the client retrieved it. Clients
- consider a network-status document "live" if it was published within the
- last 24 hours.
-
- Clients try to have a live network-status document hours from *every*
- authority, and try to periodically get new network-status documents from
- each authority in rotation as follows:
-
- If a client is missing a live network-status document for any
- authority, it tries to fetch it from a directory cache. On failure,
- the client waits briefly, then tries that network-status document
- again from another cache. The client does not build circuits until it
- has live network-status documents from more than half the authorities
- it trusts, and it has descriptors for more than 1/4 of the routers
- that it believes are running.
-
- If the most recently _retrieved_ network-status document is over 30
- minutes old, the client attempts to download a network-status document.
- When choosing which documents to download, clients treat their list of
- directory authorities as a circular ring, and begin with the authority
- appearing immediately after the authority for their most recently
- retrieved network-status document. If this attempt fails, the client
- retries at other caches several times, before moving on to the next
- network-status document in sequence.
-
- Clients discard all network-status documents over 24 hours old.
-
- If enough mirrors (currently 4) claim not to have a given network status,
- we stop trying to download that authority's network-status, until we
- download a new network-status that makes us believe that the authority in
- question is running. Clients should wait a little longer after each
- failure.
-
- Clients SHOULD try to batch as many network-status requests as possible
- into each HTTP GET.
-
- (Note: clients can and should pick caches based on the network-status
- information they have: once they have first fetched network-status info
- from an authority, they should not need to go to the authority directly
- again.)
-
-5.2. Downloading and storing router descriptors
-
- Clients try to have the best descriptor for each router. A descriptor is
- "best" if:
- * It is the most recently published descriptor listed for that router
- by at least two network-status documents.
- OR,
- * No descriptor for that router is listed by two or more
- network-status documents, and it is the most recently published
- descriptor listed by any network-status document.
-
- Periodically (currently every 10 seconds) clients check whether there are
- any "downloadable" descriptors. A descriptor is downloadable if:
- - It is the "best" descriptor for some router.
- - The descriptor was published at least 10 minutes in the past.
- (This prevents clients from trying to fetch descriptors that the
- mirrors have probably not yet retrieved and cached.)
- - The client does not currently have it.
- - The client is not currently trying to download it.
- - The client would not discard it immediately upon receiving it.
- - The client thinks it is running and valid (see 6.1 below).
-
- If at least 16 known routers have downloadable descriptors, or if
- enough time (currently 10 minutes) has passed since the last time the
- client tried to download descriptors, it launches requests for all
- downloadable descriptors, as described in 5.3 below.
-
- When a descriptor download fails, the client notes it, and does not
- consider the descriptor downloadable again until a certain amount of time
- has passed. (Currently 0 seconds for the first failure, 60 seconds for the
- second, 5 minutes for the third, 10 minutes for the fourth, and 1 day
- thereafter.) Periodically (currently once an hour) clients reset the
- failure count.
-
- No descriptors are downloaded until the client has downloaded more than
- half of the network-status documents.
-
- Clients retain the most recent descriptor they have downloaded for each
- router so long as it is not too old (currently, 48 hours), OR so long as
- it is recommended by at least one networkstatus AND no "better"
- descriptor has been downloaded. [Versions of Tor before 0.1.2.3-alpha
- would discard descriptors simply for being published too far in the past.]
- [The code seems to discard descriptors in all cases after they're 5
- days old. True? -RD]
-
-5.3. Managing downloads
-
- When a client has no live network-status documents, it downloads
- network-status documents from a randomly chosen authority. In all other
- cases, the client downloads from mirrors randomly chosen from among those
- believed to be V2 directory servers. (This information comes from the
- network-status documents; see 6 below.)
-
- When downloading multiple router descriptors, the client chooses multiple
- mirrors so that:
- - At least 3 different mirrors are used, except when this would result
- in more than one request for under 4 descriptors.
- - No more than 128 descriptors are requested from a single mirror.
- - Otherwise, as few mirrors as possible are used.
- After choosing mirrors, the client divides the descriptors among them
- randomly.
-
- After receiving any response client MUST discard any network-status
- documents and descriptors that it did not request.
-
-6. Using directory information
-
- Everyone besides directory authorities uses the approaches in this section
- to decide which servers to use and what their keys are likely to be.
- (Directory authorities just believe their own opinions, as in 3.1 above.)
-
-6.1. Choosing routers for circuits.
-
- Tor implementations only pay attention to "live" network-status documents.
- A network status is "live" if it is the most recently downloaded network
- status document for a given directory server, and the server is a
- directory server trusted by the client, and the network-status document is
- no more than 1 day old.
-
- For time-sensitive information, Tor implementations focus on "recent"
- network-status documents. A network status is "recent" if it is live, and
- if it was published in the last 60 minutes. If there are fewer
- than 3 such documents, the most recently published 3 are "recent." If
- there are fewer than 3 in all, all are "recent.")
-
- Circuits SHOULD NOT be built until the client has enough directory
- information: network-statuses (or failed attempts to download
- network-statuses) for all authorities, network-statuses for at more than
- half of the authorites, and descriptors for at least 1/4 of the servers
- believed to be running.
-
- A server is "listed" if it is included by more than half of the live
- network status documents. Clients SHOULD NOT use unlisted servers.
-
- Clients believe the flags "Valid", "Exit", "Fast", "Guard", "Stable", and
- "V2Dir" about a given router when they are asserted by more than half of
- the live network-status documents. Clients believe the flag "Running" if
- it is listed by more than half of the recent network-status documents.
-
- These flags are used as follows:
-
- - Clients SHOULD NOT use non-'Valid' or non-'Running' routers unless
- requested to do so.
-
- - Clients SHOULD NOT use non-'Fast' routers for any purpose other than
- very-low-bandwidth circuits (such as introduction circuits).
-
- - Clients SHOULD NOT use non-'Stable' routers for circuits that are
- likely to need to be open for a very long time (such as those used for
- IRC or SSH connections).
-
- - Clients SHOULD NOT choose non-'Guard' nodes when picking entry guard
- nodes.
-
- - Clients SHOULD NOT download directory information from non-'V2Dir'
- caches.
-
-6.1. Managing naming
-
- In order to provide human-memorable names for individual server
- identities, some directory servers bind names to IDs. Clients handle
- names in two ways:
-
- When a client encounters a name it has not mapped before:
-
- If all the live "Naming" network-status documents the client has
- claim that the name binds to some identity ID, and the client has at
- least three live network-status documents, the client maps the name to
- ID.
-
- When a user tries to refer to a router with a name that does not have a
- mapping under the above rules, the implementation SHOULD warn the user.
- After giving the warning, the implementation MAY use a router that at
- least one Naming authority maps the name to, so long as no other naming
- authority maps that name to a different router. If no Naming authority
- maps the name to a router, the implementation MAY use any router that
- advertises the name.
-
- Not every router needs a nickname. When a router doesn't configure a
- nickname, it publishes with the default nickname "Unnamed". Authorities
- SHOULD NOT ever mark a router with this nickname as Named; client software
- SHOULD NOT ever use a router in response to a user request for a router
- called "Unnamed".
-
-6.2. Software versions
-
- An implementation of Tor SHOULD warn when it has fetched (or has
- attempted to fetch and failed four consecutive times) a network-status
- for each authority, and it is running a software version
- not listed on more than half of the live "Versioning" network-status
- documents.
-
-6.3. Warning about a router's status.
-
- If a router tries to publish its descriptor to a Naming authority
- that has its nickname mapped to another key, the router SHOULD
- warn the operator that it is either using the wrong key or is using
- an already claimed nickname.
-
- If a router has fetched (or attempted to fetch and failed four
- consecutive times) a network-status for every authority, and at
- least one of the authorities is "Naming", and no live "Naming"
- authorities publish a binding for the router's nickname, the
- router MAY remind the operator that the chosen nickname is not
- bound to this key at the authorities, and suggest contacting the
- authority operators.
-
- ...
-
-6.4. Router protocol versions
-
- A client should believe that a router supports a given feature if that
- feature is supported by the router or protocol versions in more than half
- of the live networkstatus's "v" entries for that router. In other words,
- if the "v" entries for some router are:
- v Tor 0.0.8pre1
- v Tor 0.1.2.11
- v FutureProtocolDescription 99
- then the client should believe that the router supports any feature
- supported by 0.1.2.11.
-
- This is currently equivalent to believing the median declared version for
- a router in all live networkstatuses.
-
-7. Standards compliance
-
- All clients and servers MUST support HTTP 1.0.
-
-7.1. HTTP headers
-
- Servers MAY set the Content-Length: header. Servers SHOULD set
- Content-Encoding to "deflate" or "identity".
-
- Servers MAY include an X-Your-Address-Is: header, whose value is the
- apparent IP address of the client connecting to them (as a dotted quad).
- For directory connections tunneled over a BEGIN_DIR stream, servers SHOULD
- report the IP from which the circuit carrying the BEGIN_DIR stream reached
- them. [Servers before version 0.1.2.5-alpha reported 127.0.0.1 for all
- BEGIN_DIR-tunneled connections.]
-
- Servers SHOULD disable caching of multiple network statuses or multiple
- router descriptors. Servers MAY enable caching of single descriptors,
- single network statuses, the list of all router descriptors, a v1
- directory, or a v1 running routers document. XXX mention times.
-
-7.2. HTTP status codes
-
- XXX We should write down what return codes dirservers send in what situations.
-
Deleted: tor/trunk/doc/path-spec.txt
===================================================================
--- tor/trunk/doc/path-spec.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/path-spec.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,551 +0,0 @@
-$Id$
-
- Tor Path Specification
-
- Roger Dingledine
- Nick Mathewson
-
-Note: This is an attempt to specify Tor as currently implemented. Future
-versions of Tor will implement improved algorithms.
-
-This document tries to cover how Tor chooses to build circuits and assign
-streams to circuits. Other implementations MAY take other approaches, but
-implementors should be aware of the anonymity and load-balancing implications
-of their choices.
-
- THIS SPEC ISN'T DONE OR CORRECT YET.
-
-1. General operation
-
- Tor begins building circuits as soon as it has enough directory
- information to do so (see section 5.1 of dir-spec.txt). Some circuits are
- built preemptively because we expect to need them later (for user
- traffic), and some are built because of immediate need (for user traffic
- that no current circuit can handle, for testing the network or our
- reachability, and so on).
-
- When a client application creates a new stream (by opening a SOCKS
- connection or launching a resolve request), we attach it to an appropriate
- open circuit if one exists, or wait if an appropriate circuit is
- in-progress. We launch a new circuit only
- if no current circuit can handle the request. We rotate circuits over
- time to avoid some profiling attacks.
-
- To build a circuit, we choose all the nodes we want to use, and then
- construct the circuit. Sometimes, when we want a circuit that ends at a
- given hop, and we have an appropriate unused circuit, we "cannibalize" the
- existing circuit and extend it to the new terminus.
-
- These processes are described in more detail below.
-
- This document describes Tor's automatic path selection logic only; path
- selection can be overridden by a controller (with the EXTENDCIRCUIT and
- ATTACHSTREAM commands). Paths constructed through these means may
- violate some constraints given below.
-
-1b. Terminology
-
- A "path" is an ordered sequence of nodes, not yet built as a circuit.
-
- A "clean" circuit is one that has not yet been used for any traffic.
-
- A "fast" or "stable" or "valid" node is one that has the 'Fast' or
- 'Stable' or 'Valid' flag
- set respectively, based on our current directory information. A "fast"
- or "stable" circuit is one consisting only of "fast" or "stable" nodes.
-
- In an "exit" circuit, the final node is chosen based on waiting stream
- requests if any, and in any case it avoids nodes with exit policy of
- "reject *:*". An "internal" circuit, on the other hand, is one where
- the final node is chosen just like a middle node (ignoring its exit
- policy).
-
- A "request" is a client-side stream or DNS resolve that needs to be
- served by a circuit.
-
- A "pending" circuit is one that we have started to build, but which has
- not yet completed.
-
- A circuit or path "supports" a request if it is okay to use the
- circuit/path to fulfill the request, according to the rules given below.
- A circuit or path "might support" a request if some aspect of the request
- is unknown (usually its target IP), but we believe the path probably
- supports the request according to the rules given below.
-
-2. Building circuits
-
-2.1. When we build.
-
-2.1.1. Clients build circuits preemptively
-
- When running as a client, Tor tries to maintain at least a certain
- number of clean circuits, so that new streams can be handled
- quickly. To increase the likelihood of success, Tor tries to
- predict what circuits will be useful by choosing from among nodes
- that support the ports we have used in the recent past (by default
- one hour). Specifically, on startup Tor tries to maintain one clean
- fast exit circuit that allows connections to port 80, and at least
- two internal circuits in case we get a resolve request or hidden
- service request (at least three internal circuits if we _run_ a
- hidden service).
-
- After that, Tor will adapt the circuits that it preemptively builds
- based on the requests it sees from the user: it tries to have a clean
- fast exit circuit available for every port seen recently (one circuit
- is adequate for many predicted ports -- it doesn't keep a separate
- circuit for each port), and it tries to have the above internal
- circuits available if we've seen resolves or hidden service activity
- recently. If there are 12 clean circuits open, it doesn't open more
- even if it has more predictions. Lastly, note that if there are no
- requests from the user for an hour, Tor will predict no use and build
- no preemptive circuits.
-
- The Tor client SHOULD NOT store its list of predicted requests to a
- persistent medium.
-
-2.1.2. Clients build circuits on demand
-
- Additionally, when a client request exists that no circuit (built or
- pending) might support, we create a new circuit to support the request.
- We do so by picking a request arbitrarily, launching a circuit to
- support it, and repeating until every unattached request might be
- supported by a pending or built circuit.
-
- For hidden service interations, we can "cannibalize" a clean internal
- circuit if one is available, so we don't need to build those circuits
- from scratch on demand.
-
- We can also cannibalize clean circuits when the client asks to exit
- at a given node -- either via mapaddress or the ".exit" notation,
- or because the destination is running at the same location as an
- exit node.
-
-2.1.3. Servers build circuits for testing reachability
-
- Tor servers test reachability of their ORPort on start and whenever
- their IP address changes.
- XXXX
-
-2.1.4. Hidden-service circuits
-
- See section 4 below.
-
-2.1.5. Rate limiting of failed circuits
-
- If we fail to build a circuit N times in a X second period (see Section
- 2.3 for how this works), we stop building circuits until the X seconds
- have elapsed.
- XXX
-
-2.1.6. When to tear down circuits
-
-
-2.2. Path selection and constraints
-
- We choose the path for each new circuit before we build it. We choose the
- exit node first, followed by the other nodes in the circuit. All paths
- we generate obey the following constraints:
- - We do not choose the same router twice for the same path.
- - We do not choose any router in the same family as another in the same
- path.
- - We do not choose more than one router in a given /16 subnet
- (unless EnforceDistinctSubnets is 0).
- - We don't choose any non-running or non-valid router unless we have
- been configured to do so. By default, we are configured to allow
- non-valid routers in "middle" and "rendezvous" positions.
- - If we're using Guard nodes, the first node must be a Guard (see 5
- below)
- - XXXX Choosing the length
-
- For circuits that do not need to be not "fast", when choosing among
- multiple candidates for a path element, we choose randomly.
-
- For "fast" circuits, we pick a given router as an exit with probability
- proportional to its advertised bandwidth [the smaller of the 'rate' and
- 'observed' arguments to the "bandwidth" element in its descriptor]. If a
- router's advertised bandwidth is greater than MAX_BELIEVABLE_BANDWIDTH
- (1.5 MB/s), we clip to that value.
-
- For non-exit positions on "fast" circuits, we pick routers as above, but
- we weight the clipped advertised bandwidth of Exit-flagged nodes depending
- on the fraction of bandwidth available from non-Exit nodes. Call the
- total clipped advertised bandwidth for Exit nodes under consideration E,
- and the total clipped advertised bandwidth for non-Exit nodes under
- consideration N. If E<N/2, we do not consider Exit-flagged nodes.
- Otherwise, we weight their bandwidth with the factor (E-N/2)/(N+E-N/2) ==
- (2E - N)/(2E + N). This ensures that bandwidth is evenly distributed over
- nodes in 3-hop paths.
-
- Additionally, we may be building circuits with one or more requests in
- mind. Each kind of request puts certain constraints on paths:
-
- - All service-side introduction circuits and all rendezvous paths
- should be Stable.
- - All connection requests for connections that we think will need to
- stay open a long time require Stable circuits. Currently, Tor decides
- this by examining the request's target port, and comparing it to a
- list of "long-lived" ports. (Default: 21, 22, 706, 1863, 5050,
- 5190, 5222, 5223, 6667, 6697, 8300.)
- - DNS resolves require an exit node whose exit policy is not equivalent
- to "reject *:*".
- - Reverse DNS resolves require a version of Tor with advertised eventdns
- support (available in Tor 0.1.2.1-alpha-dev and later).
- - All connection requests require an exit node whose exit policy
- supports their target address and port (if known), or which "might
- support it" (if the address isn't known). See 2.2.1.
- - Rules for Fast? XXXXX
-
-2.2.1. Choosing an exit
-
- If we know what IP address we want to resolve, we can trivially tell
- whether a given router will support it by simulating its declared
- exit policy.
-
- Because we often connect to addresses of the form hostname:port, we do not
- always know the target IP address when we select an exit node. In these
- cases, we need to pick an exit node that "might support" connections to a
- given address port with an unknown address. An exit node "might support"
- such a connection if any clause that accepts any connections to that port
- precedes all clauses (if any) that reject all connections to that port.
-
- Unless requested to do so by the user, we never choose an exit server
- flagged as "BadExit" by more than half of the authorities who advertise
- themselves as listing bad exits.
-
-2.2.2. User configuration
-
- Users can alter the default behavior for path selection with configuration
- options.
-
- - If "ExitNodes" is provided, then every request requires an exit node on
- the ExitNodes list. (If a request is supported by no nodes on that list,
- and StrictExitNodes is false, then Tor treats that request as if
- ExitNodes were not provided.)
-
- - "EntryNodes" and "StrictEntryNodes" behave analogously.
-
- - If a user tries to connect to or resolve a hostname of the form
- <target>.<servername>.exit, the request is rewritten to a request for
- <target>, and the request is only supported by the exit whose nickname
- or fingerprint is <servername>.
-
-2.3. Handling failure
-
- If an attempt to extend a circuit fails (either because the first create
- failed or a subsequent extend failed) then the circuit is torn down and is
- no longer pending. (XXXX really?) Requests that might have been
- supported by the pending circuit thus become unsupported, and a new
- circuit needs to be constructed.
-
- If a stream "begin" attempt fails with an EXITPOLICY error, we
- decide that the exit node's exit policy is not correctly advertised,
- so we treat the exit node as if it were a non-exit until we retrieve
- a fresh descriptor for it.
-
- XXXX
-
-3. Attaching streams to circuits
-
- When a circuit that might support a request is built, Tor tries to attach
- the request's stream to the circuit and sends a BEGIN or RESOLVE relay
- cell as appropriate. If the request completes unsuccessfully, Tor
- considers the reason given in the CLOSE relay cell. [XXX yes, and?]
-
-
- After a request has remained unattached for [XXXX interval?], Tor
- abandons the attempt and signals an error to the client as appropriate
- (e.g., by closing the SOCKS connection).
-
- XXX Timeouts and when Tor auto-retries.
- * What stream-end-reasons are appropriate for retrying.
-
- If no reply to BEGIN/RESOLVE, then the stream will timeout and fail.
-
-4. Hidden-service related circuits
-
- XXX Tracking expected hidden service use (client-side and hidserv-side)
-
-5. Guard nodes
-
- XXX writeme
-
-6. Testing circuits
-
-
-
-
-(From some emails by arma)
-
-Right now the code exists to pick helper nodes, store our choices to
-disk, and use them for our entry nodes. But there are three topics
-to tackle before I'm comfortable turning them on by default. First,
-how to handle churn: since Tor nodes are not always up, and sometimes
-disappear forever, we need a plan for replacing missing helpers in a
-safe way. Second, we need a way to distinguish "the network is down"
-from "all my helpers are down", also in a safe way. Lastly, we need to
-examine the situation where a client picks three crummy helper nodes
-and is forever doomed to a lousy Tor experience. Here's my plan:
-
-How to handle churn.
- - Keep track of whether you have ever actually established a
- connection to each helper. Any helper node in your list that you've
- never used is ok to drop immediately. Also, we don't save that
- one to disk.
- - If all our helpers are down, we need more helper nodes: add a new
- one to the *end*of our list. Only remove dead ones when they have
- been gone for a very long time (months).
- - Pick from the first n (by default 3) helper nodes in your list
- that are up (according to the network-statuses) and reachable
- (according to your local firewall config).
- - This means that order matters when writing/reading them to disk.
-
-How to deal with network down.
- - While all helpers are down/unreachable and there are no established
- or on-the-way testing circuits, launch a testing circuit. (Do this
- periodically in the same way we try to establish normal circuits
- when things are working normally.)
- (Testing circuits are a special type of circuit, that streams won't
- attach to by accident.)
- - When a testing circuit succeeds, mark all helpers up and hold
- the testing circuit open.
- - If a connection to a helper succeeds, close all testing circuits.
- Else mark that helper down and try another.
- - If the last helper is marked down and we already have a testing
- circuit established, then add the first hop of that testing circuit
- to the end of our helper node list, close that testing circuit,
- and go back to square one. (Actually, rather than closing the
- testing circuit, can we get away with converting it to a normal
- circuit and beginning to use it immediately?)
-
-How to pick non-sucky helpers.
- - When we're picking a new helper nodes, don't use ones which aren't
- reachable according to our local ReachableAddresses configuration.
- (There's an attack here: if I pick my helper nodes in a very
- restrictive environment, say "ReachableAddresses 18.0.0.0/255.0.0.0:*",
- then somebody watching me use the network from another location will
- guess where I first joined the network. But let's ignore it for now.)
- - Right now we choose new helpers just like we'd choose any entry
- node: they must be "stable" (claim >1day uptime) and "fast" (advertise
- >10kB capacity). In 0.1.1.11-alpha, clients let dirservers define
- "stable" and "fast" however they like, and they just believe them.
- So the next step is to make them a function of the current network:
- e.g. line up all the 'up' nodes in order and declare the top
- three-quarter to be stable, fast, etc, as long as they meet some
- minimum too.
- - If that's not sufficient (it won't be), dirservers should introduce
- a new status flag: in additional to "stable" and "fast", we should
- also describe certain nodes as "entry", meaning they are suitable
- to be chosen as a helper. The first difference would be that we'd
- demand the top half rather than the top three-quarters. Another
- requirement would be to look at "mean time between returning" to
- ensure that these nodes spend most of their time available. (Up for
- two days straight, once a month, is not good enough.)
- - Lastly, we need a function, given our current set of helpers and a
- directory of the rest of the network, that decides when our helper
- set has become "too crummy" and we need to add more. For example,
- this could be based on currently advertised capacity of each of
- our helpers, and it would also be based on the user's preferences
- of speed vs. security.
-
-***
-
-Lasse wrote:
-> I am a bit concerned with performance if we are to have e.g. two out of
-> three helper nodes down or unreachable. How often should Tor check if
-> they are back up and running?
-
-Right now Tor believes a threshold of directory servers when deciding
-whether each server is up. When Tor observes a server to be down
-(connection failed or building the first hop of the circuit failed),
-it marks it as down and doesn't try it again, until it gets a new
-network-status from somebody, at which point it takes a new concensus
-and marks the appropriate servers as up.
-
-According to sec 5.1 of dir-spec.txt, the client will try to fetch a new
-network-status at least every 30 minutes, and more often in certain cases.
-
-With the proposed scheme, we'll also mark all our helpers as up shortly
-after the last one is marked down.
-
-> When should there be
-> added an extra node to the helper node list? This is kind of an
-> important threshold?
-
-I agree, this is an important question. I don't have a good answer yet. Is
-it terrible, anonymity-wise, to add a new helper every time only one of
-your helpers is up? Notice that I say add rather than replace -- so you'd
-only use this fourth helper when one of your main three helpers is down,
-and if three of your four are down, you'd add a fifth, but only use it
-when two of the first four are down, etc.
-
-In fact, this may be smarter than just picking a random node for your
-testing circuit, because if your network goes up and down a lot, then
-eventually you have a chance of using any entry node in the network for
-your testing circuit.
-
-We have a design choice here. Do we only try to use helpers for the
-connections that will have streams on them (revealing our communication
-partners), or do we also want to restrict the overall set of nodes that
-we'll connect to, to discourage people from enumerating all Tor clients?
-
-I'm increasingly of the belief that we want to hide our presence too,
-based on the fact that Steven and George and others keep coming up with
-attacks that start with "Assuming we know the set of users".
-
-If so, then here's a revised "How to deal with network down" section:
-
- 1) When a helper is marked down or the helper list shrinks, and as
- a result the total number of helpers that are either (up and
- reachable) or (reachable but never connected to) is <= 1, then pick
- a new helper and add it to the end of the list.
- [We count nodes that have never been connected to, since otherwise
- we might keep on adding new nodes before trying any of them. By
- "reachable" I mean "is allowed by ReachableAddresses".]
- 2) When you fail to connect to a helper that has never been connected
- to, you remove him from the list right then (and the above rule
- might kick in).
- 3) When you succeed at connecting to a helper that you've never
- connected to before, mark all reachable helpers earlier in the list
- as up, and close that circuit.
- [We close the circuit, since if the other helpers are now up, we
- prefer to use them for circuits that will reveal communication
- partners.]
-
-This certainly seems simpler. Are there holes that I'm missing?
-
-> If running from a laptop you will meet different firewall settings, so
-> how should Helper Nodes settings keep up with moving from an open
-> ReachableAddresses to a FascistFirewall setting after the helper nodes
-> have been selected?
-
-I added the word "reachable" to three places in the above list, and I
-believe that totally solves this question.
-
-And as a bonus, it leads to an answer to Nick's attack ("If I pick
-my helper nodes all on 18.0.0.0:*, then I move, you'll know where I
-bootstrapped") -- the answer is to pick your original three helper nodes
-without regard for reachability. Then the above algorithm will add some
-more that are reachable for you, and if you move somewhere, it's more
-likely (though not certain) that some of the originals will become useful.
-Is that smart or just complex?
-
-> What happens if(when?) performance of the third node is bad?
-
-My above solution solves this a little bit, in that we always try to
-have two nodes available. But what if they are both up but bad? I'm not
-sure. As my previous mail said, we need some function, given our list
-of helpers and the network directory, that will tell us when we're in a
-bad situation. I can imagine some simple versions of this function --
-for example, when both our working helpers are in the bottom half of
-the nodes, ranked by capacity.
-
-But the hard part: what's the remedy when we decide there's something
-to fix? Do we add a third, and now we have two crummy ones and a new
-one? Or do we drop one or both of the bad ones?
-
-Perhaps we believe the latest claim from the network-status concensus,
-and we count a helper the dirservers believe is crummy as "not worth
-trying" (equivalent to "not reachable under our current ReachableAddresses
-config") -- and then the above algorithm would end up adding good ones,
-but we'd go back to the originals if they resume being acceptable? That's
-an appealing design. I wonder if it will cause the typical Tor user to
-have a helper node list that comprises most of the network, though. I'm
-ok with this.
-
-> Another point you might want to keep in mind, is the possibility to
-> reuse the code in order to add a second layer helper node (meaning node
-> number two) to "protect" the first layer (node number one) helper nodes.
-> These nodes should be tied to each of the first layer nodes. E.g. there
-> is one helper node list, as described in your mail, for each of the
-> first layer nodes, following their create/destroy.
-
-True. Does that require us to add a fourth hop to our path length,
-since the first hop is from a limited set, the second hop is from a
-limited set, and the third hop might also be constrained because, say,
-we're asking for an unusual exit port?
-
-> Another of the things might worth adding to the to do list is
-> localization of server (helper) nodes. Making it possible to pick
-> countries/regions where you do (not) want your helper nodes located. (As
-> in "HelperNodesLocated us,!eu" etc.) I know this requires the use of
-> external data and may not be worth it, but it _could_ be integrated at
-> the directory servers only -- adding a list of node IP's and e.g. a
-> country/region code to the directory and thus reduce the overhead. (?)
-> Maybe extending the Family-term?
-
-I think we are heading towards doing path selection based on geography,
-but I don't have a good sense yet of how that will actually turn out --
-that is, with what mechanism Tor clients will learn the information they
-need. But this seems to be something that is orthogonal to the rest of
-this discussion, so I look forward to having somebody else solve it for
-us, and fitting it in when it's ready. :)
-
-> And I would like to keep an option to pick the first X helper nodes
-> myself and then let Tor extend this list if these nodes are down (like
-> EntryNodes in current code). Even if this opens up for some new types of
-> "relationship" attacks.
-
-Good idea. Here's how I'd like to name these:
-
-The "EntryNodes" config option is a list of seed helper nodes. When we
-read EntryNodes, any node listed in entrynodes but not in the current
-helper node list gets *pre*pended to the helper node list.
-
-The "NumEntryNodes" config option (currently called NumHelperNodes)
-specifies the number of up, reachable, good-enough helper nodes that
-will make up the pool of possible choices for first hop, counted from
-the front of the helper node list until we have enough.
-
-The "UseEntryNodes" config option (currently called UseHelperNodes)
-tells us to turn on all this helper node behavior. If you set EntryNodes,
-then this option is implied.
-
-The "StrictEntryNodes" config option, provided for backward compatibility
-and for debugging, means a) we replace the helper node list with the
-current EntryNodes list, and b) whenever we would do an operation that
-alters the helper node list, we don't. (Yes, this means that if all the
-helper nodes are down, we lose until we mark them up again. But this is
-how it behaves now.)
-
-> I am sure my next point has been asked before, but what about testing
-> the current speed of the connections when looking for new helper nodes,
-> not only testing the connectivity? I know this might contribute to a lot
-> of overhead in the network, but if this only occur e.g. when using
-> helper nodes as a Hidden Service it might not have that large an impact,
-> but could help availability for the services?
-
-If we're just going to be testing them when we're first picking them,
-then it seems we can do the same thing by letting the directory servers
-test them. This has the added benefit that all the (behaving) clients
-use the same data, so they don't end up partitioned by a node that
-(for example) performs selectively for his victims.
-
-Another idea would be to periodically keep track of what speeds you get
-through your helpers, and make decisions from this. The reason we haven't
-done this yet is because there are a lot of variables -- perhaps the
-web site is slow, perhaps some other node in the path is slow, perhaps
-your local network is slow briefly, perhaps you got unlucky, etc. I
-believe that over time (assuming the user has roughly the same browsing
-habits) all of these would average out and you'd get a usable answer,
-but I don't have a good sense of how long it would take to converge,
-so I don't know whether this would be worthwhile.
-
-> BTW. I feel confortable with all the terms helper/entry/contact nodes,
-> but I think you (the developers) should just pick one and stay with it
-> to avoid confusion.
-
-I think I'm going to try to co-opt the term 'Entry' node for this
-purpose. We're going to have to keep referring to helper nodes for the
-research community for a while though, so they realize that Tor does
-more than just let users ask for certain entry nodes.
-
-
-
-============================================================
-Some stuff that worries me about entry guards. 2006 Jun, Nickm.
-
-1. It is unlikely for two users to have the same set of entry guards.
-
-2. Observing a user is sufficient to learn its entry guards.
-
-3. So, as we move around, we leak our
Deleted: tor/trunk/doc/rend-spec.txt
===================================================================
--- tor/trunk/doc/rend-spec.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/rend-spec.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,449 +0,0 @@
-$Id$
-
- Tor Rendezvous Specification
-
-0. Overview and preliminaries
-
- Read http://tor.eff.org/doc/design-paper/tor-design.html#sec:rendezvous
- before you read this specification. It will make more sense.
-
- Rendezvous points provide location-hidden services (server
- anonymity) for the onion routing network. With rendezvous points,
- Bob can offer a TCP service (say, a webserver) via the onion
- routing network, without revealing the IP of that service.
-
- Bob does this by anonymously advertising a public key for his
- service, along with a list of onion routers to act as "Introduction
- Points" for his service. He creates forward circuits to those
- introduction points, and tells them about his public key. To
- connect to Bob, Alice first builds a circuit to an OR to act as
- her "Rendezvous Point." She then connects to one of Bob's chosen
- introduction points, optionally provides authentication or
- authorization information, and asks it to tell him about her Rendezvous
- Point (RP). If Bob chooses to answer, he builds a circuit to her
- RP, and tells it to connect him to Alice. The RP joins their
- circuits together, and begins relaying cells. Alice's 'BEGIN'
- cells are received directly by Bob's OP, which passes data to
- and from the local server implementing Bob's service.
-
- Below we describe a network-level specification of this service,
- along with interfaces to make this process transparent to Alice
- (so long as she is using an OP).
-
-0.1. Notation, conventions and prerequisites
-
- In the specifications below, we use the same notation as in
- "tor-spec.txt". The service specified here also requires the
- existence of an onion routing network as specified in that file.
-
- H(x) is a SHA1 digest of x.
- PKSign(SK,x) is a PKCS.1-padded RSA signature of x with SK.
- PKEncrypt(SK,x) is a PKCS.1-padded RSA encryption of x with SK.
- Public keys are all RSA, and encoded in ASN.1.
- All integers are stored in network (big-endian) order.
- All symmetric encryption uses AES in counter mode, except where
- otherwise noted.
-
- In all discussions, "Alice" will refer to a user connecting to a
- location-hidden service, and "Bob" will refer to a user running a
- location-hidden service.
-
-0.2. Protocol outline
-
- 1. Bob->Bob's OP: "Offer IP:Port as
- public-key-name:Port". [configuration]
- (We do not specify this step; it is left to the implementor of
- Bob's OP.)
-
- 2. Bob's OP generates keypair and rendezvous service descriptor:
- "Meet public-key X at introduction point A, B, or C." (signed)
-
- 3. Bob's OP->Introduction point via Tor: [introduction setup]
- "This pk is me."
-
- 4. Bob's OP->directory service via Tor: publishes Bob's service
- descriptor [advertisement]
-
- 5. Out of band, Alice receives a [x.y.]z.onion:port address.
- She opens a SOCKS connection to her OP, and requests
- x.y.z.onion:port.
-
- 6. Alice's OP retrieves Bob's descriptor via Tor. [descriptor lookup.]
-
- 7. Alice's OP chooses a rendezvous point, opens a circuit to that
- rendezvous point, and establishes a rendezvous circuit. [rendezvous
- setup.]
-
- 8. Alice connects to the Introduction point via Tor, and tells it about
- her rendezvous point and optional authentication/authorization
- information. (Encrypted to Bob.) [Introduction 1]
-
- 9. The Introduction point passes this on to Bob's OP via Tor, along the
- introduction circuit. [Introduction 2]
-
- 10. Bob's OP decides whether to connect to Alice, and if so, creates a
- circuit to Alice's RP via Tor. Establishes a shared circuit.
- [Rendezvous.]
-
- 11. Alice's OP sends begin cells to Bob's OP. [Connection]
-
-0.3. Constants and new cell types
-
- Relay cell types
- 32 -- RELAY_ESTABLISH_INTRO
- 33 -- RELAY_ESTABLISH_RENDEZVOUS
- 34 -- RELAY_INTRODUCE1
- 35 -- RELAY_INTRODUCE2
- 36 -- RELAY_RENDEZVOUS1
- 37 -- RELAY_RENDEZVOUS2
- 38 -- RELAY_INTRO_ESTABLISHED
- 39 -- RELAY_RENDEZVOUS_ESTABLISHED
- 40 -- RELAY_COMMAND_INTRODUCE_ACK
-
-1. The Protocol
-
-1.1. Bob configures his local OP.
-
- We do not specify a format for the OP configuration file. However,
- OPs SHOULD allow Bob to provide more than one advertised service
- per OP, and MUST allow Bob to specify one or more virtual ports per
- service. Bob provides a mapping from each of these virtual ports
- to a local IP:Port pair.
-
-1.2. Bob's OP generates service descriptors.
-
- The first time the OP provides an advertised service, it generates
- a public/private keypair (stored locally). Periodically, the OP
- generates a pair of service descriptors, one "V1" and one "V0".
-
- The "V1" descriptor in 0.1.1.6-alpha contains:
-
- V Format byte: set to 255 [1 octet]
- V Version byte: set to 1 [1 octet]
- KL Key length [2 octets]
- PK Bob's public key [KL octets]
- TS A timestamp [4 octets]
- PROTO Rendezvous protocol versions: bitmask [2 octets]
- NA Number of auth mechanisms accepted [1 octet]
- For each auth mechanism:
- AUTHT The auth type that is supported [2 octets]
- AUTHL Length of auth data [1 octet]
- AUTHD Auth data [variable]
- NI Number of introduction points [2 octets]
- For each introduction point: (as in INTRODUCE2 cells)
- ATYPE An address type (typically 4) [1 octet]
- ADDR Introduction point's IP address [4 or 16 octets]
- PORT Introduction point's OR port [2 octets]
- AUTHT The auth type that is supported [2 octets]
- AUTHL Length of auth data [1 octet]
- AUTHD Auth data [variable]
- ID Introduction point identity ID [20 octets]
- KLEN Length of onion key [2 octets]
- KEY Introduction point onion key [KLEN octets]
- SIG Signature of above fields [variable]
-
- The "V1" descriptor in 0.1.1.5-alpha-cvs contains:
-
- V Format byte: set to 255 [1 octet]
- V Version byte: set to 1 [1 octet]
- KL Key length [2 octets]
- PK Bob's public key [KL octets]
- TS A timestamp [4 octets]
- PROTO Protocol versions: bitmask [2 octets]
- NI Number of introduction points [2 octets]
- For each introduction point: (as in INTRODUCE2 cells)
- IP Introduction point's address [4 octets]
- PORT Introduction point's OR port [2 octets]
- ID Introduction point identity ID [20 octets]
- KLEN Length of onion key [2 octets]
- KEY Introduction point onion key [KLEN octets]
- SIG Signature of above fields [variable]
-
- The "V0" descriptor contains:
-
- KL Key length [2 octets]
- PK Bob's public key [KL octets]
- TS A timestamp [4 octets]
- NI Number of introduction points [2 octets]
- Ipt A list of NUL-terminated ORs [variable]
- SIG Signature of above fields [variable]
-
- KL is the length of PK, in octets.
- TS is the number of seconds elapsed since Jan 1, 1970.
-
- AUTHT specifies which authentication/authorization mechanism is
- required by the hidden service or the introduction point. AUTHD
- is arbitrary data that can be associated with an auth approach.
- Currently only AUTHT of [00 00] is supported, with an AUTHL of 0.
- See section 2 of this document for details on auth mechanisms.
-
- The members of Ipt may be either (a) nicknames, or (b) identity key
- digests, encoded in hex, and prefixed with a '$'. Clients must
- accept both forms. Services must only generate the second form.
- Once 0.0.9.x is obsoleted, we can drop the first form.
-
- [It's ok for Bob to advertise 0 introduction points. He might want
- to do that if he previously advertised some introduction points,
- and now he doesn't have any. -RD]
-
- [Once Tor 0.1.0.x is obsolete, we can stop generating or using V0
- descriptors. -NM]
-
-1.3. Bob's OP establishes his introduction points.
-
- The OP establishes a new introduction circuit to each introduction
- point. These circuits MUST NOT be used for anything but rendezvous
- introduction. To establish the introduction, Bob sends a
- RELAY_ESTABLISH_INTRO cell, containing:
-
- KL Key length [2 octets]
- PK Bob's public key [KL octets]
- HS Hash of session info [20 octets]
- SIG Signature of above information [variable]
-
- [XXX011, need to add auth information here. -RD]
-
- To prevent replay attacks, the HS field contains a SHA-1 hash based on the
- shared secret KH between Bob's OP and the introduction point, as
- follows:
- HS = H(KH | "INTRODUCE")
- That is:
- HS = H(KH | [49 4E 54 52 4F 44 55 43 45])
- (KH, as specified in tor-spec.txt, is H(g^xy | [00]) .)
-
- Upon receiving such a cell, the OR first checks that the signature is
- correct with the included public key. If so, it checks whether HS is
- correct given the shared state between Bob's OP and the OR. If either
- check fails, the OP discards the cell; otherwise, it associates the
- circuit with Bob's public key, and dissociates any other circuits
- currently associated with PK. On success, the OR sends Bob a
- RELAY_INTRO_ESTABLISHED cell with an empty payload.
-
-1.4. Bob's OP advertises his service descriptor(s).
-
- Bob's OP opens a stream to each directory server's directory port via Tor.
- (He may re-use old circuits for this.) Over this stream, Bob's OP makes
- an HTTP 'POST' request, to a URL "/tor/rendezvous/publish" relative to the
- directory server's root, containing as its body Bob's service descriptor.
-
- Bob should upload a service descriptor for each version format that
- is supported in the current Tor network.
-
- Upon receiving a descriptor, the directory server checks the signature,
- and discards the descriptor if the signature does not match the enclosed
- public key. Next, the directory server checks the timestamp. If the
- timestamp is more than 24 hours in the past or more than 1 hour in the
- future, or the directory server already has a newer descriptor with the
- same public key, the server discards the descriptor. Otherwise, the
- server discards any older descriptors with the same public key and
- version format, and associates the new descriptor with the public key.
- The directory server remembers this descriptor for at least 24 hours
- after its timestamp. At least every 18 hours, Bob's OP uploads a
- fresh descriptor.
-
-1.5. Alice receives a x.y.z.onion address.
-
- When Alice receives a pointer to a location-hidden service, it is as a
- hostname of the form "z.onion" or "y.z.onion" or "x.y.z.onion", where
- z is a base-32 encoding of a 10-octet hash of Bob's service's public
- key, computed as follows:
-
- 1. Let H = H(PK).
- 2. Let H' = the first 80 bits of H, considering each octet from
- most significant bit to least significant bit.
- 2. Generate a 16-character encoding of H', using base32 as defined
- in RFC 3548.
-
- (We only use 80 bits instead of the 160 bits from SHA1 because we
- don't need to worry about arbitrary collisions, and because it will
- make handling the url's more convenient.)
-
- The string "x", if present, is the base-32 encoding of the
- authentication/authorization required by the introduction point.
- The string "y", if present, is the base-32 encoding of the
- authentication/authorization required by the hidden service.
- Omitting a string is taken to mean auth type [00 00].
- See section 2 of this document for details on auth mechanisms.
-
- [Yes, numbers are allowed at the beginning. See RFC 1123. -NM]
-
-1.6. Alice's OP retrieves a service descriptor.
-
- Alice opens a stream to a directory server via Tor, and makes an HTTP GET
- request for the document '/tor/rendezvous/<z>' or '/tor/rendezvous1/<z>',
- where '<z>' is replaced with the encoding of Bob's public key as described
- above. (She may re-use old circuits for this.) The directory replies with
- a 404 HTTP response if it does not recognize <z>, and otherwise returns
- Bob's most recently uploaded service descriptor. (If Alice requests
- 'rendezvous1', the directory server provides a V1 descriptor or a V0
- descriptor if no V1 descriptor is available. If Alice requests
- 'rendezvous', the directory server returns a V0 descriptor.)
-
- If Alice's OP receives a 404 response, it tries the other directory
- servers, and only fails the lookup if none recognize the public key hash.
-
- Upon receiving a service descriptor, Alice verifies with the same process
- as the directory server uses, described above in section 1.4.
-
- The directory server gives a 400 response if it cannot understand Alice's
- request.
-
- Alice should cache the descriptor locally, but should not use
- descriptors that are more than 24 hours older than their timestamp.
- [Caching may make her partitionable, but she fetched it anonymously,
- and we can't very well *not* cache it. -RD]
-
-1.7. Alice's OP establishes a rendezvous point.
-
- When Alice requests a connection to a given location-hidden service,
- and Alice's OP does not have an established circuit to that service,
- the OP builds a rendezvous circuit. It does this by establishing
- a circuit to a randomly chosen OR, and sending a
- RELAY_ESTABLISH_RENDEZVOUS cell to that OR. The body of that cell
- contains:
-
- RC Rendezvous cookie [20 octets]
-
- [XXX011 this looks like an auth mechanism. should we generalize here? -RD]
-
- The rendezvous cookie is an arbitrary 20-byte value, chosen randomly by
- Alice's OP.
-
- Upon receiving a RELAY_ESTABLISH_RENDEZVOUS cell, the OR associates the
- RC with the circuit that sent it. It replies to Alice with an empty
- RELAY_RENDEZVOUS_ESTABLISHED cell to indicate success.
-
- Alice's OP MUST NOT use the circuit which sent the cell for any purpose
- other than rendezvous with the given location-hidden service.
-
-1.8. Introduction: from Alice's OP to Introduction Point
-
- Alice builds a separate circuit to one of Bob's chosen introduction
- points, and sends it a RELAY_INTRODUCE1 cell containing:
-
- Cleartext
- PK_ID Identifier for Bob's PK [20 octets]
-[XXX011 want to put intro-level auth info here, but no version. crap. -RD]
-
- Encrypted to Bob's PK:
- RP Rendezvous point's nickname [20 octets]
- RC Rendezvous cookie [20 octets]
- g^x Diffie-Hellman data, part 1 [128 octets]
- OR
- VER Version byte: set to 1. [1 octet]
- RP Rendezvous point nick or ID [42 octets]
- RC Rendezvous cookie [20 octets]
- g^x Diffie-Hellman data, part 1 [128 octets]
- OR
- VER Version byte: set to 2. [1 octet]
- IP Rendezvous point's address [4 octets]
- PORT Rendezvous point's OR port [2 octets]
- ID Rendezvous point identity ID [20 octets]
- KLEN Length of onion key [2 octets]
- KEY Rendezvous point onion key [KLEN octets]
- RC Rendezvous cookie [20 octets]
- g^x Diffie-Hellman data, part 1 [128 octets]
- OR
- VER Version byte: set to 3. [1 octet]
- ATYPE An address type (typically 4) [1 octet]
- ADDR Introduction point's IP address [4 or 16 octets]
- PORT Rendezvous point's OR port [2 octets]
- AUTHT The auth type that is supported [2 octets]
- AUTHL Length of auth data [1 octet]
- AUTHD Auth data [variable]
- ID Rendezvous point identity ID [20 octets]
- KLEN Length of onion key [2 octets]
- KEY Rendezvous point onion key [KLEN octets]
- RC Rendezvous cookie [20 octets]
- g^x Diffie-Hellman data, part 1 [128 octets]
-
- PK_ID is the hash of Bob's public key. RP is NUL-padded and terminated,
- and must contain EITHER a nickname, or an identity key digest, encoded in
- hex, and prefixed with a '$'.
-
- Implementations SHOULD accept all variants, and list the variants they
- accept in their V1 descriptor. Implementations should only generate the
- variants listed in the service's V1 descriptor; if no V1 descriptor is
- available, only the first variant should be generated. No version should
- generate the second variant (version byte=1).
-
- The hybrid encryption to Bob's PK works just like the hybrid
- encryption in CREATE cells (see main spec). Thus the payload of the
- RELAY_INTRODUCE1 cell on the wire will contain 20+42+16+20+20+128=246
- bytes. [XXXX not really]
-
-1.9. Introduction: From the Introduction Point to Bob's OP
-
- If the Introduction Point recognizes PK_ID as a public key which has
- established a circuit for introductions as in 1.3 above, it sends the body
- of the cell in a new RELAY_INTRODUCE2 cell down the corresponding circuit.
- (If the PK_ID is unrecognized, the RELAY_INTRODUCE1 cell is discarded.)
-
- After sending the RELAY_INTRODUCE2 cell, the OR replies to Alice with an
- empty RELAY_COMMAND_INTRODUCE_ACK cell. If no RELAY_INTRODUCE2 cell can
- be sent, the OR replies to Alice with a non-empty cell to indicate an
- error. (The semantics of the cell body may be determined later; the
- current implementation sends a single '1' byte on failure.)
-
- When Bob's OP receives the RELAY_INTRODUCE2 cell, it decrypts it with
- the private key for the corresponding hidden service, and extracts the
- rendezvous point's nickname, the rendezvous cookie, and the value of g^x
- chosen by Alice.
-
-1.10. Rendezvous
-
- Bob's OP builds a new Tor circuit ending at Alice's chosen rendezvous
- point, and sends a RELAY_RENDEZVOUS1 cell along this circuit, containing:
- RC Rendezvous cookie [20 octets]
- g^y Diffie-Hellman [128 octets]
- KH Handshake digest [20 octets]
-
- (Bob's OP MUST NOT use this circuit for any other purpose.)
-
- If the RP recognizes RC, it relays the rest of the cell down the
- corresponding circuit in a RELAY_RENDEZVOUS2 cell, containing:
-
- g^y Diffie-Hellman [128 octets]
- KH Handshake digest [20 octets]
-
- (If the RP does not recognize the RC, it discards the cell and
- tears down the circuit.)
-
- When Alice's OP receives a RELAY_RENDEZVOUS2 cell on a circuit which
- has sent a RELAY_ESTABLISH_RENDEZVOUS cell but which has not yet received
- a reply, it uses g^y and H(g^xy) to complete the handshake as in the Tor
- circuit extend process: they establish a 60-octet string as
- K = SHA1(g^xy | [00]) | SHA1(g^xy | [01]) | SHA1(g^xy | [02])
- and generate
- KH = K[0..15]
- Kf = K[16..31]
- Kb = K[32..47]
-
- Subsequently, the rendezvous point passes relay cells, unchanged, from
- each of the two circuits to the other. When Alice's OP sends
- RELAY cells along the circuit, it first encrypts them with the
- Kf, then with all of the keys for the ORs in Alice's side of the circuit;
- and when Alice's OP receives RELAY cells from the circuit, it decrypts
- them with the keys for the ORs in Alice's side of the circuit, then
- decrypts them with Kb. Bob's OP does the same, with Kf and Kb
- interchanged.
-
-1.11. Creating streams
-
- To open TCP connections to Bob's location-hidden service, Alice's OP sends
- a RELAY_BEGIN cell along the established circuit, using the special
- address "", and a chosen port. Bob's OP chooses a destination IP and
- port, based on the configuration of the service connected to the circuit,
- and opens a TCP stream. From then on, Bob's OP treats the stream as an
- ordinary exit connection.
- [ Except he doesn't include addr in the connected cell or the end
- cell. -RD]
-
- Alice MAY send multiple RELAY_BEGIN cells along the circuit, to open
- multiple streams to Bob. Alice SHOULD NOT send RELAY_BEGIN cells for any
- other address along her circuit to Bob; if she does, Bob MUST reject them.
-
-2.0. Authentication and authorization.
-
-Foo.
-
Deleted: tor/trunk/doc/socks-extensions.txt
===================================================================
--- tor/trunk/doc/socks-extensions.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/socks-extensions.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,76 +0,0 @@
-$Id$
-Tor's extensions to the SOCKS protocol
-
-1. Overview
-
- The SOCKS protocol provides a generic interface for TCP proxies. Client
- software connects to a SOCKS server via TCP, and requests a TCP connection
- to another address and port. The SOCKS server establishes the connection,
- and reports success or failure to the client. After the connection has
- been established, the client application uses the TCP stream as usual.
-
- Tor supports SOCKS4 as defined in [1], SOCKS4A as defined in [2], and
- SOCKS5 as defined in [3].
-
- The stickiest issue for Tor in supporting clients, in practice, is forcing
- DNS lookups to occur at the OR side: if clients do their own DNS lookup,
- the DNS server can learn which addresses the client wants to reach.
- SOCKS4 supports addressing by IPv4 address; SOCKS4A is a kludge on top of
- SOCKS4 to allow addressing by hostname; SOCKS5 supports IPv4, IPv6, and
- hostnames.
-
-1.1. Extent of support
-
- Tor supports the SOCKS4, SOCKS4A, and SOCKS5 standards, except as follows:
-
- BOTH:
- - The BIND command is not supported.
-
- SOCKS4,4A:
- - SOCKS4 usernames are ignored.
-
- SOCKS5:
- - The (SOCKS5) "UDP ASSOCIATE" command is not supported.
- - IPv6 is not supported in CONNECT commands.
- - Only the "NO AUTHENTICATION" (SOCKS5) authentication method [00] is
- supported.
-
-2. Name lookup
-
- As an extension to SOCKS4A and SOCKS5, Tor implements a new command value,
- "RESOLVE" [F0]. When Tor receives a "RESOLVE" SOCKS command, it initiates
- a remote lookup of the hostname provided as the target address in the SOCKS
- request. The reply is either an error (if the address couldn't be
- resolved) or a success response. In the case of success, the address is
- stored in the portion of the SOCKS response reserved for remote IP address.
-
- (We support RESOLVE in SOCKS4 too, even though it is unnecessary.)
-
- For SOCKS5 only, we support reverse resolution with a new command value,
- "RESOLVE_PTR" [F1]. In response to a "RESOLVE_PTR" SOCKS5 command with
- an IPv4 address as its target, Tor attempts to find the canonical
- hostname for that IPv4 record, and returns it in the "server bound
- address" portion of the reply.
- (This command was not supported before Tor 0.1.2.2-alpha.)
-
-3. Other command extensions.
-
- Tor 0.1.2.4-alpha added a new command value: "CONNECT_DIR" [F2].
- In this case, Tor will open an encrypted direct TCP connection to the
- directory port of the Tor server specified by address:port (the port
- specified should be the ORPort of the server). It uses a one-hop tunnel
- and a "BEGIN_DIR" relay cell to accomplish this secure connection.
-
-4. HTTP-resistance
-
- Tor checks the first byte of each SOCKS request to see whether it looks
- more like an HTTP request (that is, it starts with a "G", "H", or "P"). If
- so, Tor returns a small webpage, telling the user that his/her browser is
- misconfigured. This is helpful for the many users who mistakenly try to
- use Tor as an HTTP proxy instead of a SOCKS proxy.
-
-References:
- [1] http://archive.socks.permeo.com/protocol/socks4.protocol
- [2] http://archive.socks.permeo.com/protocol/socks4a.protocol
- [3] SOCKS5: RFC1928
-
Copied: tor/trunk/doc/spec/address-spec.txt (from rev 9409, tor/trunk/doc/address-spec.txt)
Copied: tor/trunk/doc/spec/control-spec-v0.txt (from rev 9409, tor/trunk/doc/control-spec-v0.txt)
Copied: tor/trunk/doc/spec/control-spec.txt (from rev 9409, tor/trunk/doc/control-spec.txt)
Copied: tor/trunk/doc/spec/dir-spec.txt (from rev 9409, tor/trunk/doc/dir-spec.txt)
Copied: tor/trunk/doc/spec/path-spec.txt (from rev 9409, tor/trunk/doc/path-spec.txt)
Copied: tor/trunk/doc/spec/rend-spec.txt (from rev 9409, tor/trunk/doc/rend-spec.txt)
Copied: tor/trunk/doc/spec/socks-extensions.txt (from rev 9409, tor/trunk/doc/socks-extensions.txt)
Copied: tor/trunk/doc/spec/tor-spec.txt (from rev 9409, tor/trunk/doc/tor-spec.txt)
Copied: tor/trunk/doc/spec/version-spec.txt (from rev 9409, tor/trunk/doc/version-spec.txt)
Deleted: tor/trunk/doc/tor-spec.txt
===================================================================
--- tor/trunk/doc/tor-spec.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/tor-spec.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,816 +0,0 @@
-$Id$
-
- Tor Protocol Specification
-
- Roger Dingledine
- Nick Mathewson
-
-Note: This document aims to specify Tor as implemented in 0.1.2.x
-and earlier. Future versions of Tor may implement improved protocols, and
-compatibility is not guaranteed.
-
-This specification is not a design document; most design criteria
-are not examined. For more information on why Tor acts as it does,
-see tor-design.pdf.
-
-0. Preliminaries
-
-0.1. Notation and encoding
-
- PK -- a public key.
- SK -- a private key.
- K -- a key for a symmetric cypher.
-
- a|b -- concatenation of 'a' and 'b'.
-
- [A0 B1 C2] -- a three-byte sequence, containing the bytes with
- hexadecimal values A0, B1, and C2, in that order.
-
- All numeric values are encoded in network (big-endian) order.
-
- H(m) -- a cryptographic hash of m.
-
-0.2. Security parameters
-
- Tor uses a stream cipher, a public-key cipher, the Diffie-Hellman
- protocol, and a hash function.
-
- KEY_LEN -- the length of the stream cipher's key, in bytes.
-
- PK_ENC_LEN -- the length of a public-key encrypted message, in bytes.
- PK_PAD_LEN -- the number of bytes added in padding for public-key
- encryption, in bytes. (The largest number of bytes that can be encrypted
- in a single public-key operation is therefore PK_ENC_LEN-PK_PAD_LEN.)
-
- DH_LEN -- the number of bytes used to represent a member of the
- Diffie-Hellman group.
- DH_SEC_LEN -- the number of bytes used in a Diffie-Hellman private key (x).
-
- HASH_LEN -- the length of the hash function's output, in bytes.
-
- PAYLOAD_LEN -- The longest allowable cell payload, in bytes. (509)
-
- CELL_LEN -- The length of a Tor cell, in bytes.
-
-0.3. Ciphers
-
- For a stream cipher, we use 128-bit AES in counter mode, with an IV of all
- 0 bytes.
-
- For a public-key cipher, we use RSA with 1024-bit keys and a fixed
- exponent of 65537. We use OAEP-MGF1 padding, with SHA-1 as its digest
- function. We leave optional the "Label" parameter unset. (For OAEP
- padding, see ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-1/pkcs-1v2-1.pdf)
-
- For Diffie-Hellman, we use a generator (g) of 2. For the modulus (p), we
- use the 1024-bit safe prime from rfc2409 section 6.2 whose hex
- representation is:
-
- "FFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD129024E08"
- "8A67CC74020BBEA63B139B22514A08798E3404DDEF9519B3CD3A431B"
- "302B0A6DF25F14374FE1356D6D51C245E485B576625E7EC6F44C42E9"
- "A637ED6B0BFF5CB6F406B7EDEE386BFB5A899FA5AE9F24117C4B1FE6"
- "49286651ECE65381FFFFFFFFFFFFFFFF"
-
- As an optimization, implementations SHOULD choose DH private keys (x) of
- 320 bits. Implementations that do this MUST never use any DH key more
- than once.
- [May other implementations reuse their DH keys?? -RD]
- [Probably not. Conceivably, you could get away with changing DH keys once
- per second, but there are too many oddball attacks for me to be
- comfortable that this is safe. -NM]
-
- For a hash function, we use SHA-1.
-
- KEY_LEN=16.
- DH_LEN=128; DH_SEC_LEN=40.
- PK_ENC_LEN=128; PK_PAD_LEN=42.
- HASH_LEN=20.
-
- When we refer to "the hash of a public key", we mean the SHA-1 hash of the
- DER encoding of an ASN.1 RSA public key (as specified in PKCS.1).
-
- All "random" values should be generated with a cryptographically strong
- random number generator, unless otherwise noted.
-
- The "hybrid encryption" of a byte sequence M with a public key PK is
- computed as follows:
- 1. If M is less than PK_ENC_LEN-PK_PAD_LEN, pad and encrypt M with PK.
- 2. Otherwise, generate a KEY_LEN byte random key K.
- Let M1 = the first PK_ENC_LEN-PK_PAD_LEN-KEY_LEN bytes of M,
- and let M2 = the rest of M.
- Pad and encrypt K|M1 with PK. Encrypt M2 with our stream cipher,
- using the key K. Concatenate these encrypted values.
- [XXX Note that this "hybrid encryption" approach does not prevent
- an attacker from adding or removing bytes to the end of M. It also
- allows attackers to modify the bytes not covered by the OAEP --
- see Goldberg's PET2006 paper for details. We will add a MAC to this
- scheme one day. -RD]
-
-0.4. Other parameter values
-
- CELL_LEN=512
-
-1. System overview
-
- Tor is a distributed overlay network designed to anonymize
- low-latency TCP-based applications such as web browsing, secure shell,
- and instant messaging. Clients choose a path through the network and
- build a ``circuit'', in which each node (or ``onion router'' or ``OR'')
- in the path knows its predecessor and successor, but no other nodes in
- the circuit. Traffic flowing down the circuit is sent in fixed-size
- ``cells'', which are unwrapped by a symmetric key at each node (like
- the layers of an onion) and relayed downstream.
-
-1.1. Keys and names
-
- Every Tor server has multiple public/private keypairs:
-
- - A long-term signing-only "Identity key" used to sign documents and
- certificates, and used to establish server identity.
- - A medium-term "Onion key" used to decrypt onion skins when accepting
- circuit extend attempts. (See 5.1.) Old keys MUST be accepted for at
- least one week after they are no longer advertised. Because of this,
- servers MUST retain old keys for a while after they're rotated.
- - A short-term "Connection key" used to negotiate TLS connections.
- Tor implementations MAY rotate this key as often as they like, and
- SHOULD rotate this key at least once a day.
-
- Tor servers are also identified by "nicknames"; these are specified in
- dir-spec.txt.
-
-2. Connections
-
- Tor uses TLS for link authentication and encryption. All implementations
- MUST support
- the TLS ciphersuite "TLS_EDH_RSA_WITH_DES_192_CBC3_SHA", and SHOULD
- support "TLS_DHE_RSA_WITH_AES_128_CBC_SHA" if it is available.
- Implementations MAY support other ciphersuites, but MUST NOT
- support any suite without ephemeral keys, symmetric keys of at
- least KEY_LEN bits, and digests of at least HASH_LEN bits.
-
- Even though the connection protocol is identical, we think of the
- initiator as either an onion router (OR) if it is willing to relay
- traffic for other Tor users, or an onion proxy (OP) if it only handles
- local requests. Onion proxies SHOULD NOT provide long-term-trackable
- identifiers in their handshakes.
-
- During the TLS handshake, the connection initiator always sends a
- two-certificate chain, consisting of an X.509 certificate using a
- short-term connection public key and a second, self- signed X.509
- certificate containing its identity key. The commonName of the first
- certificate is the OR's nickname, and the commonName of the second
- certificate is the OR's nickname, followed by a space and the string
- "<identity>". The other party sends a similar certificate chain.
-
- Implementations running Protocol 1 and earlier use an
- organizationName of "Tor" or "TOR". Future implementations (which
- support the version negotiation protocol in section 4.1) MUST NOT
- have either of these values for their organizationName.
-
- All parties receiving certificates must confirm that the identity key is
- as expected. (When initiating a connection, the expected identity key is
- the one given in the directory; when creating a connection because of an
- EXTEND cell, the expected identity key is the one given in the cell.) If
- the key is not as expected, the party must close the connection.
-
- All parties SHOULD reject connections to or from ORs that have malformed
- or missing certificates. ORs MAY accept or reject connections from OPs
- with malformed or missing certificates.
-
- Once a TLS connection is established, the two sides send cells
- (specified below) to one another. Cells are sent serially. All
- cells are CELL_LEN bytes long. Cells may be sent embedded in TLS
- records of any size or divided across TLS records, but the framing
- of TLS records MUST NOT leak information about the type or contents
- of the cells.
-
- TLS connections are not permanent. Either side may close a connection
- if there are no circuits running over it and an amount of time
- (KeepalivePeriod, defaults to 5 minutes) has passed.
-
- (As an exception, directory servers may try to stay connected to all of
- the ORs -- though this will be phased out for the Tor 0.1.2.x release.)
-
-3. Cell Packet format
-
- The basic unit of communication for onion routers and onion
- proxies is a fixed-width "cell".
-
- On a version 1 connection, each cell contains the following
- fields:
-
- CircID [2 bytes]
- Command [1 byte]
- Payload (padded with 0 bytes) [PAYLOAD_LEN bytes]
-
- The CircID field determines which circuit, if any, the cell is
- associated with.
-
- The 'Command' field holds one of the following values:
- 0 -- PADDING (Padding) (See Sec 7.2)
- 1 -- CREATE (Create a circuit) (See Sec 5.1)
- 2 -- CREATED (Acknowledge create) (See Sec 5.1)
- 3 -- RELAY (End-to-end data) (See Sec 5.5 and 6)
- 4 -- DESTROY (Stop using a circuit) (See Sec 5.4)
- 5 -- CREATE_FAST (Create a circuit, no PK) (See Sec 5.1)
- 6 -- CREATED_FAST (Circuit created, no PK) (See Sec 5.1)
-
- The interpretation of 'Payload' depends on the type of the cell.
- PADDING: Payload is unused.
- CREATE: Payload contains the handshake challenge.
- CREATED: Payload contains the handshake response.
- RELAY: Payload contains the relay header and relay body.
- DESTROY: Payload contains a reason for closing the circuit.
- (see 5.4)
- Upon receiving any other value for the command field, an OR must
- drop the cell. [XXXX Versions prior to 0.1.0.?? logged a warning
- when dropping the cell; this is bad behavior. -NM]
-
- The payload is padded with 0 bytes.
-
- PADDING cells are currently used to implement connection keepalive.
- If there is no other traffic, ORs and OPs send one another a PADDING
- cell every few minutes.
-
- CREATE, CREATED, and DESTROY cells are used to manage circuits;
- see section 4 below.
-
- RELAY cells are used to send commands and data along a circuit; see
- section 5 below.
-
-4. [This section deliberately left blank.]
-
-5. Circuit management
-
-5.1. CREATE and CREATED cells
-
- Users set up circuits incrementally, one hop at a time. To create a
- new circuit, OPs send a CREATE cell to the first node, with the
- first half of the DH handshake; that node responds with a CREATED
- cell with the second half of the DH handshake plus the first 20 bytes
- of derivative key data (see section 5.2). To extend a circuit past
- the first hop, the OP sends an EXTEND relay cell (see section 5)
- which instructs the last node in the circuit to send a CREATE cell
- to extend the circuit.
-
- The payload for a CREATE cell is an 'onion skin', which consists
- of the first step of the DH handshake data (also known as g^x).
- This value is hybrid-encrypted (see 0.3) to Bob's onion key, giving
- an onion-skin of:
- PK-encrypted:
- Padding padding [PK_PAD_LEN bytes]
- Symmetric key [KEY_LEN bytes]
- First part of g^x [PK_ENC_LEN-PK_PAD_LEN-KEY_LEN bytes]
- Symmetrically encrypted:
- Second part of g^x [DH_LEN-(PK_ENC_LEN-PK_PAD_LEN-KEY_LEN)
- bytes]
-
- The relay payload for an EXTEND relay cell consists of:
- Address [4 bytes]
- Port [2 bytes]
- Onion skin [DH_LEN+KEY_LEN+PK_PAD_LEN bytes]
- Identity fingerprint [HASH_LEN bytes]
-
- The port and address field denote the IPV4 address and port of the next
- onion router in the circuit; the public key hash is the hash of the PKCS#1
- ASN1 encoding of the next onion router's identity (signing) key. (See 0.3
- above.) (Including this hash allows the extending OR verify that it is
- indeed connected to the correct target OR, and prevents certain
- man-in-the-middle attacks.)
-
- The payload for a CREATED cell, or the relay payload for an
- EXTENDED cell, contains:
- DH data (g^y) [DH_LEN bytes]
- Derivative key data (KH) [HASH_LEN bytes] <see 5.2 below>
-
- The CircID for a CREATE cell is an arbitrarily chosen 2-byte integer,
- selected by the node (OP or OR) that sends the CREATE cell. To prevent
- CircID collisions, when one OR sends a CREATE cell to another, it chooses
- from only one half of the possible values based on the ORs' public
- identity keys: if the sending OR has a lower key, it chooses a CircID with
- an MSB of 0; otherwise, it chooses a CircID with an MSB of 1.
-
- Public keys are compared numerically by modulus.
-
- As usual with DH, x and y MUST be generated randomly.
-
-[
- To implement backward-compatible version negotiation, parties MUST
- drop CREATE cells with all-[00] onion-skins.
-]
-
-5.1.1. CREATE_FAST/CREATED_FAST cells
-
- When initializing the first hop of a circuit, the OP has already
- established the OR's identity and negotiated a secret key using TLS.
- Because of this, it is not always necessary for the OP to perform the
- public key operations to create a circuit. In this case, the
- OP MAY send a CREATE_FAST cell instead of a CREATE cell for the first
- hop only. The OR responds with a CREATED_FAST cell, and the circuit is
- created.
-
- A CREATE_FAST cell contains:
-
- Key material (X) [HASH_LEN bytes]
-
- A CREATED_FAST cell contains:
-
- Key material (Y) [HASH_LEN bytes]
- Derivative key data [HASH_LEN bytes] (See 5.2 below)
-
- The values of X and Y must be generated randomly.
-
- [Versions of Tor before 0.1.0.6-rc did not support these cell types;
- clients should not send CREATE_FAST cells to older Tor servers.]
-
- If an OR sees a circuit created with CREATE_FAST, the OR is sure to be the
- first hop of a circuit. ORs SHOULD reject attempts to create streams with
- RELAY_BEGIN exiting the circuit at the first hop: letting Tor be used as a
- single hop proxy makes exit nodes a more attractive target for compromise.
-
-5.2. Setting circuit keys
-
- Once the handshake between the OP and an OR is completed, both can
- now calculate g^xy with ordinary DH. Before computing g^xy, both client
- and server MUST verify that the received g^x or g^y value is not degenerate;
- that is, it must be strictly greater than 1 and strictly less than p-1
- where p is the DH modulus. Implementations MUST NOT complete a handshake
- with degenerate keys. Implementations MUST NOT discard other "weak"
- g^x values.
-
- (Discarding degenerate keys is critical for security; if bad keys
- are not discarded, an attacker can substitute the server's CREATED
- cell's g^y with 0 or 1, thus creating a known g^xy and impersonating
- the server. Discarding other keys may allow attacks to learn bits of
- the private key.)
-
- (The mainline Tor implementation, in the 0.1.1.x-alpha series, discarded
- all g^x values less than 2^24, greater than p-2^24, or having more than
- 1024-16 identical bits. This served no useful purpose, and we stopped.)
-
- If CREATE or EXTEND is used to extend a circuit, the client and server
- base their key material on K0=g^xy, represented as a big-endian unsigned
- integer.
-
- If CREATE_FAST is used, the client and server base their key material on
- K0=X|Y.
-
- From the base key material K0, they compute KEY_LEN*2+HASH_LEN*3 bytes of
- derivative key data as
- K = H(K0 | [00]) | H(K0 | [01]) | H(K0 | [02]) | ...
-
- The first HASH_LEN bytes of K form KH; the next HASH_LEN form the forward
- digest Df; the next HASH_LEN 41-60 form the backward digest Db; the next
- KEY_LEN 61-76 form Kf, and the final KEY_LEN form Kb. Excess bytes from K
- are discarded.
-
- KH is used in the handshake response to demonstrate knowledge of the
- computed shared key. Df is used to seed the integrity-checking hash
- for the stream of data going from the OP to the OR, and Db seeds the
- integrity-checking hash for the data stream from the OR to the OP. Kf
- is used to encrypt the stream of data going from the OP to the OR, and
- Kb is used to encrypt the stream of data going from the OR to the OP.
-
-5.3. Creating circuits
-
- When creating a circuit through the network, the circuit creator
- (OP) performs the following steps:
-
- 1. Choose an onion router as an exit node (R_N), such that the onion
- router's exit policy includes at least one pending stream that
- needs a circuit (if there are any).
-
- 2. Choose a chain of (N-1) onion routers
- (R_1...R_N-1) to constitute the path, such that no router
- appears in the path twice.
-
- 3. If not already connected to the first router in the chain,
- open a new connection to that router.
-
- 4. Choose a circID not already in use on the connection with the
- first router in the chain; send a CREATE cell along the
- connection, to be received by the first onion router.
-
- 5. Wait until a CREATED cell is received; finish the handshake
- and extract the forward key Kf_1 and the backward key Kb_1.
-
- 6. For each subsequent onion router R (R_2 through R_N), extend
- the circuit to R.
-
- To extend the circuit by a single onion router R_M, the OP performs
- these steps:
-
- 1. Create an onion skin, encrypted to R_M's public onion key.
-
- 2. Send the onion skin in a relay EXTEND cell along
- the circuit (see section 5).
-
- 3. When a relay EXTENDED cell is received, verify KH, and
- calculate the shared keys. The circuit is now extended.
-
- When an onion router receives an EXTEND relay cell, it sends a CREATE
- cell to the next onion router, with the enclosed onion skin as its
- payload. The initiating onion router chooses some circID not yet
- used on the connection between the two onion routers. (But see
- section 5.1. above, concerning choosing circIDs based on
- lexicographic order of nicknames.)
-
- When an onion router receives a CREATE cell, if it already has a
- circuit on the given connection with the given circID, it drops the
- cell. Otherwise, after receiving the CREATE cell, it completes the
- DH handshake, and replies with a CREATED cell. Upon receiving a
- CREATED cell, an onion router packs it payload into an EXTENDED relay
- cell (see section 5), and sends that cell up the circuit. Upon
- receiving the EXTENDED relay cell, the OP can retrieve g^y.
-
- (As an optimization, OR implementations may delay processing onions
- until a break in traffic allows time to do so without harming
- network latency too greatly.)
-
-5.4. Tearing down circuits
-
- Circuits are torn down when an unrecoverable error occurs along
- the circuit, or when all streams on a circuit are closed and the
- circuit's intended lifetime is over. Circuits may be torn down
- either completely or hop-by-hop.
-
- To tear down a circuit completely, an OR or OP sends a DESTROY
- cell to the adjacent nodes on that circuit, using the appropriate
- direction's circID.
-
- Upon receiving an outgoing DESTROY cell, an OR frees resources
- associated with the corresponding circuit. If it's not the end of
- the circuit, it sends a DESTROY cell for that circuit to the next OR
- in the circuit. If the node is the end of the circuit, then it tears
- down any associated edge connections (see section 6.1).
-
- After a DESTROY cell has been processed, an OR ignores all data or
- destroy cells for the corresponding circuit.
-
- To tear down part of a circuit, the OP may send a RELAY_TRUNCATE cell
- signaling a given OR (Stream ID zero). That OR sends a DESTROY
- cell to the next node in the circuit, and replies to the OP with a
- RELAY_TRUNCATED cell.
-
- When an unrecoverable error occurs along one connection in a
- circuit, the nodes on either side of the connection should, if they
- are able, act as follows: the node closer to the OP should send a
- RELAY_TRUNCATED cell towards the OP; the node farther from the OP
- should send a DESTROY cell down the circuit.
-
- The payload of a RELAY_TRUNCATED or DESTROY cell contains a single octet,
- describing why the circuit is being closed or truncated. When sending a
- TRUNCATED or DESTROY cell because of another TRUNCATED or DESTROY cell,
- the error code should be propagated. The origin of a circuit always sets
- this error code to 0, to avoid leaking its version.
-
- The error codes are:
- 0 -- NONE (No reason given.)
- 1 -- PROTOCOL (Tor protocol violation.)
- 2 -- INTERNAL (Internal error.)
- 3 -- REQUESTED (A client sent a TRUNCATE command.)
- 4 -- HIBERNATING (Not currently operating; trying to save bandwidth.)
- 5 -- RESOURCELIMIT (Out of memory, sockets, or circuit IDs.)
- 6 -- CONNECTFAILED (Unable to reach server.)
- 7 -- OR_IDENTITY (Connected to server, but its OR identity was not
- as expected.)
- 8 -- OR_CONN_CLOSED (The OR connection that was carrying this circuit
- died.)
- 9 -- FINISHED (The circuit has expired for being dirty or old.)
- 10 -- TIMEOUT (Circuit construction took too long)
- 11 -- DESTROYED (The circuit was destroyed w/o client TRUNCATE)
- 12 -- NOSUCHSERVICE (Request for unknown hidden service)
-
- [Versions of Tor prior to 0.1.0.11 didn't send reasons; implementations
- MUST accept empty TRUNCATED and DESTROY cells.]
-
-5.5. Routing relay cells
-
- When an OR receives a RELAY cell, it checks the cell's circID and
- determines whether it has a corresponding circuit along that
- connection. If not, the OR drops the RELAY cell.
-
- Otherwise, if the OR is not at the OP edge of the circuit (that is,
- either an 'exit node' or a non-edge node), it de/encrypts the payload
- with the stream cipher, as follows:
- 'Forward' relay cell (same direction as CREATE):
- Use Kf as key; decrypt.
- 'Back' relay cell (opposite direction from CREATE):
- Use Kb as key; encrypt.
- Note that in counter mode, decrypt and encrypt are the same operation.
-
- The OR then decides whether it recognizes the relay cell, by
- inspecting the payload as described in section 6.1 below. If the OR
- recognizes the cell, it processes the contents of the relay cell.
- Otherwise, it passes the decrypted relay cell along the circuit if
- the circuit continues. If the OR at the end of the circuit
- encounters an unrecognized relay cell, an error has occurred: the OR
- sends a DESTROY cell to tear down the circuit.
-
- When a relay cell arrives at an OP, the OP decrypts the payload
- with the stream cipher as follows:
- OP receives data cell:
- For I=N...1,
- Decrypt with Kb_I. If the payload is recognized (see
- section 6..1), then stop and process the payload.
-
- For more information, see section 6 below.
-
-6. Application connections and stream management
-
-6.1. Relay cells
-
- Within a circuit, the OP and the exit node use the contents of
- RELAY packets to tunnel end-to-end commands and TCP connections
- ("Streams") across circuits. End-to-end commands can be initiated
- by either edge; streams are initiated by the OP.
-
- The payload of each unencrypted RELAY cell consists of:
- Relay command [1 byte]
- 'Recognized' [2 bytes]
- StreamID [2 bytes]
- Digest [4 bytes]
- Length [2 bytes]
- Data [CELL_LEN-14 bytes]
-
- The relay commands are:
- 1 -- RELAY_BEGIN [forward]
- 2 -- RELAY_DATA [forward or backward]
- 3 -- RELAY_END [forward or backward]
- 4 -- RELAY_CONNECTED [backward]
- 5 -- RELAY_SENDME [forward or backward] [sometimes control]
- 6 -- RELAY_EXTEND [forward] [control]
- 7 -- RELAY_EXTENDED [backward] [control]
- 8 -- RELAY_TRUNCATE [forward] [control]
- 9 -- RELAY_TRUNCATED [backward] [control]
- 10 -- RELAY_DROP [forward or backward] [control]
- 11 -- RELAY_RESOLVE [forward]
- 12 -- RELAY_RESOLVED [backward]
- 13 -- RELAY_BEGIN_DIR [forward]
-
- Commands labelled as "forward" must only be sent by the originator
- of the circuit. Commands labelled as "backward" must only be sent by
- other nodes in the circuit back to the originator. Commands marked
- as either can be sent either by the originator or other nodes.
-
- The 'recognized' field in any unencrypted relay payload is always set
- to zero; the 'digest' field is computed as the first four bytes of
- the running digest of all the bytes that have been destined for
- this hop of the circuit or originated from this hop of the circuit,
- seeded from Df or Db respectively (obtained in section 5.2 above),
- and including this RELAY cell's entire payload (taken with the digest
- field set to zero).
-
- When the 'recognized' field of a RELAY cell is zero, and the digest
- is correct, the cell is considered "recognized" for the purposes of
- decryption (see section 5.5 above).
-
- (The digest does not include any bytes from relay cells that do
- not start or end at this hop of the circuit. That is, it does not
- include forwarded data. Therefore if 'recognized' is zero but the
- digest does not match, the running digest at that node should
- not be updated, and the cell should be forwarded on.)
-
- All RELAY cells pertaining to the same tunneled stream have the
- same stream ID. StreamIDs are chosen arbitrarily by the OP. RELAY
- cells that affect the entire circuit rather than a particular
- stream use a StreamID of zero -- they are marked in the table above
- as "[control]" style cells. (Sendme cells are marked as "sometimes
- control" because they can take include a StreamID or not depending
- on their purpose -- see Section 7.)
-
- The 'Length' field of a relay cell contains the number of bytes in
- the relay payload which contain real payload data. The remainder of
- the payload is padded with NUL bytes.
-
- If the RELAY cell is recognized but the relay command is not
- understood, the cell must be dropped and ignored. Its contents
- still count with respect to the digests, though. [Before
- 0.1.1.10, Tor closed circuits when it received an unknown relay
- command. Perhaps this will be more forward-compatible. -RD]
-
-6.2. Opening streams and transferring data
-
- To open a new anonymized TCP connection, the OP chooses an open
- circuit to an exit that may be able to connect to the destination
- address, selects an arbitrary StreamID not yet used on that circuit,
- and constructs a RELAY_BEGIN cell with a payload encoding the address
- and port of the destination host. The payload format is:
-
- ADDRESS | ':' | PORT | [00]
-
- where ADDRESS can be a DNS hostname, or an IPv4 address in
- dotted-quad format, or an IPv6 address surrounded by square brackets;
- and where PORT is encoded in decimal.
-
- [What is the [00] for? -NM]
- [It's so the payload is easy to parse out with string funcs -RD]
-
- Upon receiving this cell, the exit node resolves the address as
- necessary, and opens a new TCP connection to the target port. If the
- address cannot be resolved, or a connection can't be established, the
- exit node replies with a RELAY_END cell. (See 6.4 below.)
- Otherwise, the exit node replies with a RELAY_CONNECTED cell, whose
- payload is in one of the following formats:
- The IPv4 address to which the connection was made [4 octets]
- A number of seconds (TTL) for which the address may be cached [4 octets]
- or
- Four zero-valued octets [4 octets]
- An address type (6) [1 octet]
- The IPv6 address to which the connection was made [16 octets]
- A number of seconds (TTL) for which the address may be cached [4 octets]
- [XXXX Versions of Tor before 0.1.1.6 ignore and do not generate the TTL
- field. No version of Tor currently generates the IPv6 format.
-
- Tor servers before 0.1.2.0 set the TTL field to a fixed value. Later
- versions set the TTL to the last value seen from a DNS server, and expire
- their own cached entries after a fixed interval. This prevents certain
- attacks.]
-
- The OP waits for a RELAY_CONNECTED cell before sending any data.
- Once a connection has been established, the OP and exit node
- package stream data in RELAY_DATA cells, and upon receiving such
- cells, echo their contents to the corresponding TCP stream.
- RELAY_DATA cells sent to unrecognized streams are dropped.
-
- Relay RELAY_DROP cells are long-range dummies; upon receiving such
- a cell, the OR or OP must drop it.
-
-6.2.1. Opening a directory stream
-
- If a Tor server is a directory server, it should respond to a
- RELAY_BEGIN_DIR cell as if it had received a BEGIN cell requesting a
- connection to its directory port. RELAY_BEGIN_DIR cells ignore exit
- policy, since the stream is local to the Tor process.
-
- If the Tor server is not running a directory service, it should respond
- with a REASON_NOTDIRECTORY RELAY_END cell.
-
- Clients MUST generate an all-zero payload for RELAY_BEGIN_DIR cells,
- and servers MUST ignore the payload.
-
- [RELAY_BEGIN_DIR was not supported before Tor 0.1.2.2-alpha; clients
- SHOULD NOT send it to routers running earlier versions of Tor.]
-
-6.3. Closing streams
-
- When an anonymized TCP connection is closed, or an edge node
- encounters error on any stream, it sends a 'RELAY_END' cell along the
- circuit (if possible) and closes the TCP connection immediately. If
- an edge node receives a 'RELAY_END' cell for any stream, it closes
- the TCP connection completely, and sends nothing more along the
- circuit for that stream.
-
- The payload of a RELAY_END cell begins with a single 'reason' byte to
- describe why the stream is closing, plus optional data (depending on
- the reason.) The values are:
-
- 1 -- REASON_MISC (catch-all for unlisted reasons)
- 2 -- REASON_RESOLVEFAILED (couldn't look up hostname)
- 3 -- REASON_CONNECTREFUSED (remote host refused connection) [*]
- 4 -- REASON_EXITPOLICY (OR refuses to connect to host or port)
- 5 -- REASON_DESTROY (Circuit is being destroyed)
- 6 -- REASON_DONE (Anonymized TCP connection was closed)
- 7 -- REASON_TIMEOUT (Connection timed out, or OR timed out
- while connecting)
- 8 -- (unallocated) [**]
- 9 -- REASON_HIBERNATING (OR is temporarily hibernating)
- 10 -- REASON_INTERNAL (Internal error at the OR)
- 11 -- REASON_RESOURCELIMIT (OR has no resources to fulfill request)
- 12 -- REASON_CONNRESET (Connection was unexpectedly reset)
- 13 -- REASON_TORPROTOCOL (Sent when closing connection because of
- Tor protocol violations.)
- 14 -- REASON_NOTDIRECTORY (Client sent RELAY_BEGIN_DIR to a
- non-directory server.)
-
- (With REASON_EXITPOLICY, the 4-byte IPv4 address or 16-byte IPv6 address
- forms the optional data; no other reason currently has extra data.
- As of 0.1.1.6, the body also contains a 4-byte TTL.)
-
- OPs and ORs MUST accept reasons not on the above list, since future
- versions of Tor may provide more fine-grained reasons.
-
- [*] Older versions of Tor also send this reason when connections are
- reset.
- [**] Due to a bug in versions of Tor through 0095, error reason 8 must
- remain allocated until that version is obsolete.
-
- --- [The rest of this section describes unimplemented functionality.]
-
- Because TCP connections can be half-open, we follow an equivalent
- to TCP's FIN/FIN-ACK/ACK protocol to close streams.
-
- An exit connection can have a TCP stream in one of three states:
- 'OPEN', 'DONE_PACKAGING', and 'DONE_DELIVERING'. For the purposes
- of modeling transitions, we treat 'CLOSED' as a fourth state,
- although connections in this state are not, in fact, tracked by the
- onion router.
-
- A stream begins in the 'OPEN' state. Upon receiving a 'FIN' from
- the corresponding TCP connection, the edge node sends a 'RELAY_FIN'
- cell along the circuit and changes its state to 'DONE_PACKAGING'.
- Upon receiving a 'RELAY_FIN' cell, an edge node sends a 'FIN' to
- the corresponding TCP connection (e.g., by calling
- shutdown(SHUT_WR)) and changing its state to 'DONE_DELIVERING'.
-
- When a stream in already in 'DONE_DELIVERING' receives a 'FIN', it
- also sends a 'RELAY_FIN' along the circuit, and changes its state
- to 'CLOSED'. When a stream already in 'DONE_PACKAGING' receives a
- 'RELAY_FIN' cell, it sends a 'FIN' and changes its state to
- 'CLOSED'.
-
- If an edge node encounters an error on any stream, it sends a
- 'RELAY_END' cell (if possible) and closes the stream immediately.
-
-6.4. Remote hostname lookup
-
- To find the address associated with a hostname, the OP sends a
- RELAY_RESOLVE cell containing the hostname to be resolved. (For a reverse
- lookup, the OP sends a RELAY_RESOLVE cell containing an in-addr.arpa
- address.) The OR replies with a RELAY_RESOLVED cell containing a status
- byte, and any number of answers. Each answer is of the form:
- Type (1 octet)
- Length (1 octet)
- Value (variable-width)
- TTL (4 octets)
- "Length" is the length of the Value field.
- "Type" is one of:
- 0x00 -- Hostname
- 0x04 -- IPv4 address
- 0x06 -- IPv6 address
- 0xF0 -- Error, transient
- 0xF1 -- Error, nontransient
-
- If any answer has a type of 'Error', then no other answer may be given.
-
- The RELAY_RESOLVE cell must use a nonzero, distinct streamID; the
- corresponding RELAY_RESOLVED cell must use the same streamID. No stream
- is actually created by the OR when resolving the name.
-
-7. Flow control
-
-7.1. Link throttling
-
- Each node should do appropriate bandwidth throttling to keep its
- user happy.
-
- Communicants rely on TCP's default flow control to push back when they
- stop reading.
-
-7.2. Link padding
-
- Link padding can be created by sending PADDING cells along the
- connection; relay cells of type "DROP" can be used for long-range
- padding.
-
- Currently nodes are not required to do any sort of link padding or
- dummy traffic. Because strong attacks exist even with link padding,
- and because link padding greatly increases the bandwidth requirements
- for running a node, we plan to leave out link padding until this
- tradeoff is better understood.
-
-7.3. Circuit-level flow control
-
- To control a circuit's bandwidth usage, each OR keeps track of
- two 'windows', consisting of how many RELAY_DATA cells it is
- allowed to package for transmission, and how many RELAY_DATA cells
- it is willing to deliver to streams outside the network.
- Each 'window' value is initially set to 1000 data cells
- in each direction (cells that are not data cells do not affect
- the window). When an OR is willing to deliver more cells, it sends a
- RELAY_SENDME cell towards the OP, with Stream ID zero. When an OR
- receives a RELAY_SENDME cell with stream ID zero, it increments its
- packaging window.
-
- Each of these cells increments the corresponding window by 100.
-
- The OP behaves identically, except that it must track a packaging
- window and a delivery window for every OR in the circuit.
-
- An OR or OP sends cells to increment its delivery window when the
- corresponding window value falls under some threshold (900).
-
- If a packaging window reaches 0, the OR or OP stops reading from
- TCP connections for all streams on the corresponding circuit, and
- sends no more RELAY_DATA cells until receiving a RELAY_SENDME cell.
-[this stuff is badly worded; copy in the tor-design section -RD]
-
-7.4. Stream-level flow control
-
- Edge nodes use RELAY_SENDME cells to implement end-to-end flow
- control for individual connections across circuits. Similarly to
- circuit-level flow control, edge nodes begin with a window of cells
- (500) per stream, and increment the window by a fixed value (50)
- upon receiving a RELAY_SENDME cell. Edge nodes initiate RELAY_SENDME
- cells when both a) the window is <= 450, and b) there are less than
- ten cell payloads remaining to be flushed at that edge.
-
-
-A.1. Differences between spec and implementation
-
-- The current specification requires all ORs to have IPv4 addresses, but
- allows servers to exit and resolve to IPv6 addresses, and to declare IPv6
- addresses in their exit policies. The current codebase has no IPv6
- support at all.
-
Deleted: tor/trunk/doc/version-spec.txt
===================================================================
--- tor/trunk/doc/version-spec.txt 2007-01-26 01:51:40 UTC (rev 9410)
+++ tor/trunk/doc/version-spec.txt 2007-01-26 01:59:50 UTC (rev 9411)
@@ -1,49 +0,0 @@
-$Id$
-
-HOW TOR VERSION NUMBERS WORK
-============================
-
-The Old Way
------------
-
-Before 0.1.0, versions were of the format:
- MAJOR.MINOR.MICRO(status(PATCHLEVEL))?(-cvs)?
-where MAJOR, MINOR, MICRO, and PATCHLEVEL are numbers, status is one
-of "pre" (for an alpha release), "rc" (for a release candidate), or
-"." for a release. As a special case, "a.b.c" was equivalent to
-"a.b.c.0". We compare the elements in order (major, minor, micro,
-status, patchlevel, cvs), with "cvs" preceding non-cvs.
-
-We would start each development branch with a final version in mind:
-say, "0.0.8". Our first pre-release would be "0.0.8pre1", followed by
-(for example) "0.0.8pre2-cvs", "0.0.8pre2", "0.0.8pre3-cvs",
-"0.0.8rc1", "0.0.8rc2-cvs", and "0.0.8rc2". Finally, we'd release
-0.0.8. The stable CVS branch would then be versioned "0.0.8.1-cvs",
-and any eventual bugfix release would be "0.0.8.1".
-
-The New Way
------------
-
-After 0.1.0, versions are of the format:
- MAJOR.MINOR.MICRO(.PATCHLEVEL)(-status_tag)
-The stuff in parenthesis is optional. As before, MAJOR, MINOR, MICRO,
-and PATCHLEVEL are numbers, with an absent number equivalent to 0.
-All versions should be distinguishable purely by those four
-numbers. The status tag is purely informational, and lets you know how
-stable we think the release is: "alpha" is pretty unstable; "rc" is a
-release candidate; and no tag at all means that we have a final
-release. If the tag ends with "-cvs" or "-dev", you're looking at a
-development snapshot that came after a given release. If we *do*
-encounter two versions that differ only by status tag, we compare them
-lexically.
-
-Now, we start each development branch with (say) 0.1.1.1-alpha. The
-patchlevel increments consistently as the status tag changes, for
-example, as in: 0.1.1.2-alpha, 0.1.1.3-alpha, 0.1.1.4-rc 0.1.1.5-rc,
-Eventually, we release 0.1.1.6. The next patch release is 0.1.1.7.
-
-Between these releases, CVS is versioned with a -cvs tag: after
-0.1.1.1-alpha comes 0.1.1.1-alpha-cvs, and so on. But starting with
-0.1.2.1-alpha-dev, we switched to SVN and started using the "-dev"
-suffix instead of the "-cvs" suffix.
-
More information about the tor-commits
mailing list