[or-cvs] r10774: specified and implemented basic functionality for distributi (in tor/branches/114-dist-storage: doc doc/spec doc/spec/proposals src/common src/or)
kloesing at seul.org
kloesing at seul.org
Mon Jul 9 15:13:20 UTC 2007
Author: kloesing
Date: 2007-07-09 11:13:19 -0400 (Mon, 09 Jul 2007)
New Revision: 10774
Modified:
tor/branches/114-dist-storage/doc/spec/dir-spec.txt
tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt
tor/branches/114-dist-storage/doc/spec/rend-spec.txt
tor/branches/114-dist-storage/doc/tor.1.in
tor/branches/114-dist-storage/src/common/container.c
tor/branches/114-dist-storage/src/common/container.h
tor/branches/114-dist-storage/src/common/crypto.c
tor/branches/114-dist-storage/src/or/circuitbuild.c
tor/branches/114-dist-storage/src/or/circuitlist.c
tor/branches/114-dist-storage/src/or/circuituse.c
tor/branches/114-dist-storage/src/or/config.c
tor/branches/114-dist-storage/src/or/connection_edge.c
tor/branches/114-dist-storage/src/or/directory.c
tor/branches/114-dist-storage/src/or/dirserv.c
tor/branches/114-dist-storage/src/or/dirvote.c
tor/branches/114-dist-storage/src/or/dns.c
tor/branches/114-dist-storage/src/or/dnsserv.c
tor/branches/114-dist-storage/src/or/main.c
tor/branches/114-dist-storage/src/or/or.h
tor/branches/114-dist-storage/src/or/rendclient.c
tor/branches/114-dist-storage/src/or/rendcommon.c
tor/branches/114-dist-storage/src/or/rendservice.c
tor/branches/114-dist-storage/src/or/router.c
tor/branches/114-dist-storage/src/or/routerlist.c
tor/branches/114-dist-storage/src/or/routerparse.c
tor/branches/114-dist-storage/src/or/test.c
Log:
specified and implemented basic functionality for distributing service descriptors
Modified: tor/branches/114-dist-storage/doc/spec/dir-spec.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/dir-spec.txt 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/doc/spec/dir-spec.txt 2007-07-09 15:13:19 UTC (rev 10774)
@@ -39,7 +39,7 @@
that served signed "directory" documents containing a list of signed
"router descriptors", along with short summary of the status of each
router. Thus, clients could get up-to-date information on the state of
- the network automatically, and be certain that they list they were getting
+ the network automatically, and be certain that the list they were getting
was attested by a trusted directory authority.
Later versions (0.0.8) added directory caches, which download
@@ -95,9 +95,10 @@
issues:
* A great deal of bandwidth used to transmit router descriptors was
- used by two fields that are not actually used by Tor routers. We
- save about 60% by moving them into a separate document that most
- clients do not fetch or use.
+ used by two fields that are not actually used by Tor routers
+ (namely read-history and write-history). We save about 60% by
+ moving them into a separate document that most clients do not
+ fetch or use.
* It was possible under certain perverse circumstances for clients
to download an unusual set of network status documents, thus
@@ -203,7 +204,7 @@
following lightweight extensible information format.
The highest level object is a Document, which consists of one or more
- Items. Every Item begins with a KeywordLine, followed by one or more
+ Items. Every Item begins with a KeywordLine, followed by zero or more
Objects. A KeywordLine begins with a Keyword, optionally followed by
whitespace and more non-newline characters, and ends with a newline. A
Keyword is a sequence of one or more characters in the set [A-Za-z0-9-].
@@ -324,20 +325,24 @@
descriptor.
In lines that take multiple arguments, extra arguments SHOULD be
- accepted and ignored.
+ accepted and ignored. Many of the nonterminals below are defined in
+ section 2.3.
"router" nickname address ORPort SOCKSPort DirPort NL
[At start, exactly once.]
- Indicates the beginning of a router descriptor. "address" must be an
- IPv4 address in dotted-quad format. The last three numbers indicate
- the TCP ports at which this OR exposes functionality. ORPort is a port
- at which this OR accepts TLS connections for the main OR protocol;
+ Indicates the beginning of a router descriptor. "nickname" must be a
+ valid router nickname as specified in 2.3. "address" must be an IPv4
+ address in dotted-quad format. The last three numbers indicate the
+ TCP ports at which this OR exposes functionality. ORPort is a port at
+ which this OR accepts TLS connections for the main OR protocol;
SOCKSPort is deprecated and should always be 0; and DirPort is the
port at which this OR accepts directory-related HTTP connections. If
any port is not supported, the value 0 is given instead of a port
- number.
+ number. (At least one of DirPort and ORPort SHOULD be set;
+ authorities MAY reject any descriptor with both DirPort and ORPort of
+ 0.)
"bandwidth" bandwidth-avg bandwidth-burst bandwidth-observed NL
@@ -471,14 +476,14 @@
[At most once]
Declare whether this version of Tor is using the newer enhanced
- dns logic. Versions of Tor without eventdns SHOULD NOT be used for
- reverse hostname lookups.
+ dns logic. Versions of Tor with this field set to false SHOULD NOT
+ be used for reverse hostname lookups.
[All versions of Tor before 0.1.2.2-alpha should be assumed to have
this option set to 0 if it is not present. All Tor versions at
0.1.2.2-alpha or later should be assumed to have this option set to
1 if it is not present. Until 0.1.2.1-alpha-dev, this option was
- not generated, even when eventdns was in use. Versions of Tor
+ not generated, even when the new DNS code was in use. Versions of Tor
before 0.1.2.1-alpha-dev did not parse this option, so it should be
marked "opt". The dnsworker logic has been removed, so this option
should not be used by new server code. However, it can still be
@@ -844,6 +849,7 @@
circuits.
"Fast" if the router is suitable for high-bandwidth circuits.
"Guard" if the router is suitable for use as an entry guard.
+ "HSDir" if the router is a hidden service directory.
"Named" if the router's identity-nickname mapping is canonical,
and this authority binds names.
"Stable" if the router is suitable for long-lived circuits.
@@ -932,6 +938,10 @@
supports the functionality clients need. (Currently, this is
0.2.0.?????-alpha or later.)
+ "HSDir" -- A router is a hidden service directory if it stores and serves
+ v2 hidden service descriptors and the authority managed to connect to it
+ successfully within the last 24 hours.
+
Directory server administrators may label some servers or IPs as
blacklisted, and elect not to include them in their network-status lists.
@@ -987,7 +997,7 @@
The signatures at the end of a consensus document are sorted in
ascending order by identity digest.
-3.4. Detached signatures
+3.5. Detached signatures
Assuming full connectivity, every authority should compute and sign the
same consensus directory in each period. Therefore, it isn't necessary to
Modified: tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1,7 +1,7 @@
Filename: 114-distributed-storage.txt
Title: Distributed Storage for Tor Hidden Service Descriptors
-Version: $Revision: 10624 $
-Last-Modified: $Date: 2007-06-16 23:58:33 +0100 (Sa, 16 Jun 2007) $
+Version: $Revision: 10773 $
+Last-Modified: $Date: 2007-07-09 16:54:48 +0200 (Mo, 09 Jul 2007) $
Author: Karsten Loesing
Created: 13-May-2007
Status: Open
@@ -11,33 +11,35 @@
13-May-2007 Initial proposal
14-May-2007 Added changes suggested by Lasse Overlier
30-May-2007 Changed descriptor format, key length discussion, typos
+ 09-Jul-2007 Incorporated suggestions by Roger, added status of specification
+ and implementation for upcoming GSoC mid-term evaluation
Overview:
The basic idea of this proposal is to distribute the tasks of storing and
serving hidden service descriptors from currently three authoritative
- directory nodes among a large subset of all onion routers. The two reasons
- to do this are better scalability and improved security properties. Further,
+ directory nodes among a large subset of all onion routers. The two reasons to
+ do this are better scalability and improved security properties. Further,
this proposal suggests changes to the hidden service descriptor format to
- prevent new security threats coming from decentralization and to gain
- even better security properties.
-
+ prevent new security threats coming from decentralization and to gain even
+ better security properties.
+
Motivation:
The current design of hidden services exhibits the following performance and
security problems:
First, the three hidden service authoritative directories constitute a
- performance bottleneck in the system. The directory nodes are responsible
- for storing and serving all hidden service descriptors. At the moment there
- are about 1000 descriptors at a time, but this number is assumed to increase
- in the future. Further, there is no replication protocol for descriptors
- between the three directory nodes, so that hidden services must ensure the
+ performance bottleneck in the system. The directory nodes are responsible for
+ storing and serving all hidden service descriptors. At the moment there are
+ about 1000 descriptors at a time, but this number is assumed to increase in
+ the future. Further, there is no replication protocol for descriptors between
+ the three directory nodes, so that hidden services must ensure the
availability of their descriptors by manually publishing them on all
directory nodes. Whenever a fourth or fifth hidden service authoritative
directory is added, hidden services will need to maintain an equally
- increasing number of replicas. These scalability issues have an impact on
- the current usage of hidden services and put an even higher burden on the
+ increasing number of replicas. These scalability issues have an impact on the
+ current usage of hidden services and put an even higher burden on the
development of new kinds of applications for hidden services that might
require storing even bigger numbers of descriptors.
@@ -48,48 +50,52 @@
descriptor contents to determine which onion routers work as introduction
points for a given hidden service and need to be attacked or threatened to
shut it down. Furthermore, the contents of a hidden service descriptor offer
- only minimal security properties to the hidden service. Whoever gets aware
- of the service ID can easily find out whether the service is active at the
+ only minimal security properties to the hidden service. Whoever gets aware of
+ the service ID can easily find out whether the service is active at the
moment and which introduction points it has. This applies to (former)
clients, (former) introduction points, and of course to the directory nodes.
- It requires only to request the descriptor for the given service ID --- which
+ It requires only to request the descriptor for the given service ID, which
can be performed by anyone anonymously.
This proposal suggests two major changes to approach the described
performance and security problems:
- The first change affects the storage location for hidden service
- descriptors. Descriptors are distributed among a large subset of all onion
- routers instead of three fixed directory nodes. Each storing node is
- responsible for a subset of descriptors for a limited time only. It is not
- able to choose which descriptors it stores at a certain time, because this
- is determined by its onion ID which is hard to change frequently and in time
- (only routers which are stable for a given time are accepted as storing
- nodes). In order to resist single node failures and untrustworthy nodes,
- descriptors are replicated among a certain number of storing nodes. A simple
- replication protocol makes sure that descriptors don't get lost when the
- node population changes. Therefore, a storing node periodically requests the
- descriptors from its siblings. Connections to storing nodes are established
- by extending existing circuits by one hop to the storing node. This also
- ensures that contents are encrypted. The effect of this first change is that
- the probability that a single node operator learns about a certain hidden
- service is very small and that it is very hard to track a service over time,
- even when it collaborates with other node operators.
-
+ The first change affects the storage location for hidden service descriptors.
+ Descriptors are distributed among a large subset of all onion routers instead
+ of three fixed directory nodes. Each storing node is responsible for a subset
+ of descriptors for a limited time only. It is not able to choose which
+ descriptors it stores at a certain time, because this is determined by its
+ onion ID which is hard to change frequently and in time (only routers which
+ are stable for a given time are accepted as storing nodes). In order to
+ resist single node failures and untrustworthy nodes, descriptors are
+ replicated among a certain number of storing nodes. A first replication
+ protocol makes sure that descriptors don't get lost when the node population
+ changes; therefore, a storing node periodically requests the descriptors from
+ its siblings. A second replication protocol distributes descriptors among
+ non-consecutive nodes of the ID ring to prevent a group of adversaries from
+ generating new onion keys until they have consecutive IDs to create a 'black
+ hole' in the ring and make random services unavailable. Connections to
+ storing nodes are established by extending existing circuits by one hop to
+ the storing node. This also ensures that contents are encrypted. The effect
+ of this first change is that the probability that a single node operator
+ learns about a certain hidden service is very small and that it is very hard
+ to track a service over time, even when it collaborates with other node
+ operators.
+
The second change concerns the content of hidden service descriptors.
- Obviously, security problems cannot be solved only by decentralizing
- storage; in fact, they could also get worse if done without caution. At
- first, a descriptor ID needs to change periodically in order to be stored on
- changing nodes over time. Next, the descriptor ID needs to be computable only
- for the service's clients, but should be unpredictable for all other nodes.
- Further, the storing node needs to be able to verify that the hidden service
- is the true originator of the descriptor with the given ID even though it is
- not a client. Finally, a storing node should learn as little information as
+ Obviously, security problems cannot be solved only by decentralizing storage;
+ in fact, they could also get worse if done without caution. At first, a
+ descriptor ID needs to change periodically in order to be stored on changing
+ nodes over time. Next, the descriptor ID needs to be computable only for the
+ service's clients, but should be unpredictable for all other nodes. Further,
+ the storing node needs to be able to verify that the hidden service is the
+ true originator of the descriptor with the given ID even though it is not a
+ client. Finally, a storing node should learn as little information as
necessary by storing a descriptor, because it might not be as trustworthy as
a directory node; for example it does not need to know the list of
- introduction points. Therefore, a second key is applied that is only known
- to the hidden service provider and its clients and that is not included in
- the descriptor. It is used to calculate descriptor IDs and to encrypt the
+ introduction points. Therefore, a second key is applied that is only known to
+ the hidden service provider and its clients and that is not included in the
+ descriptor. It is used to calculate descriptor IDs and to encrypt the
introduction points. This second key can either be given to all clients
together with the hidden service ID, or to a group or a single client as
authentication token. In the future this second key could be the result of
@@ -99,166 +105,465 @@
Design:
- The proposed design is described by the changes that are necessary to the
- current design. Changes are grouped by content, rather than by affected
- specification documents.
+ The proposed design is described by the required changes to the current
+ design. These requirements are grouped by content, rather than by affected
+ specification documents or code files, and numbered for reference below.
- Tor clients and servers:
+ Hidden service clients, servers, and directories:
- All participants can combine the network status lists received from
- all directory authorities to one routing list containing only those
- servers that store and serve hidden service descriptors and which
- are contained in the majority of network status lists. A participant
- only trusts its own routing list and never learns about routing
- information from other parties. This list should only be created
- on demand by Tor clients and servers that are involved in the new
- hidden service protocol, i.e. hidden service directory node, hidden
- service provider, and hidden service client.
+ /1/ Create routing list
- All parties that are involved in the new hidden service protocol calculate
- the clock skew between their local time and the times of directory
- authorities. If the clock skew exceeds 1 minute (as opposed to 30 minutes
- as in the current implementation), the user is warned upon performing the
- first operation that is related to hidden services. However, the local
- time is not adjusted automatically, because then they would be open
- to attacks based on false times from directory authorities.
+ All participants can filter the consensus status document received from the
+ directory authorities to one routing list containing only those servers
+ that store and serve hidden service descriptors and which are running for
+ at least 24 hours. A participant only trusts its own routing list and never
+ learns about routing information from other parties.
+ - rend-spec.txt, section 1.4: Added description of how to obtain a routing
+ list of hidden service directories.
+
+ - routerparse.c: Changed routerstatus_parse_entry_from_string to parse the
+ "HSDir" flag in vote and consensus status documents.
+ - routerlist.c: Changed router_get_routerlist() to initialize routing list.
+ - or.h: Added hs_dirs member to routerlist_t.
+
+ [July 9: Specified and running, though the routing list is compiled for
+ each request anew.]
+
+ /2/ Determine responsible hidden service directory
+
+ All participants can determine the hidden service directory that is
+ responsible for storing and serving a given ID, as well as the hidden
+ service directories that replicate its content. Every hidden service
+ directory is responsible for the descriptor IDs in the interval from
+ its predecessor, exclusive, to its own ID, inclusive. Further, a hidden
+ service directory holds replicas for its n predecessors, where n denotes
+ the number of consecutive replicas. (requires /1/)
+
+ - rend-spec.txt, section 1.4: Added description of how to determine the
+ responsible node(s) for a given descriptor ID.
+
+ - routerlist.c: Added get_responsible_hs_dir() to determine the router that
+ is responsible for a given descriptor ID.
+ - container.h: Added prototype for smartlist_digest_next_circular().
+ - container.c: Added implementation for smartlist_digest_next_circular().
+
+ [July 9: Specified and running.]
+
+ Hidden service clients and providers:
+
+ /3/ Send tunneled HTTP request to hidden service directory in BEGIN_DIR cell
+
+ - rend-spec.txt, section 1.4: Added the requirement that requests need to
+ be sent via Tor.
+ - rend-spec.txt, section 1.6: Added the requirement that requests need to
+ be sent via Tor.
+
+ [July 9: Pending]
+
Hidden service directory nodes:
- Every onion router can decide whether it wants to store and serve hidden
- service descriptors by setting a new config option HiddenServiceDirectory
- 0|1 to 1. This option should be 1 by default for those onion routers that
- have their directory port open, because the smaller the group of storing
- nodes is, the poorer the security properties are.
+ /4/ Process tunneled HTTP request in BEGIN_DIR cell
- HS directory nodes include the fact that they store and serve hidden
- service descriptors in router descriptors that they send to directory
- authorities.
+ - rend-spec.txt, section 3.2: Added the requirement that requests need to
+ be contained within BEGIN_DIR cells.
+ - rend-spec.txt, section 3.3: Added the requirement that requests need to
+ be contained within BEGIN_DIR cells.
+
+ [July 9: Pending]
+
+ /5/ Advertise hidden service directory functionality
- HS directory nodes accept publish and fetch requests for hidden service
- descriptors and store/retrieve them to/from their local memory. (It is not
- necessary to make descriptors persistent, because after disconnecting, the
- onion router would not be accepted as storing node anyway, because it is
- not stable.) All requests and replies are formatted as HTTP messages.
- Requests are directed to the router's directory port and are contained
- within BEGIN_DIR cells. A HS directory node stores a descriptor only when
- it thinks that it is responsible for storing that descriptor based on its
- own routing table. Every HS directory node is responsible for the
- descriptor IDs in the interval of its n-th predecessor in the ID circle up
- to its own ID (n denotes the number of replicas).
+ Every onion router that has its directory port open can decide whether it
+ wants to store and serve hidden service descriptors by setting a new config
+ option "HSDir" 0|1 to 1. An onion router with this config option being set
+ includes the flag "hidden-service-dir" in its router descriptors that it
+ sends to directory authorities.
- A HS directory node replicates descriptors for which it is responsible by
- downloading them from other HS directory nodes. Therefore, it checks its
- routing table periodically every 10 minutes for changes. Whenever it
- realizes that a predecessor has left the network, it establishes a
- connection to the new n-th predecessor and requests its stored descriptors
- in the interval of its (n+1)-th predecessor and the requested n-th
- predecessor. Whenever it realizes that a new onion router has joined with
- an ID higher than its former n-th predecessor, it adds it to its
- predecessors and discards all descriptors in the interval of its (n+1)-th
- and its n-th predecessor.
+ - tor.1.in: Added the config option HSDir.
+ - dir-spec.txt, section 2.1: Added the flag hidden-service-dir to the
+ router descriptor format.
+ - rend-spec.txt, section 3.1: Added process of configuring a hidden service
+ directory.
+ - router.c: Changed router_dump_router_to_string() to include the
+ hidden-service-dir flag in a router descriptor if configured.
+ - or.h: Added HSDir to or_options_t.
+ - config.c: Added config option HSDir.
+
+ [July 9: Specified and running.]
+
+ /6/ Accept v2 publish requests, parse and store v2 descriptors
+
+ Hidden service directory nodes accept publish requests for hidden service
+ descriptors and store them to their local memory. (It is not necessary to
+ make descriptors persistent, because after disconnecting, the onion router
+ would not be accepted as storing node anyway, because it has not been
+ running for at least 24 hours.) All requests and replies are formatted as
+ HTTP messages. Requests are directed to the router's directory port and are
+ contained within BEGIN_DIR cells. A hidden service directory node stores a
+ descriptor only when it thinks that it is responsible for storing that
+ descriptor based on its own routing table. Every hidden service directory
+ node is responsible for the descriptor IDs in the interval of its n-th
+ predecessor in the ID circle up to its own ID (n denotes the number of
+ consecutive replicas). (requires /1/ and /4/)
+
+ - rend-spec.txt, section 1.2: Added the new v2 hidden service descriptor
+ format.
+ - rend-spec.txt, section 3.2: Added the acceptance of v2 publish requests.
+
+ - routerparse.c: Added rend_parse_v2_service_descriptor() to parse a v2
+ hidden service descriptor.
+ - routerparse.c: Added desc_token_table[] to parse v2 hidden service
+ descriptors.
+ - routerparse.c: Added 8 keywords to directory_keyword to parse v2 hidden
+ service descriptors.
+ - rendcommon.c: Added rend_cache_store_v2_dir() to allow a hidden service
+ directory to store a v2 descriptor in the local cache under its
+ descriptor ID instead of its service ID.
+ - rendcommon.c: Moved the parsing part from rend_cache_store() to the new
+ function rend_cache_store_parse() to reuse it for v2 descriptors.
+ - or.h: Added constant REND_DESC_ID_V2_LEN to reflect that v2 descriptor
+ IDs are longer than v0/1 onion addresses.
+
+ [July 9: Base version specified and running; no checking of published
+ descriptors, tunneling over BEGIN_DIR cells not yet implemented.]
+
+ /7/ Accept v2 fetch requests
+
+ Same as /6/, but with fetch requests for hidden service descriptors.
+ (requires /4/)
+
+ - rend-spec.txt, section 3.3: Added the processing of v2 fetch requests.
+
+ - rendcommon.c: Added rend_cache_lookup_v2_dir() to allow a hidden service
+ directory to look up a v2 descriptor in the local cache under its
+ descriptor ID instead of its service ID.
+ - or.h: Added constant REND_DESC_ID_V2_LEN to reflect that v2 descriptor
+ IDs are longer than v0/1 onion addresses.
+
+ [July 9: Base version specified and running; tunneling over BEGIN_DIR
+ cells not yet implemented.]
+
+ /8/ Replicate descriptors with neighbors
+
+ A hidden service directory node replicates descriptors from its two
+ predecessors by downloading them once an hour. Further, it checks its
+ routing table periodically for changes. Whenever it realizes that a
+ predecessor has left the network, it establishes a connection to the new
+ n-th predecessor and requests its stored descriptors in the interval of its
+ (n+1)-th predecessor and the requested n-th predecessor. Whenever it
+ realizes that a new onion router has joined with an ID higher than its
+ former n-th predecessor, it adds it to its predecessors and discards all
+ descriptors in the interval of its (n+1)-th and its n-th predecessor.
+ (requires /1/)
+
+ - rend-spec.txt, section 3.3: Added the replication of v2 descriptors.
+
+ [July 9: To some extend specified, but not yet implemented.]
+
Authoritative directory nodes:
- Directory nodes include a new flag for routers that decided to provide
- storage for hidden service descriptors and that are stable for a given
- time. The requirement to be stable prevents a node from frequently
- changing its onion key to become responsible for an identifier it wants
- to target.
+ /9/ Confirm a router's hidden service directory functionality
+ Directory nodes include a new flag "HSDir" for routers that decided to
+ provide storage for hidden service descriptors and that are running for at
+ least 24 hours. The last requirement prevents a node from frequently
+ changing its onion key to become responsible for an identifier it wants to
+ target.
+
+ - dir-spec.txt, section 3.2: Added the status flag "HSDir" to the vote and
+ consensus status document format.
+ - dir-spec.txt, section 3.3: Added a rule for how an authority decides
+ whether a router is assigned the flag "HSDir".
+ - rend-spec.txt, section 3.1: Added the decision on whether an onion router
+ is confirmed to act as hidden service directory or not.
+
+ - routerparse.c: Changed router_parse_entry_from_string() to parse the
+ "hidden-service-dir" flag in router descriptors.
+ - routerparse.c: Added an entry to routerdesc_token_table[] to parse the
+ "hidden-service-directory" flag in router descriptors.
+ - routerparse.c: Added 1 keyword to directory_keyword to parse the
+ "hidden-service-dir" flag in router descriptors.
+ - or.h: Added is_hs_dir member to routerinfo_t and to routerstatus_t.
+ - dirserv.c: Changed routerstatus_format_entry() to include the "HSDir"
+ flag in vote and consensus status documents.
+ - dirserv.c: Changed set_routerstatus_from_routerinfo() to set the "HSDir"
+ flag.
+
+ [July 9: Base version specified and running in which all nodes that have
+ the hidden-service-dir flag set in their router descriptor get the
+ HSDir flag, not only those which are running for at least 24 hours.]
+
Hidden service provider:
- When setting up the hidden service at introduction points, a hidden service
- provider does not pass its own public key, but the public key of a freshly
- generated key pair. It also includes this public key in the hidden service
- descriptor together with the other introduction point information. The
- reason is that the introduction point does not need to know for which
- hidden service it works, and should not know it to prevent it from
- tracking the hidden service's activity.
+ /10/ Configure v2 hidden service
- Each hidden service provider publishes a new descriptor whenever
- its content
- changes or a new publication period starts for this descriptor. If the
- current publication period would only last for less than 60 minutes, the
+ Each hidden service provider that has set the config option
+ "PublishV2HidServDescriptors" 0|1 to 1 is configured to publish v2
+ descriptors and conform to the v2 connection establishment protocol. When
+ configuring a hidden service, a hidden service provider checks if it has
+ already created a random secret_cookie and a hostname2 file; if not, it
+ creates both of them. (requires /2/)
+
+ - tor.1.in: Added the config option PublishV2HidServDescriptors.
+ - tor.1.in: Added the files hostname2 and secret_cookie.
+ - rend-spec.txt, section 1.1: Added requirement to create secret_cookie and
+ hostname2 file.
+
+ - rendservice.c: Added rend_get_hostname2() to assemble a v2 onion address.
+ - rendservice.c: Changed rend_service_load_keys() to write a secret_cookie
+ and a hostname2 file.
+ - rendservice.c: Extended rend_service_t by a member secret_cookie.
+ - or.h: Added PublishV2HidServDescriptors to or_options_t.
+ - config.c: Added config option PublishV2HidServDescriptors.
+
+ [July 9: Specified and running.]
+
+ /11/ Establish introduction points with fresh key
+
+ If configured to publish only v2 descriptors and no v0/v1 descriptors any
+ more, a hidden service provider that is setting up the hidden service at
+ introduction points does not pass its own public key, but the public key
+ of a freshly generated key pair. It also includes these fresh public keys
+ in the hidden service descriptor together with the other introduction point
+ information. The reason is that the introduction point does not need to and
+ therefore should not know for which hidden service it works, so as to
+ prevent it from tracking the hidden service's activity. (If a hidden
+ service provider supports both, v0/v1 and v2 descriptors, v0/v1 clients
+ rely on the fact that all introduction points accept the same public key,
+ so that this new feature cannot be used.)
+
+ - rend-spec.txt, section 1.3: Instead of Bob's public key, the hidden
+ service provider uses a freshly generated public key for every
+ introduction point.
+
+ [July 9: Specified, but not yet implemented.]
+
+ /12/ Encode v2 descriptors and send v2 publish requests
+
+ If configured to publish v2 descriptors, a hidden service provider
+ publishes a new descriptor whenever its content changes or a new
+ publication period starts for this descriptor. If the current publication
+ period would only last for less than 60 minutes (= 2 x 30 minutes to allow
+ the server to be 30 minutes behind and the client 30 minutes ahead), the
hidden service provider publishes both a current descriptor and one for
the next period. Publication is performed by sending the descriptor to all
hidden service directories that are responsible for keeping replicas for
- the descriptor ID.
+ the descriptor ID. This includes two non-consecutive replicas that are
+ stored at 3 consecutive nodes each. (requires /1/ and /3/)
+ - rend-spec.txt, section 1.2: Added the new v2 hidden service descriptor
+ format.
+ - rend-spec.txt, section 1.4: Bob's OP does not only upload v0/v1 service
+ descriptors to the authoritative directories, but also v2 service
+ descriptors to the hidden service directories.
+
+ - rendservice.c: Changed upload_service_descriptor() to upload v2 hidden
+ service descriptors, if configured.
+ - rendservice.c: Changed rend_consider_services_upload() to also initiate
+ the upload of v2 descriptors, if configured.
+ - rendservice.c: Extended rend_service_t by a member secret_cookie.
+ - rendcommon.c: Added rend_compute_v2_descriptor_fields() to prepare the
+ encoding of a v2 descriptor.
+ - rendcommon.c: Added rend_encode_v2_descriptor() to encode a v2
+ descriptor.
+ - or.h: Added 7 new members to rend_service_descriptor_t to store
+ v2-specific information.
+ - or.h: Added constant DIR_PURPOSE_UPLOAD_RENDDESC_V2.
+ - directory.c: Added directory_post_to_hs_dir().
+ - directory.c: Changed directory_initiate_command() to also recognize v2
+ publish requests.
+ - directory.c: Changed directory_send_command() to also prepare v2 publish
+ requests.
+ - directory.c: Changed directory_handle_command_post() to handle v2 publish
+ requests.
+ - crypto.c: Added implementation for crypto_cipher_encrypt_cbc().
+
+ [July 9: Base version specified and running; yet, replication is not
+ implemented, republication does not depend on publication periods, yet.]
+
Hidden service client:
- Instead of downloading descriptors from a hidden service authoritative
- directory, a hidden service client downloads it from a randomly chosen
- hidden service directory that is responsible for keeping replica for the
- descriptor ID.
+ /13/ Send v2 fetch requests
- When contacting an introduction point, the client does not use the
- public key of the hidden service provider, but the freshly-generated public
- key that is included in the hidden service descriptor.
+ A hidden service client that has set the config option
+ "FetchV2HidServDescriptors" 0|1 to 1 handles SOCKS requests for v2 onion
+ addresses by requesting a v2 descriptor from a randomly chosen hidden
+ service directory that is responsible for keeping replica for the
+ descriptor ID. In total there are six replicas of which the first and the
+ last three are stored on consecutive nodes. The probability of picking one
+ of the three consecutive replicas is 1/6, 2/6, and 3/6 to incorporate the
+ fact that the availability will be the highest on the node with next higher
+ ID. A hidden service client relies on the hidden service provider to store
+ two sets of descriptors to compensate clock skew between service and
+ client. (requires /1/, /2/, and /3/)
- Hidden service descriptor:
+ - tor.1.in: Added the config option FetchV2HidServDescriptors.
+ - rend-spec.txt, section 1.5: Added the new v2 onion address format.
+ - rend-spec.txt, section 1.6: Alice's OP downloads the service descriptors
+ similarly as Bob's OP uploaded them in 1.4.
- The descriptor ID needs to change periodically in order for the descriptor
- to be stored on changing nodes over time. It further may only be computable
- by a hidden service provider and all of his clients to prevent unauthorized
- nodes from tracking the service activity by periodically checking whether
- there is a descriptor for this service. Finally, the hidden service
- directory needs to be able to verify that the hidden service provider is
- the true originator of the descriptor with the given ID. Therefore, the
- ID is derived from the public key of the hidden service provider, the
- current time period, and a shared secret between hidden service provider
- and clients. Only the hidden service provider and the clients are able to
- generate future IDs, but together with the descriptor content the hidden
- service directory is able to verify its origin. The formula for calculating
- a descriptor ID is as follows:
+ - rendcommon.c: Changed rend_cache_lookup_entry to enable it to also lookup
+ v2 descriptors.
+ - rendcommon.c: Added rend_compute_desc_id() to generate v2 descriptor IDs
+ from v2 onion addresses.
+ - rendcommon.c: Changed rend_valid_service_id() to also consider v2 onion
+ addresses as valid and return the version number of the request (1 or 2).
+ - rendclient.c: Added rend_client_refetch_v2_renddesc() to fetch v2 service
+ descriptors using the secret cookie.
+ - rendclient.c: Changed rend_client_remove_intro_point() to copy the secret
+ cookie if the local descriptor has expired or there are no introduction
+ points left.
+ - or.h: Added FetchV2HidServDescriptors to or_options_t.
+ - or.h: Added constant REND_DESC_ID_V2_LEN to reflect that v2 descriptor
+ IDs are longer than v0/1 onion addresses.
+ - or.h: Added constant DIR_PURPOSE_FETCH_RENDDESC_V2.
+ - directory.c: Added directory_get_from_hs_dir().
+ - directory.c: Changed directory_initiate_command() to also recognize v2
+ fetch requests.
+ - directory.c: Changed directory_send_command() to also prepare v2 fetch
+ requests.
+ - directory.c: Changed directory_handle_command_get() to handle v2 fetch
+ requests.
+ - connection_edge.c: Changed connection_ap_handshake_rewrite_and_attach()
+ to fetch v2 service descriptors.
+ - connection_edge.c: Changed parse_extended_hostname() to accept both,
+ current and v2 onion addresses.
+ - config.c: Added config options FetchV2HidServDescriptors.
- descriptor-id = h(permanent-id + h(time-period + cookie))
+ [July 9: Base version specified and running in which only one node is
+ responsible for a specific descriptor ID.]
- "permanent-id" is the hashed value of the public key of the hidden service
- provider, "time-period" is a periodically changing value, e.g. the current
- date, and "cookie" is a shared secret between the hidden service provider
- and its clients. (The "time-period" should be constructed in a way that
- periods do not change at the same moment for all descriptors by including
- the "permanent-id" in the construction.) Amongst other things, the
- descriptor contains the public key of the hidden service provider, the
- value of h(time-period + cookie), and the signature of the descriptor
- content with the private key of the hidden service provider.
+ /14/ Process v2 fetch reply and parse v2 descriptors
- The introduction points that are included in the descriptor are encrypted
- using a key that is derived from the same shared key that is used to
- generate the descriptor ID. [correction to use another key than
- h(time-period + cookie) as encryption key for introduction points made by
- LO]
+ A hidden service client that has sent a request for a v2 descriptor can
+ parse it and store it to the local cache of rendezvous service descriptors.
- A new text-based format is proposed for descriptors instead of an
- extension of the existing binary format for reasons of future
- extensibility.
+ - rend-spec.txt, section 1.2: Added the new v2 hidden service descriptor
+ format.
+ - rend-spec.txt, section 1.6: Alice's OP parses the reply received from the
+ hidden service directory.
- The complete hidden service descriptor format looks like this:
+ - routerparse.c: Added rend_parse_v2_service_descriptor() to parse a v2
+ hidden service descriptor.
+ - routerparse.c: Added rend_decrypt_introduction_points() to decrypt and
+ parse the list of introduction points.
+ - routerparse.c: Added ipo_token_table[] to parse the decrypted
+ introduction points of v2 hidden service descriptors.
+ - routerparse.c: Added desc_token_table[] to parse v2 hidden service
+ descriptors.
+ - routerparse.c: Added 8 to directory_keyword to parse v2 hidden service
+ descriptors, and 5 to parse the decrypted list of introduction points.
+ - rendcommon.c: Added rend_cache_store_v2_client() to parse a v2 descriptor
+ and parse the encrypted list of introduction points.
+ - or.h: Added secret_cookie to edge_connection_t, to dir_connection_t, and
+ to origin_circuit_t to be able to decrypt introduction points when
+ receiving a v2 descriptor.
+ - or.h: Added 7 new members to rend_service_descriptor_t to store
+ v2-specific information.
+ - directory.c: Changed connection_dir_client_reached_eof() to also parse v2
+ fetch replies.
+ - crypto.c: Added implementation for crypto_cipher_decrypt_cbc().
- {
- descriptor-id = h(permanent-id + h(time-period + cookie))
- permanent-public-key (with permanent-id = h(permanent-public-key))
- h(time-period + cookie)
- timestamp
- {
- list of intro points (ID, IP, onion port, onion key, service key)
- } encrypted with cookie
- } signed with permanent-private-key
+ [July 9: Specified and running.]
+ /15/ Establish connection to v2 hidden service
+
+ A hidden service client can establish a connection to a hidden service
+ using a v2 descriptor. This includes using the secret cookie for decrypting
+ the introduction points contained in the descriptor. When contacting an
+ introduction point, the client does not use the public key of the hidden
+ service provider, but the freshly-generated public key that is included in
+ the hidden service descriptor. Whether or not a fresh key is used instead
+ of the key of the hidden service depends on the available protocol versions
+ that are included in the descriptor; by this, connection establishment is
+ to a certain extend decoupled from fetching the descriptor.
+
+ - rend-spec.txt, section 1.8: Alice uses the public key that is included in
+ the descriptor instead of Bob's permanent service key.
+
+ - rendclient.c: Changed rend_client_introduction_acked() to copy the secret
+ cookie in case the introduction point denied the request.
+ - rendclient.c: Changed rend_client_remove_intro_point() to copy the secret
+ cookie if the local descriptor has expired or there are no introduction
+ points left.
+ - or.h: Added secret_cookie to edge_connection_t, to dir_connection_t, and
+ to origin_circuit_t to be able to decrypt introduction points when
+ receiving a v2 descriptor.
+ - or.h: Added 7 new members to rend_service_descriptor_t to store
+ v2-specific information.
+ - circuitlist.c: Changed _circuit_mark_for_close() to pass the secret
+ cookie to rend_client_remove_intro_point() when an intro circ has failed.
+ - circuituse.c: Changed circuit_get_open_circ_or_launch() to fetch a v2
+ descriptor with the secret cookie, if no descriptor is available, or copy
+ the secret cookie to the circuit, in case it dies later, so that it can
+ be used to fetch a new descriptor.
+
+ [July 9: Base version specified and running, but without fresh key.]
+
+ Hidden service descriptor:
+
+ (Requirements concerning the descriptor format are contained in /6/ and /7/.)
+
+ The new v2 hidden service descriptor format looks like this:
+
+ onion-address = h(public-key) + cookie
+ descriptor-id = h(h(public-key) + h(time-period + cookie))
+ descriptor-content = {
+ descriptor-id,
+ version,
+ public-key,
+ h(time-period + cookie),
+ timestamp,
+ protocol-versions,
+ { introduction-points } encrypted with cookie
+ } signed with private-key
+
+ The "descriptor-id" needs to change periodically in order for the
+ descriptor to be stored on changing nodes over time. It may only be
+ computable by a hidden service provider and all of his clients to prevent
+ unauthorized nodes from tracking the service activity by periodically
+ checking whether there is a descriptor for this service. Finally, the
+ hidden service directory needs to be able to verify that the hidden service
+ provider is the true originator of the descriptor with the given ID.
+
+ Therefore, "descriptor-id" is derived from the "public-key" of the hidden
+ service provider, the current "time-period" which changes every 24 hours,
+ and a secret "cookie" shared between hidden service provider and clients.
+ (The "time-period" is constructed in a way that time periods do not change
+ at the same moment for all descriptors by deriving a value between 0:00 and
+ 23:59 hours from "public-key" and making the descriptors of this hidden
+ service provider expire at that time of the day.) The "descriptor-id" is
+ defined to be 160 bits long. [extending the "descriptor-id" length
+ suggested by LO]
+
+ Only the hidden service provider and the clients are able to generate
+ future "descriptor-ID"s. Hence, the "onion-address" is extended from now
+ the hash value of "public-key" by the secret "cookie". The "public-key" is
+ determined to be 80 bits long, whereas the "cookie" is dimensioned to be
+ 120 bits long. This makes a total of 200 bits or 40 base32 chars, which is
+ quite a lot to handle for a human, but necessary to provide sufficient
+ protection against an adversary from generating a key pair with same
+ "public-key" hash or guessing the "cookie".
+
A hidden service directory can verify that a descriptor was created by the
- hidden service provider by checking if the descriptor-id corresponds to
- the permanent-public-key and if the signature can be verified with the
- permanent-public-key.
+ hidden service provider by checking if the "descriptor-id" corresponds to
+ the "public-key" and if the signature can be verified with the
+ "public-key".
- A client can download the descriptor by creating the same descriptor-id
- and verify its origin by performing the same operations as the hidden
- service directory.
+ The "introduction-points" that are included in the descriptor are encrypted
+ using the same "cookie" that is shared between hidden service provider and
+ clients. [correction to use another key than h(time-period + cookie) as
+ encryption key for introduction points made by LO]
+ A new text-based format is proposed for descriptors instead of an extension
+ of the existing binary format for reasons of future extensibility.
+
Security implications:
- The security implications of the proposed changes are grouped by the roles
- of nodes that could perform attacks or on which attacks could be performed.
+ The security implications of the proposed changes are grouped by the roles of
+ nodes that could perform attacks or on which attacks could be performed.
Attacks by authoritative directory nodes
@@ -271,12 +576,12 @@
Attacks by hidden service directory nodes
- A hidden service directory node could misuse a stored descriptor to track
- a hidden service's activity and usage pattern by clients. Though there is
- no countermeasure against this kind of attack, it is very expensive to
- track a certain hidden service over time. An attacker would need to run a
- large number of stable onion routers that work as hidden service directory
- nodes to have a good probability to become responsible for its changing
+ A hidden service directory node could misuse a stored descriptor to track a
+ hidden service's activity and usage pattern by clients. Though there is no
+ countermeasure against this kind of attack, it is very expensive to track a
+ certain hidden service over time. An attacker would need to run a large
+ number of stable onion routers that work as hidden service directory nodes
+ to have a good probability to become responsible for its changing
descriptor IDs. For each period, the probability is:
1-(N-c choose r)/(N choose r) for N-c>=r and 1 else with N as total
@@ -290,8 +595,8 @@
content. The client would detect a false descriptor, because it could not
contain a correct signature. But an old content or an empty reply could
confuse the client. Therefore, the countermeasure is to replicate
- descriptors among a small number of hidden service directories, e.g. 5.
- The probability of a group of collaborating nodes to make a hidden service
+ descriptors among a small number of hidden service directories. The
+ probability of a group of collaborating nodes to make a hidden service
completely unavailable is in each period:
(c choose r)/(N choose r) for c>=r and N>=r, and 0 else with N as total
@@ -328,93 +633,308 @@
directory nodes and attack them. There is nothing that could prevent them
from doing so, because honest clients need the full descriptor content to
establish a connection to the hidden service. At the moment, the only
- countermeasure against dishonest clients is to change the secret cookie
- and pass it only to the honest clients.
+ countermeasure against dishonest clients is to change the secret cookie and
+ pass it only to the honest clients.
+Compatibility:
+
+ The proposed design is meant to replace the current design for hidden service
+ descriptors and their storage in the long run.
+
+ There should be a first transition phase in which both, the current design
+ and the proposed design are served in parallel. Onion routers should start
+ serving as hidden service directories, and hidden service providers and
+ clients should make use of the new design if both sides support it. Hidden
+ service providers should be allowed to publish descriptors of the current
+ format in parallel, and authoritative directories should continue storing and
+ serving these descriptors.
+
+ After the first transition phase, hidden service providers should stop
+ publishing descriptors on authoritative directories, and hidden service
+ clients should not try to fetch descriptors from the authoritative
+ directories. However, the authoritative directories should continue serving
+ hidden service descriptors for a second transition phase. As of this point,
+ all v2 config options should be set to a default value of 1.
+
+ After the second transition phase, the authoritative directories should stop
+ serving hidden service descriptors.
+
Specification:
The proposed changes affect multiple sections in several specification
- documents that are only mentioned in the following. The detailed
- specification will follow as soon as the design decisions above are final.
+ documents that are only mentioned in the following. (As for now, all changes
+ to specification documents are limited to the SVN branch 114-dist-storage.)
- dir-spec-v2.txt
+ tor.1.in
- 2.1 The router descriptor format needs to include an additional flag to
- denote that a router is a hidden service directory.
+ Added the config options HSDir (/5/), PublishV2HidServDescriptors (/10/),
+ and FetchV2HidServDescriptors (/13/).
- 3 The network status format needs to be extended by a new status flag to
- denote that a router is a hidden service directory.
+ Added the files hostname2 and secret_cookie (/10/).
- 4 The sections on directory caches need to be extended by new sections for
- the operation of hidden service directories, including replication of
- descriptors.
+ dir-spec.txt
+ 2.1 Added the flag hidden-service-dir to the router descriptor format
+ (/5/).
+
+ 3.2 Added the status flag HSDir to the vote and consensus status
+ document format (/9/).
+
+ 3.3 Added a rule for how an authority decides whether a router is assigned
+ the flag HSDir (/9/).
+
rend-spec.txt
- 1.2 The new descriptor format needs to be added.
+ 0.4 Added history
+ 1.1 Added requirement to create secret_cookie and hostname2 file (/10/).
+
+ 1.2 Added the new v2 hidden service descriptor format (/6/, /12/ and
+ /14/).
+
1.3 Instead of Bob's public key, the hidden service provider uses a
- freshly generated public key for every introduction point.
+ freshly generated public key for every introduction point (/11/).
- 1.4 Bob's OP does not upload his service descriptor to the authoritative
- directories, but to the hidden service directories.
+ 1.4 Added description of how to obtain a routing list of hidden service
+ directories (/1/).
- 1.6 Alice's OP downloads the service descriptors similarly as Bob
- published them in 1.4.
+ 1.4 Added description of how to determine the responsible node(s) for a
+ given descriptor ID (/2/).
+ 1.4 Bob's OP does not only upload v0/v1 service descriptors to the
+ authoritative directories, but also v2 service descriptors to the hidden
+ service directories (/12/).
+
+ 1.4 Added the requirement that requests need to be sent via Tor (/3/).
+
+ 1.5 Added the new v2 onion address format (/13/).
+
+ 1.6 Added the requirement that requests need to be sent via Tor (/3/).
+
+ 1.6 Alice's OP downloads the service descriptors similarly as Bob's OP
+ uploaded them in 1.4 (/13/).
+
+ 1.6 Alice's OP parses the reply received from the hidden service directory
+ (/14/).
+
1.8 Alice uses the public key that is included in the descriptor instead
- of Bob's permanent service key.
+ of Bob's permanent service key (/15/).
- tor-spec.txt
+ 3.1: Added process of configuring a hidden service directory (/5/).
- 6.2.1 Directory streams need to be used for connections to hidden service
- directories.
+ 3.1: Added the decision on whether an onion router is confirmed to act as
+ hidden service directory or not (/9/).
-Compatibility:
+ 3.2: Added the requirement that requests need to be contained within
+ BEGIN_DIR cells (/4/).
- The proposed design is meant to replace the current design for hidden service
- descriptors and their storage in the long run.
+ 3.2: Added the acceptance of v2 publish requests (/6/).
- There should be a first transition phase in which both, the current design
- and the proposed design are served in parallel. Onion routers should start
- serving as hidden service directories, and hidden service providers and
- clients should make use of the new design if both sides support it. But
- hidden service providers should continue publishing descriptors of the
- current format, and authoritative directories should store and serve these
- descriptors.
+ 3.3: Added the requirement that requests need to be contained within
+ BEGIN_DIR cells (/4/).
+
+ 3.3: Added the processing of v2 fetch requests (/7/).
- After the first transition phase, hidden service providers should stop
- publishing descriptors on authoritative directories, and hidden service
- clients should not try to fetch descriptors from the authoritative
- directories. However, the authoritative directories should continue serving
- hidden service descriptors for a second transition phase.
+ 3.3: Added the replication of v2 descriptors (/8/).
- After the second transition phase, the authoritative directories should stop
- serving hidden service descriptors.
-
Implementation:
- There are three key lengths that might need some discussion:
+ The proposed changes affect the following changes in the source code. (As for
+ now, all changes to code are limited to the SVN branch 114-dist-storage.)
- 1) descriptor-id, formerly known as onion address: It is generated by OPs
- internally and used for storing and looking up descriptors. There is no
- need to remember a descriptor-id for a human. In order to reduce
- the success rate of collisions it could be extended to the full output
- of SHA-1 of 160 bits instead of 80 bits. [extending the descriptor-id
- length suggested by LO]
+ container.h
- 2) permanent-id: This is the first part of the onion address that a client
- passes to his OP. The overall onion address should be easy to memorize.
- Therefore, its overall length should only be extended from the existing
- 80 bits to as few bits as necessary. The length of the permanent-id has
- an influence on the probability that an adversary creates an own key
- pair that leads to the same descriptor-id in a given time-period as an
- honest service's key. 32 bits should provide sufficient protection to
- avoid collisions, given the fact that key generation is expensive and
- the attack needed to be performed for every time-period.
+ Added prototype for smartlist_digest_next_circular() (/2/).
- 3) cookie: This is the second part of the onion address that is passed to
- an OP. In order to provide confidentiality of introduction points, this
- secret key should have 128 bits. In total, this leads to an onion
- address of 160 bits instead of the current 80 bits.
+ container.c
+ Added implementation for smartlist_digest_next_circular() (/2/).
+
+ crypto.h
+
+ Added 3 prototypes according to the changes in crypto.c (various
+ requirements).
+
+ crypto.c
+
+ Added implementation for crypto_cipher_encrypt_cbc() (/12/).
+
+ Added implementation for crypto_cipher_decrypt_cbc() (/14/).
+
+ Added implementation for base32_decode() (various requirements).
+
+ circuitlist.c
+
+ Changed _circuit_mark_for_close() to pass the secret cookie to
+ rend_client_remove_intro_point() when an intro circ has failed (/15/).
+
+ circuituse.c
+
+ Changed circuit_get_open_circ_or_launch() to fetch a v2 descriptor with the
+ secret cookie, if no descriptor is available, or copy the secret cookie to
+ the circuit, in case it dies later, so that it can be used to fetch a new
+ descriptor (/15/).
+
+ config.c
+
+ Added config options FetchV2HidServDescriptors (/13/), HSDir (/5/), and
+ PublishV2HidServDescriptors (/10/).
+
+ connection_edge.c
+
+ Changed connection_ap_handshake_rewrite_and_attach() to fetch v2 service
+ descriptors (/13/).
+
+ Changed parse_extended_hostname() to accept both, current and v2 onion
+ addresses (/13/).
+
+ directory.c
+
+ Added directory_post_to_hs_dir() (/12/).
+
+ Added directory_get_from_hs_dir() (/13/).
+
+ Changed directory_initiate_command() to also recognize v2 publish (/12/)
+ and fetch (/13/) requests.
+
+ Changed directory_send_command() to also prepare v2 publish (/12/) and
+ fetch (/13/) requests.
+
+ Changed connection_dir_client_reached_eof() to also parse v2 fetch replies
+ (/14/).
+
+ Changed directory_handle_command_get() to handle v2 fetch requests (/13/).
+
+ Changed directory_handle_command_post() to handle v2 publish requests
+ (/12/).
+
+ dirserv.c
+
+ Changed routerstatus_format_entry() to include the "HSDir" flag in vote and
+ consensus status documents (/9/).
+
+ Changed set_routerstatus_from_routerinfo() to set the "HSDir" flag (/9/).
+
+ or.h
+
+ Added constants DIR_PURPOSE_UPLOAD_RENDDESC_V2 (/12/) and
+ DIR_PURPOSE_FETCH_RENDDESC_V2 (/13/).
+
+ Added constant REND_DESC_ID_V2_LEN to reflect that v2 descriptor IDs are
+ longer than v0/1 onion addresses (/6/, /7/, and /13/).
+
+ Added rend_version and secret_cookie to edge_connection_t, to
+ dir_connection_t, and to origin_circuit_t to be able to decrypt
+ introduction points when receiving a v2 descriptor (/14/ and /15/).
+
+ Added is_hs_dir member to routerinfo_t and to routerstatus_t (/9/).
+
+ Added hs_dirs member to routerlist_t (/1/).
+
+ Added FetchV2HidServDescriptors (/13/), HSDir (/5/), and
+ PublishV2HidServDescriptors (/10/) to or_options_t.
+
+ Added 7 new members to rend_service_descriptor_t to store v2-specific
+ information (/12/, /14/, and /15/).
+
+ Added 11 prototypes and changed the signature of 1 according to the
+ changes in .c files (various requirements).
+
+ rendclient.c
+
+ Changed rend_client_introduction_acked() to copy the secret cookie in case
+ the introduction point denied the request (/15/).
+
+ Added rend_client_refetch_v2_renddesc() to fetch v2 service descriptors
+ using the secret cookie (/13/).
+
+ Changed rend_client_remove_intro_point() to copy the secret cookie if the
+ local descriptor has expired or there are no introduction points left (/13/
+ and /15/).
+
+ rendcommon.c
+
+ Added rend_compute_v2_descriptor_fields() to prepare the encoding of a v2
+ descriptor (/12/).
+
+ Added rend_compute_desc_id() to generate v2 descriptor IDs from v2 onion
+ addresses (/13/).
+
+ Added rend_encode_v2_descriptor() to encode a v2 descriptor (/12/).
+
+ Changed rend_valid_service_id() to also consider v2 onion addresses as
+ valid and return the version number of the request (1 or 2) (/13/).
+
+ Changed rend_cache_lookup_entry to enable it to also lookup v2 descriptors
+ (/13/).
+
+ Added rend_cache_lookup_v2_dir() to allow a hidden service directory to
+ look up a v2 descriptor in the local cache under its descriptor ID instead
+ of its service ID (/7/).
+
+ Moved the parsing part from rend_cache_store() to the new function
+ rend_cache_store_parse() to reuse it for v2 descriptors (/6/).
+
+ Added rend_cache_store_v2_client() to parse a v2 descriptor and parse the
+ encrypted list of introduction points (/14/).
+
+ Added rend_cache_store_v2_dir() to allow a hidden service directory to
+ store a v2 descriptor in the local cache under its descriptor ID instead of
+ its service ID (/6/).
+
+ rendservice.c
+
+ Extended rend_service_t by a member secret_cookie (/10/ and /12/).
+
+ Added rend_get_hostname2() to assemble a v2 onion address (/10/).
+
+ Changed rend_service_load_keys() to write a secret_cookie and a hostname2
+ file (/10/).
+
+ Changed upload_service_descriptor() to upload v2 hidden service
+ descriptors, if configured (/12/).
+
+ Changed rend_consider_services_upload() to also initiate the upload of v2
+ descriptors, if configured (/12/).
+
+ router.c
+
+ Changed router_dump_router_to_string() to include the "hidden-service-dir"
+ flag in a router descriptor if configured (/5/).
+
+ routerlist.c
+
+ Changed router_get_routerlist() to initialize routing list (/1/).
+
+ Added get_responsible_hs_dir() to determine the router that is responsible
+ for a given descriptor ID (/2/).
+
+ routerparse.c
+
+ Added 14 keywords to directory_keyword; 1 to parse the "hidden-service-dir"
+ flag in router descriptors (/9/), 8 to parse v2 hidden service descriptors
+ (/6/ and /14/), and 5 to parse the decrypted list of introduction points
+ (/14/).
+
+ Added an entry to routerdesc_token_table[] to parse the
+ "hidden-service-directory" flag in router descriptors (/9/).
+
+ Added desc_token_table[] to parse v2 hidden service descriptors (/6/ and
+ /14/).
+
+ Added ipo_token_table[] to parse the decrypted introduction points of v2
+ hidden service descriptors (/14/).
+
+ Changed router_parse_entry_from_string() to parse the "hidden-service-dir"
+ flag in router descriptors (/9/).
+
+ Changed routerstatus_parse_entry_from_string to parse the "HSDir" flag in
+ vote and consensus status documents (/1/).
+
+ Added rend_parse_v2_service_descriptor() to parse a v2 hidden service
+ descriptor (/6/ and /14/).
+
+ Added rend_decrypt_introduction_points() to decrypt and parse the list of
+ introduction points (/14/).
+
+
\ No newline at end of file
Modified: tor/branches/114-dist-storage/doc/spec/rend-spec.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/rend-spec.txt 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/doc/spec/rend-spec.txt 2007-07-09 15:13:19 UTC (rev 10774)
@@ -109,6 +109,16 @@
39 -- RELAY_RENDEZVOUS_ESTABLISHED
40 -- RELAY_COMMAND_INTRODUCE_ACK
+0.4. History
+
+ [TODO114 Write something about the history of versions 0 and 1, and explain
+ the reasons for version 2 and the migration strategy. Some passages could be
+ copied from proposal 114.]
+
+ Remark on notation: The specification passages for version 2 are not
+ intermixed with passages for versions 0 and 1, but rather appended to the
+ according sections and prefixed with "Changes in version 2:".
+
1. The Protocol
1.1. Bob configures his local OP.
@@ -118,16 +128,95 @@
per OP, and MUST allow Bob to specify one or more virtual ports per
service. Bob provides a mapping from each of these virtual ports
to a local IP:Port pair.
+
+ Changes in version 2: If Bob's OP is configured to publish v2 hidden service
+ descriptors, it checks if it has already created a random secret cookie file
+ and a file containing the v2 onion address of it, and if not, it creates
+ both.
1.2. Bob's OP generates service descriptors.
The first time the OP provides an advertised service, it generates
a public/private keypair (stored locally). Periodically, the OP
generates a pair of service descriptors, one "V1" and one "V0".
- If configured, the OP also generates a "V2" service descriptor.
- The "V2" descriptor in branch 114-dist-storage contains:
+ The "V1" descriptor in 0.1.1.6-alpha contains:
+ V Format byte: set to 255 [1 octet]
+ V Version byte: set to 1 [1 octet]
+ KL Key length [2 octets]
+ PK Bob's public key [KL octets]
+ TS A timestamp [4 octets]
+ PROTO Rendezvous protocol versions: bitmask [2 octets]
+ NA Number of auth mechanisms accepted [1 octet]
+ For each auth mechanism:
+ AUTHT The auth type that is supported [2 octets]
+ AUTHL Length of auth data [1 octet]
+ AUTHD Auth data [variable]
+ NI Number of introduction points [2 octets]
+ For each introduction point: (as in INTRODUCE2 cells)
+ ATYPE An address type (typically 4) [1 octet]
+ ADDR Introduction point's IP address [4 or 16 octets]
+ PORT Introduction point's OR port [2 octets]
+ AUTHT The auth type that is supported [2 octets]
+ AUTHL Length of auth data [1 octet]
+ AUTHD Auth data [variable]
+ ID Introduction point identity ID [20 octets]
+ KLEN Length of onion key [2 octets]
+ KEY Introduction point onion key [KLEN octets]
+ SIG Signature of above fields [variable]
+
+ The "V1" descriptor in 0.1.1.5-alpha-cvs contains:
+
+ V Format byte: set to 255 [1 octet]
+ V Version byte: set to 1 [1 octet]
+ KL Key length [2 octets]
+ PK Bob's public key [KL octets]
+ TS A timestamp [4 octets]
+ PROTO Protocol versions: bitmask [2 octets]
+ NI Number of introduction points [2 octets]
+ For each introduction point: (as in INTRODUCE2 cells)
+ IP Introduction point's address [4 octets]
+ PORT Introduction point's OR port [2 octets]
+ ID Introduction point identity ID [20 octets]
+ KLEN Length of onion key [2 octets]
+ KEY Introduction point onion key [KLEN octets]
+ SIG Signature of above fields [variable]
+
+ The "V0" descriptor contains:
+
+ KL Key length [2 octets]
+ PK Bob's public key [KL octets]
+ TS A timestamp [4 octets]
+ NI Number of introduction points [2 octets]
+ Ipt A list of NUL-terminated ORs [variable]
+ SIG Signature of above fields [variable]
+
+ KL is the length of PK, in octets.
+ TS is the number of seconds elapsed since Jan 1, 1970.
+
+ AUTHT specifies which authentication/authorization mechanism is
+ required by the hidden service or the introduction point. AUTHD
+ is arbitrary data that can be associated with an auth approach.
+ Currently only AUTHT of [00 00] is supported, with an AUTHL of 0.
+ See section 2 of this document for details on auth mechanisms.
+
+ The members of Ipt may be either (a) nicknames, or (b) identity key
+ digests, encoded in hex, and prefixed with a '$'. Clients must
+ accept both forms. Services must only generate the second form.
+ Once 0.0.9.x is obsoleted, we can drop the first form.
+
+ [It's ok for Bob to advertise 0 introduction points. He might want
+ to do that if he previously advertised some introduction points,
+ and now he doesn't have any. -RD]
+
+ [Once Tor 0.1.0.x is obsolete, we can stop generating or using V0
+ descriptors. -NM]
+
+ Changes in version 2: Bob's OP generates v2 service descriptors in addition
+ or as replacement to v1 service descriptors. The format of a v2 service
+ descriptor is as follows:
+
"rendezvous-service-descriptor" descriptor-id NL
[At start, exactly once]
@@ -255,90 +344,13 @@
The public key that can be used to encrypt messages to the hidden
service.
-[TODO is it necessary to include a digest of the list of introduction points
-(plus some service-specific data to make digests incomparable) to allow any
-node to decide whether changes are cosmetic?! -KL]
-
"signature" NL signature-string
[At end, exactly once]
A signature of all fields above with the private key of the hidden
service.
-
- The "V1" descriptor in 0.1.1.6-alpha contains:
- V Format byte: set to 255 [1 octet]
- V Version byte: set to 1 [1 octet]
- KL Key length [2 octets]
- PK Bob's public key [KL octets]
- TS A timestamp [4 octets]
- PROTO Rendezvous protocol versions: bitmask [2 octets]
- NA Number of auth mechanisms accepted [1 octet]
- For each auth mechanism:
- AUTHT The auth type that is supported [2 octets]
- AUTHL Length of auth data [1 octet]
- AUTHD Auth data [variable]
- NI Number of introduction points [2 octets]
- For each introduction point: (as in INTRODUCE2 cells)
- ATYPE An address type (typically 4) [1 octet]
- ADDR Introduction point's IP address [4 or 16 octets]
- PORT Introduction point's OR port [2 octets]
- AUTHT The auth type that is supported [2 octets]
- AUTHL Length of auth data [1 octet]
- AUTHD Auth data [variable]
- ID Introduction point identity ID [20 octets]
- KLEN Length of onion key [2 octets]
- KEY Introduction point onion key [KLEN octets]
- SIG Signature of above fields [variable]
-
- The "V1" descriptor in 0.1.1.5-alpha-cvs contains:
-
- V Format byte: set to 255 [1 octet]
- V Version byte: set to 1 [1 octet]
- KL Key length [2 octets]
- PK Bob's public key [KL octets]
- TS A timestamp [4 octets]
- PROTO Protocol versions: bitmask [2 octets]
- NI Number of introduction points [2 octets]
- For each introduction point: (as in INTRODUCE2 cells)
- IP Introduction point's address [4 octets]
- PORT Introduction point's OR port [2 octets]
- ID Introduction point identity ID [20 octets]
- KLEN Length of onion key [2 octets]
- KEY Introduction point onion key [KLEN octets]
- SIG Signature of above fields [variable]
-
- The "V0" descriptor contains:
-
- KL Key length [2 octets]
- PK Bob's public key [KL octets]
- TS A timestamp [4 octets]
- NI Number of introduction points [2 octets]
- Ipt A list of NUL-terminated ORs [variable]
- SIG Signature of above fields [variable]
-
- KL is the length of PK, in octets.
- TS is the number of seconds elapsed since Jan 1, 1970.
-
- AUTHT specifies which authentication/authorization mechanism is
- required by the hidden service or the introduction point. AUTHD
- is arbitrary data that can be associated with an auth approach.
- Currently only AUTHT of [00 00] is supported, with an AUTHL of 0.
- See section 2 of this document for details on auth mechanisms.
-
- The members of Ipt may be either (a) nicknames, or (b) identity key
- digests, encoded in hex, and prefixed with a '$'. Clients must
- accept both forms. Services must only generate the second form.
- Once 0.0.9.x is obsoleted, we can drop the first form.
-
- [It's ok for Bob to advertise 0 introduction points. He might want
- to do that if he previously advertised some introduction points,
- and now he doesn't have any. -RD]
-
- [Once Tor 0.1.0.x is obsolete, we can stop generating or using V0
- descriptors. -NM]
-
1.3. Bob's OP establishes his introduction points.
The OP establishes a new introduction circuit to each introduction
@@ -369,6 +381,18 @@
currently associated with PK. On success, the OR sends Bob a
RELAY_INTRO_ESTABLISHED cell with an empty payload.
+ Changes in version 2: If configured to publish only v2 descriptors and no
+ v0/v1 descriptors, Bob's OP does not include its own public key in the
+ RELAY_ESTABLISH_INTRO cell, but the public key of a freshly generated key
+ pair. The OP also includes these fresh public keys in the hidden service
+ descriptor together with the other introduction point information. The
+ reason is that the introduction point does not need to and therefore should
+ not know for which hidden service it works, so as to prevent it from
+ tracking the hidden service's activity. However, as long as Bob publishes
+ both, v0/v1 and v2 descriptors, v0/v1 clients rely on the fact that all
+ introduction points accept the same public key, so that this new feature
+ cannot be used. Bob then includes its own public key as usual.
+
1.4. Bob's OP advertises his service descriptor(s).
Bob's OP opens a stream to each directory server's directory port via Tor.
@@ -391,6 +415,31 @@
after its timestamp. At least every 18 hours, Bob's OP uploads a
fresh descriptor.
+ Changes in version 2: Instead of publishing service descriptors to the
+ directory servers, Bob's OP publishes v2 descriptors to a changing subset of
+ all hidden service directories. Therefore, Bob's OP opens a stream via Tor
+ to all responsible hidden service directories. (He may re-use old circuits
+ for this.) Over this stream, Bob's OP makes an HTTP 'POST' request to a URL
+ "/tor/rendezvous/publish2" relative to the hidden service directory's root,
+ containing as its body Bob's service descriptor.
+
+ At any time, there are 6 hidden service directories responsible for keeping
+ replicas of a descriptor; they consist of 2 sets of 3 hidden service
+ directories with consecutive onion IDs. Bob's OP learns about the complete
+ list of hidden service directories by filtering the consensus status
+ document received from the directory authorities. Bob's OP should compile
+ this list once and keep it current for performance reasons. A hidden service
+ directory is deemed responsible for all descriptor IDs in the interval from
+ its direct predecessor, exclusive, to its own ID, inclusive; it further
+ holds replicas for its 2 predecessors.
+
+ Bob's OP publishes a new v2 descriptor whenever its content changes or a new
+ publication period of 24 hours starts for this descriptor. If the current
+ publication period would only last for less than 60 minutes (= 2 x 30
+ minutes to allow the server to be 30 minutes behind and the client 30
+ minutes ahead), Bob's OP publishes the descriptor under the ID of both, the
+ current and the next publication period.
+
1.5. Alice receives a x.y.z.onion address.
When Alice receives a pointer to a location-hidden service, it is as a
@@ -417,6 +466,13 @@
[Yes, numbers are allowed at the beginning. See RFC 1123. -NM]
+ Changes in version 2: A v2 onion address consists of the service ID (called
+ "z" in the description above) and the secret cookie "c" that is shared
+ between hidden service provider and clients. A valid v2 onion address has
+ the form "z.c.onion" where z is a 16-character long base-32 string and
+ where c is the 24-character long base-32 encoding of a 120 bits long secret
+ key.
+
1.6. Alice's OP retrieves a service descriptor.
Alice opens a stream to a directory server via Tor, and makes an HTTP GET
@@ -443,6 +499,25 @@
[Caching may make her partitionable, but she fetched it anonymously,
and we can't very well *not* cache it. -RD]
+ Changes in version 2: Analog to the description in section 1.4, Alice's OP
+ fetches v2 descriptors from a randomly chosen hidden service directory out
+ of the changing subset of 6 nodes. The probability of picking the first or
+ the second set of 3 nodes is 1/2 each, and the probability of picking one of
+ the 3 consecutive nodes is 1/2 for the responsible node and 1/3 (1/6) for
+ its first (second) predecessor. Alice relies on Bob to care about a
+ potential clock skew between the two by possibly storing two sets of
+ descriptors.
+
+ Alice's OP opens a stream via Tor to the chosen hidden service directory.
+ (She may re-use old circuits for this.) Over this stream, Alice's OP makes
+ an HTTP 'GET' request for the document "/tor/rendezvous2/<z>", where z is
+ replaced with the encoding of the descriptor ID. The directory replies with
+ a 404 HTTP response if it does not recognize <z>, and otherwise returns Bob's most
+ recently uploaded service descriptor.
+
+ When parsing the service descriptor, Alice decrypts the list of introduction
+ points with the secret cookie.
+
1.7. Alice's OP establishes a rendezvous point.
When Alice requests a connection to a given location-hidden service,
@@ -522,6 +597,10 @@
RELAY_INTRODUCE1 cell on the wire will contain 20+42+16+20+20+128=246
bytes. [XXXX not really]
+ Changes in version 2: Instead of Bob's public key, Alice uses the public key
+ that was contained in the v2 descriptor for the chosen introduction point to
+ create the RELAY_INTRODUCE1 cell as described above.
+
1.9. Introduction: From the Introduction Point to Bob's OP
If the Introduction Point recognizes PK_ID as a public key which has
@@ -597,3 +676,62 @@
Foo.
+3. Hidden service directory operation
+
+Changes in version 2: This section is new in version 2. It contains all
+operations of a hidden service directory that are required for the protocol
+described in section 1 to succeed with v2 hidden service descriptors.
+
+3.1. Configuring as hidden service directory
+
+ Every onion router that has its directory port open can decide whether it
+ wants to store and serve hidden service descriptors by setting a new config
+ option "HSDir" 0|1 to 1. An onion router with this config option being set
+ includes the flag "hidden-service-dir" in its router descriptors that it
+ sends to directory authorities.
+
+ The directory authorities include a new flag "HSDir" for routers that
+ decided to provide storage for hidden service descriptors and that are
+ running for at least 24 hours.
+
+3.2. Accepting publish requests
+
+ Hidden service directory nodes accept publish requests for hidden service
+ descriptors and store them to their local memory. (It is not necessary to
+ make descriptors persistent, because after disconnecting, the onion router
+ would not be accepted as storing node anyway, because it has not been
+ running for at least 24 hours.) All requests and replies are formatted as
+ HTTP messages. Requests are contained within BEGIN_DIR cells, directed to
+ the router's directory port, and formatted as HTTP POST requests to the URL
+ "/tor/rendezvous/publish2" relative to the hidden service directory's root,
+ containing as its body a v2 service descriptor.
+
+ A hidden service directory node parses every received descriptor and only
+ stores it when it thinks that it is responsible for storing that descriptor
+ based on its own routing table. See section 1.4 for more information on how
+ to determine responsibility for a certain descriptor ID.
+
+3.3. Processing fetch requests
+
+ Hidden service directory nodes process fetch requests for hidden service
+ descriptors by looking them up in their local memory. (They do not need to
+ determine if they are responsible for the passed ID, because it does no harm
+ if they deliver a descriptor for which they are not (any more) responsible.)
+ All requests and replies are formatted as HTTP messages. Requests are
+ contained within BEGIN_DIR cells, directed to the router's directory port,
+ and formatted as HTTP GET requests for the document "/tor/rendezvous2/<z>",
+ where z is replaced with the encoding of the descriptor ID.
+
+3.4. Replicate with neighbours
+
+ A hidden service directory node replicates descriptors from its two
+ predecessors once an hour by requesting all descriptors for which they are
+ responsible (not including the descriptors they hold replicas of) in a
+ single request. Further, whenever a hidden service directory realizes that a
+ predecessor has left the network, it establishes a connection to the new 3rd
+ predecessor and requests its stored descriptors. Whenever it realizes that a
+ new onion router has joined with an ID higher than its former 3rd
+ predecessor, it discards all descriptors in the interval of its now 4th and
+ 3rd predecessor. See section 1.4 for more information on how to determine
+ responsibility for a certain descriptor ID.
+
Modified: tor/branches/114-dist-storage/doc/tor.1.in
===================================================================
--- tor/branches/114-dist-storage/doc/tor.1.in 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/doc/tor.1.in 2007-07-09 15:13:19 UTC (rev 10774)
@@ -174,6 +174,12 @@
(Default: 1)
.LP
.TP
+\fBFetchV2HidServDescriptors \fR\fB0\fR|\fB1\fR\fP
+If set to 0, Tor will never fetch any v2 hidden service descriptors from
+the rendezvous directories.
+(Default: 0)
+.LP
+.TP
\fBFetchServerDescriptors \fR\fB0\fR|\fB1\fR\fP
If set to 0, Tor will never fetch any network status summaries or server
descriptors from the directory servers. This option is only useful if
@@ -917,6 +923,12 @@
accepts and serves hidden service descriptors. (Default: 0)
.LP
.TP
+\fBHSDir \fR\fB0\fR|\fB1\fR\fP
+When this option is set, Tor also accepts and serves v2 hidden service
+descriptors. The Tor node does not need to be an authoritative directory for
+this. (Default: 0)
+.LP
+.TP
\fBHSAuthorityRecordStats \fR\fB0\fR|\fB1\fR\fP
When this option is set in addition to \fBHSAuthoritativeDir\fP, Tor
periodically (every 15 minutes) writes statistics about hidden service
@@ -1046,6 +1058,12 @@
(Default: 1)
.LP
.TP
+\fBPublishV2HidServDescriptors \fR\fB0\fR|\fB1\fR\fP
+If set to 0, Tor will run any hidden services you configure, but it won't
+advertise them to the hidden service directories using v2 descriptors.
+(Default: 0)
+.LP
+.TP
\fBRendPostPeriod \fR\fIN\fR \fBseconds\fR|\fBminutes\fR|\fBhours\fR|\fBdays\fR|\fBweeks\fP
Every time the specified period elapses, Tor uploads any rendezvous
service descriptors to the directory servers. This information is also
@@ -1158,8 +1176,19 @@
The <base32-encoded-fingerprint>.onion domain name for this hidden service.
.LP
.TP
+.B \fIHiddenServiceDirectory\fP/hostname2
+The <base32-encoded-service-id>.<base32-encoded-cookie>.onion v2 domain name
+for this hidden service.
+.LP
+.TP
.B \fIHiddenServiceDirectory\fP/private_key
The private key for this hidden service.
+.LP
+.TP
+.B \fIHiddenServiceDirectory\fP/secret_cookie
+The randomly generated secret cookie that is used to create unpredictable
+descriptor IDs and to encrypt the introduction points of v2 hidden service
+descriptors.
.SH SEE ALSO
.BR privoxy (1),
.BR tsocks (1),
Modified: tor/branches/114-dist-storage/src/common/container.c
===================================================================
--- tor/branches/114-dist-storage/src/common/container.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/common/container.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -247,6 +247,25 @@
return 0;
}
+/** Assuming the members of <b>sl</b> are in ascending order, return the
+ * first element that is greater than <b>id</b>; if all elements are smaller
+ * than <b>id</b>, return the first element in <b>sl</b>; if <b>sl</b> is
+ * NULL, or has no elements, return NULL.
+ */
+const char *
+smartlist_digest_next_circular(const smartlist_t *sl, const char *id)
+{
+ int i;
+ if (!sl) return NULL;
+ if (sl->num_used == 0) return NULL;
+ for (i=0; i < sl->num_used; i++) {
+ if (memcmp((const char*)sl->list[i],id,DIGEST_LEN) > 0) {
+ return (const char*)sl->list[i];
+ }
+ }
+ return (const char*)sl->list[0];
+}
+
/** Return true iff some element E of sl2 has smartlist_isin(sl1,E).
*/
int
Modified: tor/branches/114-dist-storage/src/common/container.h
===================================================================
--- tor/branches/114-dist-storage/src/common/container.h 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/common/container.h 2007-07-09 15:13:19 UTC (rev 10774)
@@ -46,6 +46,8 @@
int smartlist_string_num_isin(const smartlist_t *sl, int num) ATTR_PURE;
int smartlist_digest_isin(const smartlist_t *sl, const char *element)
ATTR_PURE;
+const char *smartlist_digest_next_circular(const smartlist_t *sl,
+ const char *id);
int smartlist_overlap(const smartlist_t *sl1, const smartlist_t *sl2)
ATTR_PURE;
void smartlist_intersect(smartlist_t *sl1, const smartlist_t *sl2);
@@ -55,22 +57,24 @@
#ifdef DEBUG_SMARTLIST
/** Return the number of items in sl.
*/
-extern INLINE int smartlist_len(const smartlist_t *sl) ATTR_PURE {
+static INLINE int smartlist_len(const smartlist_t *sl) ATTR_PURE;
+static INLINE int smartlist_len(const smartlist_t *sl) {
tor_assert(sl);
return (sl)->num_used;
}
/** Return the <b>idx</b>th element of sl.
*/
-extern INLINE void *smartlist_get(const smartlist_t *sl, int idx) ATTR_PURE {
+static INLINE void *smartlist_get(const smartlist_t *sl, int idx) ATTR_PURE;
+static INLINE void *smartlist_get(const smartlist_t *sl, int idx) {
tor_assert(sl);
tor_assert(idx>=0);
- tor_assert(sl->num_used < idx);
+ tor_assert(sl->num_used > idx);
return sl->list[idx];
}
-extern INLINE void smartlist_set(smartlist_t *sl, int idx, void *val) {
+static INLINE void smartlist_set(smartlist_t *sl, int idx, void *val) {
tor_assert(sl);
tor_assert(idx>=0);
- tor_assert(sl->num_used < idx);
+ tor_assert(sl->num_used > idx);
sl->list[idx] = val;
}
#else
Modified: tor/branches/114-dist-storage/src/common/crypto.c
===================================================================
--- tor/branches/114-dist-storage/src/common/crypto.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/common/crypto.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1901,7 +1901,7 @@
tor_assert(destlen < SIZE_T_CEILING);
for (i=0,bit=0; bit < nbits; ++i, bit+=5) {
- /* set v to the 16-bit value starting at src[bit/8], 0-padded. */
+ /* set v to the 16-bit value starting at src[bits/8], 0-padded. */
v = ((uint8_t)src[bit/8]) << 8;
if (bit+5<nbits) v += (uint8_t)src[(bit/8)+1];
/* set u to the 5-bit value at the bit'th bit of src. */
Modified: tor/branches/114-dist-storage/src/or/circuitbuild.c
===================================================================
--- tor/branches/114-dist-storage/src/or/circuitbuild.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/circuitbuild.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -2418,6 +2418,13 @@
r = entry_is_live(entry, need_uptime, need_capacity, 0);
if (r && !smartlist_isin(exit_family, r)) {
smartlist_add(live_entry_guards, r);
+ if (!entry->made_contact) {
+ /* Always start with the first not-yet-contacted entry
+ * guard. Otherwise we might add several new ones, pick
+ * the second new one, and now we've expanded our entry
+ * guard list without needing to. */
+ goto choose_and_finish;
+ }
if (smartlist_len(live_entry_guards) >= options->NumEntryGuards)
break; /* we have enough */
}
@@ -2451,6 +2458,7 @@
/* live_entry_guards may be empty below. Oh well, we tried. */
}
+ choose_and_finish:
r = smartlist_choose(live_entry_guards);
smartlist_free(live_entry_guards);
smartlist_free(exit_family);
Modified: tor/branches/114-dist-storage/src/or/circuitlist.c
===================================================================
--- tor/branches/114-dist-storage/src/or/circuitlist.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/circuitlist.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1006,7 +1006,8 @@
safe_str(ocirc->rend_query),
safe_str(build_state_get_exit_nickname(ocirc->build_state)));
rend_client_remove_intro_point(ocirc->build_state->chosen_exit,
- ocirc->rend_query);
+ ocirc->rend_query, ocirc->rend_version,
+ ocirc->secret_cookie);
}
if (circ->n_conn)
connection_or_send_destroy(circ->n_circ_id, circ->n_conn, reason);
Modified: tor/branches/114-dist-storage/src/or/circuituse.c
===================================================================
--- tor/branches/114-dist-storage/src/or/circuituse.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/circuituse.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1008,7 +1008,11 @@
log_info(LD_REND,
"No intro points for '%s': refetching service descriptor.",
safe_str(conn->rend_query));
- rend_client_refetch_renddesc(conn->rend_query);
+ if (conn->rend_version == 2)
+ rend_client_refetch_v2_renddesc(conn->rend_query,
+ conn->secret_cookie);
+ else
+ rend_client_refetch_renddesc(conn->rend_query);
conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT;
return 0;
}
@@ -1081,8 +1085,11 @@
/* help predict this next time */
rep_hist_note_used_internal(time(NULL), need_uptime, 1);
if (circ) {
- /* write the service_id into circ */
+ /* write the service_id, secret_cookie, and rend_version into circ */
strlcpy(circ->rend_query, conn->rend_query, sizeof(circ->rend_query));
+ strlcpy(circ->secret_cookie, conn->secret_cookie,
+ sizeof(circ->secret_cookie));
+ circ->rend_version = conn->rend_version;
if (circ->_base.purpose == CIRCUIT_PURPOSE_C_ESTABLISH_REND &&
circ->_base.state == CIRCUIT_STATE_OPEN)
rend_client_rendcirc_has_opened(circ);
Modified: tor/branches/114-dist-storage/src/or/config.c
===================================================================
--- tor/branches/114-dist-storage/src/or/config.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/config.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -179,6 +179,7 @@
VAR("Group", STRING, Group, NULL),
VAR("HardwareAccel", BOOL, HardwareAccel, "0"),
VAR("HashedControlPassword",STRING, HashedControlPassword, NULL),
+ VAR("HSDir", BOOL, HSDir, "0"),
VAR("HiddenServiceDir", LINELIST_S, RendConfigLines, NULL),
VAR("HiddenServiceExcludeNodes", LINELIST_S, RendConfigLines, NULL),
VAR("HiddenServiceNodes", LINELIST_S, RendConfigLines, NULL),
@@ -192,8 +193,6 @@
VAR("HttpsProxyAuthenticator",STRING,HttpsProxyAuthenticator,NULL),
OBSOLETE("IgnoreVersion"),
VAR("KeepalivePeriod", INTERVAL, KeepalivePeriod, "5 minutes"),
- VAR("LoadV2HidServDescriptorsFromDisk",BOOL,
- LoadV2HidServDescriptorsFromDisk,"0"),
VAR("Log", LINELIST, Logs, NULL),
OBSOLETE("LinkPadding"),
OBSOLETE("LogLevel"),
@@ -256,10 +255,6 @@
VAR("SocksPort", UINT, SocksPort, "9050"),
VAR("SocksTimeout", INTERVAL, SocksTimeout, "2 minutes"),
OBSOLETE("StatusFetchPeriod"),
- VAR("StoreV2HidServDescriptorsToDisk",BOOL,
- StoreV2HidServDescriptorsToDisk,"0"),
-// VAR("StoreAndServeV2HidServDescriptors", BOOL,
-// StoreAndServeV2HidServDescriptors, "0"),
VAR("StrictEntryNodes", BOOL, StrictEntryNodes, "0"),
VAR("StrictExitNodes", BOOL, StrictExitNodes, "0"),
OBSOLETE("SysLog"),
Modified: tor/branches/114-dist-storage/src/or/connection_edge.c
===================================================================
--- tor/branches/114-dist-storage/src/or/connection_edge.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/connection_edge.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1408,29 +1408,25 @@
connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL);
return -1;
}
-/** How long after we receive a hidden service descriptor do we consider
- * it valid? */
-#define NUM_SECONDS_BEFORE_HS_REFETCH (60*15)
- /* if necessary and configured, try to read descriptor from disk */
- if (get_options()->LoadV2HidServDescriptorsFromDisk &&
- (r==0 || (r>0 && now - entry->received >=
- NUM_SECONDS_BEFORE_HS_REFETCH))) {
- if (rend_load_service_descriptor_from_disk(socks->address,
- time(NULL)) < 0) {
- log_warn(LD_REND, "Loading of renddesc from disk failed for id %s",
- escaped_safe_str(socks->address));
- } else {
- log_debug(LD_REND,"Loading of renddesc from disk successful for id %s",
- escaped_safe_str(socks->address));
- r = rend_cache_lookup_entry(socks->address, -1, &entry);
- }
+ if (rend_valid_service_id(socks->address) == 2) {
+ conn->rend_version = 2;
+ strlcpy(conn->secret_cookie, socks->address+17,
+ sizeof(conn->secret_cookie));
+ } else {
+ conn->rend_version = 1;
}
if (r==0) {
conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT;
log_info(LD_REND, "Unknown descriptor %s. Fetching.",
safe_str(conn->rend_query));
- rend_client_refetch_renddesc(conn->rend_query);
+ if (conn->rend_version == 2)
+ rend_client_refetch_v2_renddesc(conn->rend_query, conn->secret_cookie);
+ else
+ rend_client_refetch_renddesc(conn->rend_query);
} else { /* r > 0 */
+/** How long after we receive a hidden service descriptor do we consider
+ * it valid? */
+#define NUM_SECONDS_BEFORE_HS_REFETCH (60*15)
if (now - entry->received < NUM_SECONDS_BEFORE_HS_REFETCH) {
conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT;
log_info(LD_REND, "Descriptor is here and fresh enough. Great.");
@@ -1442,7 +1438,11 @@
conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT;
log_info(LD_REND, "Stale descriptor %s. Refetching.",
safe_str(conn->rend_query));
- rend_client_refetch_renddesc(conn->rend_query);
+ if (conn->rend_version == 2)
+ rend_client_refetch_v2_renddesc(conn->rend_query,
+ conn->secret_cookie);
+ else
+ rend_client_refetch_renddesc(conn->rend_query);
}
}
return 0;
@@ -1717,10 +1717,14 @@
}
daddr = tbuf = &tmp_buf[0] + 6; /* after end of "[DEST " */
- while (*tbuf != '\0' && *tbuf != ' ')
- tbuf++;
- *tbuf = '\0';
- tbuf++;
+ if (!(tbuf = strchr(tbuf, ' '))) {
+ log_warn(LD_APP,"Natd handshake was ill-formed; closing. The client "
+ "said: %s",
+ escaped(tmp_buf));
+ connection_mark_unattached_ap(conn, END_STREAM_REASON_INVALID_NATD_DEST);
+ return -1;
+ }
+ *tbuf++ = '\0';
/* pretend that a socks handshake completed so we don't try to
* send a socks reply down a natd conn */
@@ -1961,6 +1965,8 @@
conn->_base.state = AP_CONN_STATE_CIRCUIT_WAIT;
+ control_event_stream_status(conn, STREAM_EVENT_NEW, 0);
+
/* attaching to a dirty circuit is fine */
if (connection_ap_handshake_attach_circuit(conn) < 0) {
connection_mark_unattached_ap(conn, END_STREAM_REASON_CANT_ATTACH);
@@ -2597,27 +2603,26 @@
hostname_type_t
parse_extended_hostname(char *address)
{
- char *s;
+ char *s;
- s = strrchr(address,'.');
- if (!s)
- return NORMAL_HOSTNAME; /* no dot, thus normal */
- if (!strcmp(s+1,"exit")) {
+ s = strrchr(address,'.');
+ if (!s)
+ return NORMAL_HOSTNAME; /* no dot, thus normal */
+ if (!strcmp(s+1,"exit")) {
+ *s = 0; /* nul-terminate it */
+ return EXIT_HOSTNAME; /* .exit */
+ }
+ if (strcmp(s+1,"onion"))
+ return NORMAL_HOSTNAME; /* neither .exit nor .onion, thus normal */
+
+ /* so it is .onion */
*s = 0; /* nul-terminate it */
- return EXIT_HOSTNAME; /* .exit */
- }
- if (strcmp(s+1,"onion"))
- return NORMAL_HOSTNAME; /* neither .exit nor .onion, thus normal */
+ if (rend_valid_service_id(address))
+ return ONION_HOSTNAME; /* success */
- /* so it is .onion */
- *s = 0; /* nul-terminate it */
- if (rend_valid_service_id(address) || rend_valid_v2_service_id(address)) {
- return ONION_HOSTNAME; /* success */
- }
-
- /* otherwise, return to previous state and return 0 */
- *s = '.';
- return BAD_HOSTNAME;
+ /* otherwise, return to previous state and return 0 */
+ *s = '.';
+ return BAD_HOSTNAME;
}
/** Check if the address is of the form "y.noconnect"
Modified: tor/branches/114-dist-storage/src/or/directory.c
===================================================================
--- tor/branches/114-dist-storage/src/or/directory.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/directory.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -190,6 +190,55 @@
}
}
+/** Start a connection to (1 or more?) hidden service directories, uploading
+ * the payload 'payload' (length 'payload_len').
+ *
+ * desc_id = 32 base32 chars, desc = NUL-terminated ASCII-based v2 descriptor
+ *
+ * TODO114 enable tunneling when available!!
+ */
+void
+directory_post_to_hs_dir(const char *desc_id, const char *desc)
+{
+ tor_assert(desc_id);
+ tor_assert(strlen(desc_id) == 32);
+ tor_assert(desc);
+ routerstatus_t *hs_dir = get_responsible_hs_dir(desc_id);
+ directory_initiate_command_routerstatus(hs_dir,
+ DIR_PURPOSE_UPLOAD_RENDDESC_V2,
+ ROUTER_PURPOSE_GENERAL, 0,
+ NULL, desc, strlen(desc));
+}
+
+/** Start a connection to (1 or more?) hidden service directories, downloading
+ * the payload 'payload' (length 'payload_len').
+ *
+ * desc_id = 32 base32 chars, query = 16 base32 chars,
+ * secret_cookie = 24 base32 chars
+ *
+ * TODO114 enable tunneling when available!!
+ */
+void
+directory_get_from_hs_dir(const char *desc_id, const char *query,
+ const char *secret_cookie)
+{
+ tor_assert(desc_id);
+ tor_assert(strlen(desc_id) == 32);
+ tor_assert(query);
+ tor_assert(strlen(query) == 16);
+ tor_assert(secret_cookie);
+ tor_assert(strlen(secret_cookie) == 24);
+ routerstatus_t *hs_dir = get_responsible_hs_dir(desc_id);
+ char orig_request[16+1+24+1];
+ tor_snprintf(orig_request, sizeof(orig_request), "%s.%s", query,
+ secret_cookie);
+ directory_initiate_command_routerstatus(hs_dir,
+ DIR_PURPOSE_FETCH_RENDDESC_V2,
+ ROUTER_PURPOSE_GENERAL, 0,
+ desc_id, orig_request, 0);
+ return;
+}
+
/** Start a connection to a random running directory server, using
* connection purpose 'purpose' and requesting 'resource'.
* If <b>retry_if_no_servers</b>, then if all the possible servers seem
@@ -310,9 +359,9 @@
* upload or download a server or rendezvous
* descriptor. <b>dir_purpose</b> determines what
* kind of directory connection we're launching, and must be one of
- * DIR_PURPOSE_{FETCH|UPLOAD}_{DIR|RENDDESC}. <b>router_purpose</b>
- * specifies the descriptor purposes we have in mind (currently only
- * used for FETCH_DIR).
+ * DIR_PURPOSE_{FETCH|UPLOAD}_{DIR|RENDDESC|RENDDESC_V2}.
+ * <b>router_purpose</b> specifies the descriptor purposes we have in mind
+ * (currently only used for FETCH_DIR).
*
* When uploading, <b>payload</b> and <b>payload_len</b> determine the content
* of the HTTP post. Otherwise, <b>payload</b> should be NULL.
@@ -489,12 +538,18 @@
case DIR_PURPOSE_FETCH_RENDDESC:
log_debug(LD_DIR,"initiating hidden-service descriptor fetch");
break;
+ case DIR_PURPOSE_FETCH_RENDDESC_V2:
+ log_debug(LD_DIR,"initiating hidden-service v2 descriptor fetch");
+ break;
case DIR_PURPOSE_UPLOAD_DIR:
log_debug(LD_OR,"initiating server descriptor upload");
break;
case DIR_PURPOSE_UPLOAD_RENDDESC:
log_debug(LD_REND,"initiating hidden-service descriptor upload");
break;
+ case DIR_PURPOSE_UPLOAD_RENDDESC_V2:
+ log_debug(LD_REND,"initiating hidden-service v2 descriptor upload");
+ break;
case DIR_PURPOSE_FETCH_RUNNING_LIST:
log_debug(LD_DIR,"initiating running-routers fetch");
break;
@@ -514,6 +569,15 @@
conn = TO_DIR_CONN(connection_new(CONN_TYPE_DIR, AF_INET));
+ // TODO114 this is a (dirty) workaround!
+ if (dir_purpose == DIR_PURPOSE_FETCH_RENDDESC_V2) {
+ tor_assert(payload);
+ strlcpy(conn->rend_query, payload, sizeof(conn->rend_query));
+ conn->rend_version = 2;
+ strlcpy(conn->secret_cookie, payload+17, sizeof(conn->secret_cookie));
+ payload = NULL;
+ }
+
/* set up conn so it's got all the data we need to remember */
conn->_base.addr = addr;
conn->_base.port = want_to_tunnel ? or_port : dir_port;
@@ -696,12 +760,27 @@
url = tor_malloc(len);
tor_snprintf(url, len, "/tor/rendezvous/%s", resource);
break;
+ case DIR_PURPOSE_FETCH_RENDDESC_V2:
+ tor_assert(resource);
+ tor_assert(!payload);
+ tor_assert(strlen(resource) <= REND_DESC_ID_V2_LEN);
+ httpcommand = "GET";
+ len = strlen(resource)+32;
+ url = tor_malloc(len);
+ tor_snprintf(url, len, "/tor/rendezvous2/%s", resource);
+ break;
case DIR_PURPOSE_UPLOAD_RENDDESC:
tor_assert(!resource);
tor_assert(payload);
httpcommand = "POST";
url = tor_strdup("/tor/rendezvous/publish");
break;
+ case DIR_PURPOSE_UPLOAD_RENDDESC_V2:
+ tor_assert(!resource);
+ tor_assert(payload);
+ httpcommand = "POST";
+ url = tor_strdup("/tor/rendezvous/publish2");
+ break;
default:
tor_assert(0);
return;
@@ -1367,11 +1446,7 @@
(int)body_len, status_code, escaped(reason));
switch (status_code) {
case 200:
- /* TODO when implementing storing/fetching v2 descriptors, replace NULL
- * by secret_cookie that was given to us in the onion address when the
- * client requested the hidden service (need to memorize that
- * somewhere). This way it won't work!! */
- if (rend_cache_store(body, body_len, 0, 1, NULL) < 0) {
+ if (rend_cache_store(body, body_len, 0) < 0) {
log_warn(LD_REND,"Failed to store rendezvous descriptor.");
/* alice's ap_stream will notice when connection_mark_for_close
* cleans it up */
@@ -1399,6 +1474,37 @@
}
}
+ if (conn->_base.purpose == DIR_PURPOSE_FETCH_RENDDESC_V2) {
+ switch (status_code) {
+ case 200:
+ if (rend_cache_store_v2_client(body, conn->secret_cookie) < 0) {
+ log_warn(LD_REND,"Failed to store rendezvous descriptor.");
+ /* alice's ap_stream will notice when connection_mark_for_close
+ * cleans it up */
+ } else {
+ /* success. notify pending connections about this. */
+ conn->_base.purpose = DIR_PURPOSE_HAS_FETCHED_RENDDESC;
+ rend_client_desc_here(conn->rend_query);
+ }
+ break;
+ case 404:
+ /* not there. pending connections will be notified when
+ * connection_mark_for_close cleans it up. */
+ break;
+ case 400:
+ log_warn(LD_REND,
+ "http status 400 (%s). Dirserver didn't like our "
+ "v2 rendezvous query?", escaped(reason));
+ break;
+ default:
+ log_warn(LD_REND,"http status %d (%s) response unexpected while "
+ "fetching v2 hidden service descriptor (server '%s:%d').",
+ status_code, escaped(reason), conn->_base.address,
+ conn->_base.port);
+ break;
+ }
+ }
+
if (conn->_base.purpose == DIR_PURPOSE_UPLOAD_RENDDESC) {
switch (status_code) {
case 200:
@@ -1910,6 +2016,31 @@
return 0;
}
+ if (options->HSDir &&
+ !strcmpstart(url,"/tor/rendezvous2/")) {
+ /* v2 rendezvous descriptor fetch */
+ const char *descp;
+ size_t desc_len;
+ const char *query = url+strlen("/tor/rendezvous2/");
+
+ switch (rend_cache_lookup_v2_dir(query, &descp, &desc_len)) {
+ case 1: /* valid */
+ write_http_response_header(conn, desc_len, "text/plain",
+ NULL, 0);
+ /* need to send descp separately, because it may include nuls */
+ connection_write_to_buf(descp, desc_len, TO_CONN(conn));
+ break;
+ case 0: /* well-formed but not present */
+ write_http_status_line(conn, 404, "Not found");
+ break;
+ case -1: /* not well-formed */
+ write_http_status_line(conn, 400, "Bad request");
+ break;
+ }
+ tor_free(url);
+ return 0;
+ }
+
if (options->HSAuthoritativeDir &&
(!strcmpstart(url,"/tor/rendezvous/") ||
!strcmpstart(url,"/tor/rendezvous1/"))) {
@@ -1919,6 +2050,9 @@
int versioned = !strcmpstart(url,"/tor/rendezvous1/");
const char *query = url+strlen("/tor/rendezvous/")+(versioned?1:0);
+ // TODO114 it is not correct to allow all versions, but only
+ // 0 or 1; what if we were by chance a v2 client for that service? then we
+ // should better not return the v2 descriptor as result to a 0/1 request!
switch (rend_cache_lookup_desc(query, versioned?-1:0, &descp, &desc_len)) {
case 1: /* valid */
write_http_response_header(conn, desc_len, "application/octet-stream",
@@ -2015,6 +2149,26 @@
conn->_base.state = DIR_CONN_STATE_SERVER_WRITING;
+ if (parse_http_url(headers, &url) < 0) {
+ write_http_status_line(conn, 400, "Bad request");
+ return 0;
+ }
+ log_debug(LD_DIRSERV,"rewritten url as '%s'.", url);
+
+ if (options->HSDir &&
+ !strcmpstart(url,"/tor/rendezvous/publish2")) {
+ /* rendezvous descriptor post */
+ if (rend_cache_store_v2_dir(body) < 0) {
+ log_fn(LOG_PROTOCOL_WARN, LD_DIRSERV,
+ "Rejected rend descriptor (length %d) from %s.",
+ (int)body_len, conn->_base.address);
+ write_http_status_line(conn, 400, "Invalid service descriptor rejected");
+ } else {
+ write_http_status_line(conn, 200, "Service descriptor stored");
+ }
+ goto done;
+ }
+
if (!authdir_mode(options)) {
/* we just provide cached directories; we don't want to
* receive anything. */
@@ -2023,12 +2177,6 @@
return 0;
}
- if (parse_http_url(headers, &url) < 0) {
- write_http_status_line(conn, 400, "Bad request");
- return 0;
- }
- log_debug(LD_DIRSERV,"rewritten url as '%s'.", url);
-
if (authdir_mode_handles_descs(options) &&
!strcmp(url,"/tor/")) { /* server descriptor post */
const char *msg = NULL;
@@ -2059,7 +2207,7 @@
if (options->HSAuthoritativeDir &&
!strcmpstart(url,"/tor/rendezvous/publish")) {
/* rendezvous descriptor post */
- if (rend_cache_store(body, body_len, 1, 1, NULL) < 0) {
+ if (rend_cache_store(body, body_len, 1) < 0) {
// char tmp[1024*2+1];
log_fn(LOG_PROTOCOL_WARN, LD_DIRSERV,
"Rejected rend descriptor (length %d) from %s.",
Modified: tor/branches/114-dist-storage/src/or/dirserv.c
===================================================================
--- tor/branches/114-dist-storage/src/or/dirserv.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/dirserv.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1638,13 +1638,14 @@
return 0;
cp = buf + strlen(buf);
r = tor_snprintf(cp, buf_len - (cp-buf),
- "s%s%s%s%s%s%s%s%s%s%s\n",
+ "s%s%s%s%s%s%s%s%s%s%s%s\n",
/* These must stay in alphabetical order. */
rs->is_authority?" Authority":"",
rs->is_bad_exit?" BadExit":"",
rs->is_exit?" Exit":"",
rs->is_fast?" Fast":"",
rs->is_possible_guard?" Guard":"",
+ rs->is_hs_dir?" HSDir":"",
rs->is_named?" Named":"",
rs->is_stable?" Stable":"",
rs->is_running?" Running":"",
@@ -1713,6 +1714,7 @@
(exits_can_be_guards ? guard_bandwidth_including_exits :
guard_bandwidth_excluding_exits);
rs->is_bad_exit = listbadexits && ri->is_bad_exit;
+ rs->is_hs_dir = ri->is_hs_dir; /* TODO only the very first approach... */
/* 0.1.1.9-alpha is the first version to support fetch by descriptor
* hash. */
rs->is_v2_dir = ri->dir_port &&
Modified: tor/branches/114-dist-storage/src/or/dirvote.c
===================================================================
--- tor/branches/114-dist-storage/src/or/dirvote.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/dirvote.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -71,7 +71,7 @@
return NULL;
}
-/** Helper for sorting a list of time_t* */
+/** Helper for sorting a list of time_t*. */
static int
_compare_times(const void **_a, const void **_b)
{
@@ -84,7 +84,7 @@
return 0;
}
-/** Helper for sorting a list of int* */
+/** Helper for sorting a list of int*. */
static int
_compare_ints(const void **_a, const void **_b)
{
@@ -164,7 +164,7 @@
}
/** Given a sorted list of strings <b>lst</b>, return the member that appears
- * most. Break ties in favor of later-occuring members. */
+ * most. Break ties in favor of later-occurring members. */
static const char *
get_most_frequent_member(smartlist_t *lst)
{
Modified: tor/branches/114-dist-storage/src/or/dns.c
===================================================================
--- tor/branches/114-dist-storage/src/or/dns.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/dns.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1407,6 +1407,9 @@
name[len] = '\0';
strlcat(name, suffix, sizeof(name));
+ log_info(LD_EXIT, "Testing whether our DNS server is hijacking nonexistent "
+ "domains with requrest for bogus hostname \"%s\"", name);
+
addr = tor_strdup(name);
r = evdns_resolve_ipv4(name, DNS_QUERY_NO_SEARCH,
evdns_wildcard_check_callback, addr);
Modified: tor/branches/114-dist-storage/src/or/dnsserv.c
===================================================================
--- tor/branches/114-dist-storage/src/or/dnsserv.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/dnsserv.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -109,8 +109,6 @@
return;
}
- /* XXXX020 Send a stream event to the controller. */
-
/* Make a new dummy AP connection, and attach the request to it. */
conn = TO_EDGE_CONN(connection_new(CONN_TYPE_AP, AF_INET));
conn->_base.state = AP_CONN_STATE_RESOLVE_WAIT;
@@ -131,6 +129,8 @@
connection_add(TO_CONN(conn));
+ control_event_stream_status(conn, STREAM_EVENT_NEW, 0);
+
/* Now, throw the connection over to get rewritten (which will answer it
* immediately if it's in the cache, or completely bogus, or automapped),
* and then attached to a circuit. */
Modified: tor/branches/114-dist-storage/src/or/main.c
===================================================================
--- tor/branches/114-dist-storage/src/or/main.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/main.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1698,6 +1698,8 @@
/* Have the log set up with our application name. */
tor_snprintf(buf, sizeof(buf), "Tor %s", get_version());
log_set_application_name(buf);
+ /* Initialize threading. */
+ tor_threads_init();
/* Initialize the history structures. */
rep_hist_init();
/* Initialize the service cache. */
Modified: tor/branches/114-dist-storage/src/or/or.h
===================================================================
--- tor/branches/114-dist-storage/src/or/or.h 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/or.h 2007-07-09 15:13:19 UTC (rev 10774)
@@ -367,7 +367,13 @@
#define DIR_PURPOSE_UPLOAD_RENDDESC 9
/** Purpose for connection at a directory server. */
#define DIR_PURPOSE_SERVER 10
-#define _DIR_PURPOSE_MAX 10
+/** A connection to a directory server: upload a v2 rendezvous
+ * descriptor. */
+#define DIR_PURPOSE_UPLOAD_RENDDESC_V2 11
+/** A connection to a directory server: download a v2 rendezvous
+ * descriptor. */
+#define DIR_PURPOSE_FETCH_RENDDESC_V2 12
+#define _DIR_PURPOSE_MAX 12
#define _EXIT_PURPOSE_MIN 1
/** This exit stream wants to do an ordinary connect. */
@@ -597,6 +603,9 @@
/** Length of 'y' portion of 'y.onion' URL. */
#define REND_SERVICE_ID_LEN 16
+/** Length of v2 descriptor ID (32 base32 chars = 160 bits). */
+#define REND_DESC_ID_V2_LEN 32
+
#define CELL_DIRECTION_IN 1
#define CELL_DIRECTION_OUT 2
@@ -905,6 +914,11 @@
char rend_query[REND_SERVICE_ID_LEN+1]; /**< What rendezvous service are we
* querying for? (AP only) */
+ uint8_t rend_version; /** What rendezvous version are we querying for? */
+
+ char secret_cookie[24+1]; /** Secret cookie for decrypting introduction
+ * points, if this request is for fetching a v2 rend desc. */
+
/** Number of times we've reassigned this application connection to
* a new circuit. We keep track because the timeout is longer if we've
* already retried several times. */
@@ -952,6 +966,10 @@
char identity_digest[DIGEST_LEN]; /**< Hash of the public RSA key for
* the directory server's signing key. */
+ uint8_t rend_version; /** What rendezvous version are we querying for? */
+
+ char secret_cookie[24+1]; /** Secret cookie for decrypting introduction
+ * points, if this request is for fetching a v2 rend desc. */
} dir_connection_t;
/** Subtype of connection_t for an connection to a controller. */
@@ -1149,6 +1167,8 @@
unsigned int is_exit:1; /**< Do we think this is an OK exit? */
unsigned int is_bad_exit:1; /**< Do we think this exit is censored, borked,
* or otherwise nasty? */
+ unsigned int is_hs_dir:1; /** True iff this router is a hidden service
+ * directory. */
/** Tor can use this router for general positions in circuits. */
#define ROUTER_PURPOSE_GENERAL 0
@@ -1222,6 +1242,8 @@
* an exit node. */
unsigned int is_bad_directory:1; /**< Do we think this directory is junky,
* underpowered, or otherwise useless? */
+ unsigned int is_hs_dir:1; /** True iff this router is a hidden service
+ * directory. */
/** True iff we know version info for this router. (i.e., a "v" entry was
* included.) We'll replace all these with a big tor_version_t or a char[]
* if the number of traits we care about ever becomes incredibly big. */
@@ -1374,6 +1396,8 @@
tor_mmap_t *mmap_descriptors;
/** Mmaped file holding extra-info documents. */
tor_mmap_t *mmap_extrainfo;
+ /** List of hidden service directories, sorted by their descriptor IDs. */
+ smartlist_t *hs_dirs;
} routerlist_t;
/** Information on router used when extending a circuit. We don't need a
@@ -1628,6 +1652,11 @@
*/
char rend_query[REND_SERVICE_ID_LEN+1];
+ uint8_t rend_version; /** What rendezvous version are we querying for? */
+
+ char secret_cookie[24+1]; /** Secret cookie for decrypting introduction
+ * points, if this request is for fetching a v2 rend desc. */
+
/** The next stream_id that will be tried when we're attempting to
* construct a new AP stream originating at this circuit. */
uint16_t next_stream_id;
@@ -1871,14 +1900,10 @@
/** Boolean: do we publish hidden service descriptors to the HS auths? */
int PublishHidServDescriptors;
int PublishV2HidServDescriptors; /**< and v2 hidden service descriptors? */
- /** do we store v2 hidden service descriptors to disk? */
- int StoreV2HidServDescriptorsToDisk;
- /** do we first try to read v2 hidden service descriptors from disk? */
- int LoadV2HidServDescriptorsFromDisk;
int FetchServerDescriptors; /**< Do we fetch server descriptors as normal? */
int FetchHidServDescriptors; /** and hidden service descriptors? */
int FetchV2HidServDescriptors; /** and v2 hidden service descriptors? */
-// int StoreAndServeV2HidServDescriptors; /** Do we act as hs dir? */
+ int HSDir; /** Do we act as hs dir? */
int FetchUselessDescriptors; /**< Do we fetch non-running descriptors too? */
int AllDirActionsPrivate; /**< Should every directory action be sent
* through a Tor circuit? */
@@ -2697,6 +2722,9 @@
int decode_hex, int sort_uniq);
char *directory_dump_request_log(void);
int router_supports_extrainfo(const char *identity_digest, int is_authority);
+void directory_post_to_hs_dir(const char *desc_id, const char *payload);
+void directory_get_from_hs_dir(const char *desc_id, const char *query,
+ const char *secret_cookie);
/********************************* dirserv.c ***************************/
@@ -3039,8 +3067,11 @@
int rend_client_introduction_acked(origin_circuit_t *circ, const char *request,
size_t request_len);
void rend_client_refetch_renddesc(const char *query);
+void rend_client_refetch_v2_renddesc(const char *query,
+ const char *secret_cookie);
int rend_client_remove_intro_point(extend_info_t *failed_intro,
- const char *query);
+ const char *query, uint8_t rend_version,
+ const char *secret_cookie);
int rend_client_rendezvous_acked(origin_circuit_t *circ, const char *request,
size_t request_len);
int rend_client_receive_rendezvous(origin_circuit_t *circ, const char *request,
@@ -3051,7 +3082,6 @@
int rend_client_send_introduction(origin_circuit_t *introcirc,
origin_circuit_t *rendcirc);
-int rend_load_service_descriptor_from_disk(const char *service_id, time_t now);
/********************************* rendcommon.c ***************************/
@@ -3073,7 +3103,7 @@
* its elements correspond to the elements of intro_points. */
extend_info_t **intro_point_extend_info;
/** Descriptor identifiers (current and next) for distributed storage. */
- char desc_id[2][20];
+ char desc_id[2][32+1];
/** Number of seconds that the current desc_id will be valid. */
int seconds_valid;
/** Permanent part of the computation of desc_id that is derived from pk
@@ -3081,7 +3111,7 @@
char permanent_id[10];
/** Changing part (current and next) of the computation of desc_id that is
* computed from time_period and secret_cookie. */
- char secret_id_part[2][20];
+ char secret_id_part[2][32+1];
/** Derived value of current and next time period. */
char time_period[2][4];
/** AES-encrypted list of introduction points. */
@@ -3092,7 +3122,7 @@
* introduction points. Is randomly generated by the hidden service provider
* on first configuration of the service and passed by a client as part of
* the onion address; storing nodes do not know this and set it to NULL. */
- char secret_cookie[16];
+ char secret_cookie[24+1];
} rend_service_descriptor_t;
int rend_cmp_service_ids(const char *one, const char *two);
@@ -3122,19 +3152,19 @@
void rend_cache_clean(void);
void rend_cache_free_all(void);
int rend_valid_service_id(const char *query);
-int rend_valid_v2_service_id(const char *query);
int rend_cache_lookup_desc(const char *query, int version, const char **desc,
size_t *desc_len);
int rend_cache_lookup_entry(const char *query, int version,
rend_cache_entry_t **entry_out);
-int rend_cache_store(const char *desc, size_t desc_len, int published,
- int version, const char *secret_cookie);
+int rend_cache_lookup_v2_dir(const char *query, const char **desc,
+ size_t *desc_len);
+int rend_cache_store(const char *desc, size_t desc_len, int published);
+int rend_cache_store_v2_client(const char *desc, const char *secret_cookie);
+int rend_cache_store_v2_dir(const char *desc);
int rend_cache_size(void);
int rend_encode_v2_descriptor(char *current_desc, char *next_desc,
rend_service_descriptor_t *desc,
- time_t now, //crypto_pk_env_t *key,
- const char *secret_cookie);//,
- //char *descriptor_id_base32);
+ time_t now, const char *secret_cookie);
int rend_compute_desc_id(char *desc_id, const char *permanent_id,
const char *secret_cookie, time_t now);
@@ -3404,6 +3434,8 @@
int trusted_dirs_load_certs_from_string(const char *contents, int from_store);
void trusted_dirs_flush_certs_to_disk(void);
+routerstatus_t *get_responsible_hs_dir(const char *id);
+
/********************************* routerparse.c ************************/
#define MAX_STATUS_TAG_LEN 32
Modified: tor/branches/114-dist-storage/src/or/rendclient.c
===================================================================
--- tor/branches/114-dist-storage/src/or/rendclient.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/rendclient.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -221,7 +221,8 @@
* If none remain, refetch the service descriptor.
*/
if (rend_client_remove_intro_point(circ->build_state->chosen_exit,
- circ->rend_query) > 0) {
+ circ->rend_query, circ->rend_version,
+ circ->secret_cookie) > 0) {
/* There are introduction points left. Re-extend the circuit to
* another intro point and try again. */
extend_info_t *extend_info;
@@ -247,57 +248,6 @@
return 0;
}
-/* Loads the rendezvous service descriptor for <b>address</b> from a file with
- * file name "<corresponding descriptor id>.onion" from the data directory,
- * parses it and stores it to the descritor cache, if it is valid. Returns 0 if
- * successful, -1 otherwise. */
-int
-rend_load_service_descriptor_from_disk(const char *address, time_t now)
-{
-
- char *s = NULL; /* string read from file */
- char *fname; /* file name */
- int flen; /* length of file name */
- char service_id[10]; /* service id part of address in binary format */
- char secret_cookie[15]; /* cookie part of address in binary format */
- char descriptor_id_digest[20]; /* computed descriptor id in binary format */
- char descriptor_id_base32[32+1]; /* base32-formatted descriptor id */
-
- /* address is formatted like "oo4qinerb64ybxkd.xvqwspfbixjhz2i2qwixb2ej" */
-
- /* extract service id */
- base32_decode(service_id, 10, address, 16);
-
- /* extract cookie */
- base32_decode(secret_cookie, 15, address+17, 24);
-
- /* compute current descriptor id */
- rend_compute_desc_id(descriptor_id_digest, service_id, secret_cookie, now);
-
- /* encode base 32 representation of descriptor id */
- base32_encode(descriptor_id_base32, 32+1, descriptor_id_digest, 20);
-
- /* construct file name */
- flen = strlen(get_options()->DataDirectory) + 40;
- fname = tor_malloc(flen);
- tor_snprintf(fname, flen, "%s"PATH_SEPARATOR"%s.rsd",
- get_options()->DataDirectory, descriptor_id_base32);
-
- /* read string from file */
- if (!(s = read_file_to_str(fname, 0, NULL))) {
- log_warn(LD_FS, "Unable to read descriptor \"%s\"", fname);
- return -1;
- }
-
- /* parse and store descriptor */
- if (rend_cache_store(s, strlen(s), 0, 2, secret_cookie) < 0) {
- log_warn(LD_FS, "Could not add descriptor to the cache");
- return -1;
- }
-
- return 0;
-}
-
/** If we are not currently fetching a rendezvous service descriptor
* for the service ID <b>query</b>, start a directory connection to fetch a
* new one.
@@ -305,12 +255,8 @@
void
rend_client_refetch_renddesc(const char *query)
{
- if (!get_options()->FetchHidServDescriptors &&
- !get_options()->FetchV2HidServDescriptors)
+ if (!get_options()->FetchHidServDescriptors)
return;
-
- /* TODO fetching v2 hidden service descriptors not implemented yet! */
-
if (connection_get_by_type_state_rendquery(CONN_TYPE_DIR, 0, query)) {
log_info(LD_REND,"Would fetch a new renddesc here (for %s), but one is "
"already in progress.", escaped_safe_str(query));
@@ -320,6 +266,34 @@
}
}
+/** If we are not currently fetching a rendezvous service descriptor
+ * for the service ID <b>query</b>, start a directory connection to fetch a
+ * new one.
+ *
+ * query = 16 base32 chars, secret_cookie = 24 base32 chars
+ */
+void
+rend_client_refetch_v2_renddesc(const char *query, const char *secret_cookie)
+{
+ tor_assert(query);
+ tor_assert(strlen(query) == 16);
+ tor_assert(secret_cookie);
+ tor_assert(strlen(secret_cookie) == 24);
+
+ if (!get_options()->FetchV2HidServDescriptors) {
+ log_warn(LD_DIR, "We received an onion address for a v2 service "
+ "descriptor, but fetching v2 service descriptors is not configured. "
+ "You need to set the config option FetchV2HidServDescriptors in order "
+ "to perform this request!");
+ return;
+ }
+ char descriptor_id[32+1];
+ rend_compute_desc_id(descriptor_id, query,
+ secret_cookie, time(NULL));
+ directory_get_from_hs_dir(descriptor_id, query, secret_cookie);
+ return;
+}
+
/** Remove failed_intro from ent. If ent now has no intro points, or
* service is unrecognized, then launch a new renddesc fetch.
*
@@ -327,7 +301,8 @@
* unrecognized, 1 if recognized and some intro points remain.
*/
int
-rend_client_remove_intro_point(extend_info_t *failed_intro, const char *query)
+rend_client_remove_intro_point(extend_info_t *failed_intro, const char *query,
+ uint8_t rend_version, const char *secret_cookie)
{
int i, r;
rend_cache_entry_t *ent;
@@ -341,7 +316,10 @@
if (r==0) {
log_info(LD_REND, "Unknown service %s. Re-fetching descriptor.",
escaped_safe_str(query));
- rend_client_refetch_renddesc(query);
+ if (rend_version == 2)
+ rend_client_refetch_v2_renddesc(query, secret_cookie);
+ else
+ rend_client_refetch_renddesc(query);
return 0;
}
@@ -377,7 +355,10 @@
log_info(LD_REND,
"No more intro points remain for %s. Re-fetching descriptor.",
escaped_safe_str(query));
- rend_client_refetch_renddesc(query);
+ if (rend_version == 2)
+ rend_client_refetch_v2_renddesc(query, secret_cookie);
+ else
+ rend_client_refetch_renddesc(query);
/* move all pending streams back to renddesc_wait */
while ((conn = connection_get_by_type_state_rendquery(CONN_TYPE_AP,
Modified: tor/branches/114-dist-storage/src/or/rendcommon.c
===================================================================
--- tor/branches/114-dist-storage/src/or/rendcommon.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/rendcommon.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -52,7 +52,6 @@
rend_compute_v2_descriptor_fields(rend_service_descriptor_t *desc,
time_t now, const char *secret_cookie)
{
-
char permanent_id_temp[DIGEST_LEN];
double time_period_part1;
double time_period_part2;
@@ -60,14 +59,19 @@
uint32_t time_period;
int i;
crypto_digest_env_t *digest;
+ char desc_id_binary[20];
+ char secret_id_part_bytes[20];
/* obtain permanent-id from public key */
crypto_pk_get_digest(desc->pk, permanent_id_temp);
memcpy(desc->permanent_id, permanent_id_temp, 10);
- /* copy secret_cookie */
- memcpy(desc->secret_cookie, secret_cookie, 15);
- desc->secret_cookie[15] = 0;
+ /* secret_cookie */
+ strlcpy(desc->secret_cookie, secret_cookie, sizeof(desc->secret_cookie));
+ char secret_cookie_bin[16];
+ base32_decode(secret_cookie_bin, sizeof(secret_cookie_bin),
+ desc->secret_cookie, strlen(desc->secret_cookie));
+ secret_cookie_bin[15] = 0;
/* calculate current time-period */
time_period_part1 = ((double) now) / ((double) TIME_PERIOD_LENGTH);
@@ -94,16 +98,19 @@
/* secret-id-part = h(time-period + cookie) */
digest = crypto_new_digest_env();
crypto_digest_add_bytes(digest, desc->time_period[i], 4);
- crypto_digest_add_bytes(digest, desc->secret_cookie, 16);
- crypto_digest_get_digest(digest, desc->secret_id_part[i], 20);
+ crypto_digest_add_bytes(digest, secret_cookie_bin, 16);
+ crypto_digest_get_digest(digest, secret_id_part_bytes, 20);
crypto_free_digest_env(digest);
/* descriptor id */
digest = crypto_new_digest_env();
crypto_digest_add_bytes(digest, desc->permanent_id, 10);
- crypto_digest_add_bytes(digest, desc->secret_id_part[i], 20);
- crypto_digest_get_digest(digest, desc->desc_id[i], 20);
+ crypto_digest_add_bytes(digest, secret_id_part_bytes, 20);
+ crypto_digest_get_digest(digest, desc_id_binary, 20);
crypto_free_digest_env(digest);
+
+ base32_encode(desc->secret_id_part[i], 32+1, secret_id_part_bytes, 20);
+ base32_encode(desc->desc_id[i], 32+1, desc_id_binary, 20);
}
return 0;
@@ -122,8 +129,20 @@
uint32_t time_period;
crypto_digest_env_t *digest;
char secret_id_part_digest[20];
- char secret_cookie_16[16];
-
+ char service_id_binary[10]; /* service id part of address in binary format */
+ char secret_cookie_binary[16]; /* cookie part of address in binary format;
+ the 16th byte is by definition 0. */
+ char descriptor_id_binary[20]; /* computed descriptor id in binary format */
+ //char descriptor_id[32+1];
+ /* check assertions */
+ tor_assert(service_id);
+ tor_assert(strlen(service_id) == 16);
+ tor_assert(secret_cookie);
+ tor_assert(strlen(secret_cookie) == 24);
+ /* decode base32 strings to binary strings */
+ base32_decode(service_id_binary, 10, service_id, 16);
+ base32_decode(secret_cookie_binary, 15, secret_cookie, 24);
+ secret_cookie_binary[15] = 0;
/* time-period */
time_period_part1 = ((double) now) / ((double) TIME_PERIOD_LENGTH);
permanent_id_value=((((uint64_t) service_id[0]) + 256) % 256) << 32;
@@ -138,23 +157,20 @@
time_period_bytes[1] = (char) (time_period >> 16);
time_period_bytes[2] = (char) (time_period >> 8);
time_period_bytes[3] = (char) time_period;
-
/* secret-id-part = h(time-period + cookie) */
digest = crypto_new_digest_env();
crypto_digest_add_bytes(digest, time_period_bytes, 4);
- memcpy(secret_cookie_16, secret_cookie, 15);
- secret_cookie_16[15] = 0;
- crypto_digest_add_bytes(digest, secret_cookie_16, 16);
+ crypto_digest_add_bytes(digest, secret_cookie_binary, 16);
crypto_digest_get_digest(digest, secret_id_part_digest, 20);
crypto_free_digest_env(digest);
-
/* descriptor id */
digest = crypto_new_digest_env();
- crypto_digest_add_bytes(digest, service_id, 10);
+ crypto_digest_add_bytes(digest, service_id_binary, 10);
crypto_digest_add_bytes(digest, secret_id_part_digest, 20);
- crypto_digest_get_digest(digest, desc_id, 20);
+ crypto_digest_get_digest(digest, descriptor_id_binary, 20);
crypto_free_digest_env(digest);
-
+ /* encode binary string to base32 string */
+ base32_encode(desc_id, 32+1, descriptor_id_binary, 20);
return 0;
}
@@ -172,8 +188,8 @@
size_t iposlen; /* max length of unencrypted introduction points */
char *ipos; /* unencrypted introduction points */
int ipowritten; /* actual length of unencrypted introduction points */
- char secret_id_part_base32[32+1];
- char descriptor_id_base32[32+1];
+ //char secret_id_part_base32[32+1];
+ //char descriptor_id_base32[32+1];
int i;
char *ipos_enc;
int enclen;
@@ -182,15 +198,17 @@
size_t written = 0;
char desc_digest[DIGEST_LEN];
char *buf;
+ /* check assertions */
+ tor_assert(desc);
+ tor_assert(secret_cookie);
+ tor_assert(strlen(secret_cookie) == 24);
/* prepare descriptor */
rend_compute_v2_descriptor_fields(desc, now, secret_cookie);
/* PEM-encode the public key */
- if (crypto_pk_write_public_key_to_string(desc->pk, &pkey, &pkeylen) < 0) {
- log_warn(LD_DIR, "write public key failed!");
+ if (crypto_pk_write_public_key_to_string(desc->pk, &pkey, &pkeylen) < 0)
return -1;
- }
/* encode timestamp */
format_iso_time(published, desc->timestamp);
@@ -257,7 +275,11 @@
/* encrypt introduction points */
ipos_enc = tor_malloc_zero(ipowritten + 32);
- enclen = crypto_cipher_encrypt_cbc(desc->secret_cookie, ipos_enc,
+ char secret_cookie_bin[16];
+ base32_decode(secret_cookie_bin, sizeof(secret_cookie_bin),
+ desc->secret_cookie, strlen(desc->secret_cookie));
+ secret_cookie_bin[15] = 0;
+ enclen = crypto_cipher_encrypt_cbc(secret_cookie_bin, ipos_enc,
ipos, ipowritten);
/* free memory for unencrypted introduction points */
@@ -274,10 +296,10 @@
/* encode both, the current and the next descriptor */
for (i = 0; i < 2; i++) {
buf = (i == 0 ? current_desc : next_desc);
- base32_encode(secret_id_part_base32, 32+1, desc->secret_id_part[i], 20);
- base32_encode(descriptor_id_base32, 32+1, desc->desc_id[i], 20);
+ //base32_encode(secret_id_part_base32, 32+1, desc->secret_id_part[i], 20);
+ //base32_encode(descriptor_id_base32, 32+1, desc->desc_id[i], 20);
- /* complete descriptor; TODO make 4000 a little more dynamic */
+ /* complete descriptor; TODO114 make 4000 a little more dynamic */
result = tor_snprintf(buf, 4000,
"rendezvous-service-descriptor %s\n"
"version 2\n"
@@ -287,9 +309,9 @@
"protocol-versions %d\n"
"introduction-points\n-----BEGIN AES ENCRYPTED MESSAGE-----\n%s"
"-----END AES ENCRYPTED MESSAGE-----\n",
- descriptor_id_base32,
+ desc->desc_id[i],
pkey,
- secret_id_part_base32,
+ desc->secret_id_part[i],
published,
desc->protocols,
ipos_encrypted_base64);
@@ -319,7 +341,6 @@
}
buf[written++] = '\n';
buf[written++] = 0;
-
}
/* free memory */
@@ -579,34 +600,23 @@
}
}
-/** Return true iff <b>query</b> is a syntactically valid service ID (as
- * generated by rend_get_service_id). */
+/** Return the version number (1 or 2) of <b>query</b> iff is a syntactically
+ * valid service ID (as generated by rend_get_service_id), or 0 otherwise. */
int
rend_valid_service_id(const char *query)
{
- if (strlen(query) != REND_SERVICE_ID_LEN)
- return 0;
-
- if (strspn(query, BASE32_CHARS) != REND_SERVICE_ID_LEN)
- return 0;
-
- return 1;
+ if (strlen(query) == REND_SERVICE_ID_LEN) {
+ if (strspn(query, BASE32_CHARS) == REND_SERVICE_ID_LEN)
+ return 1;
+ } else if (strlen(query) == 16+1+24) {
+ if (strspn(query, BASE32_CHARS) == 16 &&
+ strspn(query+16, ".") == 1 &&
+ strspn(query+17, BASE32_CHARS) == 24)
+ return 2;
+ }
+ return 0;
}
-/** Return true iff <b>query</b> is a syntactically valid v2 service ID */
-int
-rend_valid_v2_service_id(const char *query)
-{
- if (strlen(query) != 16 + 1 + 24)
- return 0;
- if (strspn(query, BASE32_CHARS) != 40
- && strspn(query+16, ".") != 1
- && strspn(query+17, BASE32_CHARS) != 24)
- return 0;
-
- return 1;
-}
-
/** If we have a cached rend_cache_entry_t for the service ID <b>query</b>,
* set *<b>e</b> to that entry and return 1. Else return 0. If
* <b>version</b> is nonnegative, only return an entry in that descriptor
@@ -616,34 +626,37 @@
int
rend_cache_lookup_entry(const char *query, int version, rend_cache_entry_t **e)
{
- char key[REND_SERVICE_ID_LEN+2]; /* 1<query>\0 or 0<query>\0 */
+ char key[REND_SERVICE_ID_LEN+2]; /* 2<query>\0, 1<query>\0, or 0<query>\0 */
tor_assert(rend_cache);
- /* if request is v2 formatted, remove secret_cookie part */
- if (rend_valid_v2_service_id(query)) {
- char query_without_cookie[REND_SERVICE_ID_LEN+1];
- *e = NULL;
- strncpy(query_without_cookie, query, REND_SERVICE_ID_LEN);
- tor_snprintf(key, sizeof(key), "1%s", query_without_cookie);
+log_warn(LD_DIR, "KL6 rend_cache_lookup_entry, query is %s, version is %d", query, version);
+ if (rend_valid_service_id(query) != 1) {
+log_warn(LD_DIR, "KL6 rend_cache_lookup_entry, no valid service id %s", query);
+ return -1;
+ }
+ *e = NULL;
+ /* version -1 or 2 */
+ if (version < 0 || version == 2) {
+log_warn(LD_DIR, "KL6 rend_cache_lookup_entry, looking up v2");
+ tor_snprintf(key, sizeof(key), "2%s", query);
*e = strmap_get_lc(rend_cache, key);
- if (!*e)
- return 0;
- return 1;
}
-
- /* no v2 request? look for v0 or v1 renddesc */
- if (!rend_valid_service_id(query))
- return -1;
- *e = NULL;
- if (version != 0) {
+ /* version -1 or 1 */
+ if (!*e && (version < 0 || version == 1)) {
+log_warn(LD_DIR, "KL6 rend_cache_lookup_entry, looking up v1");
tor_snprintf(key, sizeof(key), "1%s", query);
*e = strmap_get_lc(rend_cache, key);
}
- if (!*e && version != 1) {
+ /* version -1 or 0 */
+ if (!*e && version < 1) {
+log_warn(LD_DIR, "KL6 rend_cache_lookup_entry, looking up v0");
tor_snprintf(key, sizeof(key), "0%s", query);
*e = strmap_get_lc(rend_cache, key);
}
- if (!*e)
+ if (!*e) {
+log_warn(LD_DIR, "KL6 rend_cache_lookup_entry, nothing found");
return 0;
+ }
+log_warn(LD_DIR, "KL6 rend_cache_lookup_entry, found something!");
return 1;
}
@@ -668,7 +681,29 @@
return 1;
}
-/** Parse *desc, calculate its service id, and store it in the cache.
+/** query is 32 chars long, write result in e.
+ */
+int
+rend_cache_lookup_v2_dir(const char *query, const char **desc,
+ size_t *desc_len)
+{
+ rend_cache_entry_t *e = NULL;
+ tor_assert(rend_cache);
+ tor_assert(strlen(query) == 32);
+ char key[REND_DESC_ID_V2_LEN+2]; /* 2<query>\0 */
+ tor_snprintf(key, sizeof(key), "2%s", query);
+ e = (rend_cache_entry_t*) strmap_get_lc(rend_cache, key);
+ if (e) {
+ *desc = e->desc;
+ *desc_len = e->len;
+ return 1;
+ }
+ return 0;
+}
+
+/** here happens the actual storing of a previously parsed descriptor, whether
+ * client-side or directory-side.
+ *
* If we have a newer descriptor with the same ID, ignore this one.
* If we have an older descriptor with the same ID, replace it.
* Return -1 if it's malformed or otherwise rejected; return 0 if
@@ -678,43 +713,20 @@
* The version can be either >= 2 for v2 descriptors or <= 1 for v0/v1
* descriptors. If published is 0, secret_cookie needs to contain a key,
* otherwise it is NULL.
+ *
+ * parsed = parsed descriptor, key = key under which it will be stored,
+ * published = flag, whether this request was triggered by a directory store
+ * operation (true) or by a client (false).
*/
-int
-rend_cache_store(const char *desc, size_t desc_len, int published, int version,
- const char *secret_cookie)
+static int
+rend_cache_store_parsed(const char *query, rend_service_descriptor_t *parsed,
+ const char *desc, size_t desc_len, const char *key,
+ int published)
{
rend_cache_entry_t *e;
- rend_service_descriptor_t *parsed;
- char query[REND_SERVICE_ID_LEN+1];
- char key[REND_SERVICE_ID_LEN+2]; /* 1<query>\0 or 0<query>\0 */
- time_t now;
+ time_t now = time(NULL);
or_options_t *options = get_options();
- tor_assert(rend_cache);
- if (version >= 2) {
- parsed = tor_malloc_zero(sizeof(rend_service_descriptor_t));
- if (rend_parse_v2_service_descriptor(parsed, desc) < 0) {
- log_warn(LD_FS, "Could not parse descriptor");
- return -1;
- }
- if (!published &&
- rend_decrypt_introduction_points(parsed, secret_cookie) < 0) {
- log_warn(LD_PROTOCOL,"Couldn't decrypt introduction points.");
- return -1;
- }
- } else {
- parsed = rend_parse_service_descriptor(desc,desc_len);
- if (!parsed) {
- log_warn(LD_PROTOCOL,"Couldn't parse service descriptor.");
- return -1;
- }
- }
- if (rend_get_service_id(parsed->pk, query)<0) {
- log_warn(LD_BUG,"Couldn't compute service ID.");
- rend_service_descriptor_free(parsed);
- return -1;
- }
- tor_snprintf(key, sizeof(key), "%c%s", parsed->version?'1':'0', query);
- now = time(NULL);
+log_warn(LD_DIR, "KL6 storing desc under key %s", key);
if (parsed->timestamp < now-REND_CACHE_MAX_AGE-REND_CACHE_MAX_SKEW) {
log_fn(LOG_PROTOCOL_WARN, LD_REND,
"Service descriptor %s is too old.", safe_str(query));
@@ -767,6 +779,81 @@
return 1;
}
+/** Parse *desc, calculate its service id, and request to store it in the
+ * cache.
+ */
+int
+rend_cache_store(const char *desc, size_t desc_len, int published)
+{
+ rend_service_descriptor_t *parsed;
+ char query[REND_SERVICE_ID_LEN+1];
+ char key[REND_SERVICE_ID_LEN+2]; /* 1<query>\0 or 0<query>\0 */
+ tor_assert(rend_cache);
+ parsed = rend_parse_service_descriptor(desc,desc_len);
+ if (!parsed) {
+ log_warn(LD_PROTOCOL,"Couldn't parse service descriptor.");
+ return -1;
+ }
+ if (rend_get_service_id(parsed->pk, query)<0) {
+ log_warn(LD_BUG,"Couldn't compute service ID.");
+ rend_service_descriptor_free(parsed);
+ return -1;
+ }
+ tor_snprintf(key, sizeof(key), "%c%s", parsed->version?'1':'0', query);
+ return rend_cache_store_parsed(query, parsed, desc, desc_len, key,
+ published);
+}
+
+/** store a v2 desc in the local cache under its version and service id and
+ * decrypt the included list of introduction points with secret_cookie
+ */
+int
+rend_cache_store_v2_client(const char *desc, const char *secret_cookie)
+{
+ rend_service_descriptor_t *parsed;
+ char key[REND_SERVICE_ID_LEN+2];
+ char query[REND_SERVICE_ID_LEN+1];
+ tor_assert(rend_cache);
+ tor_assert(secret_cookie);
+ tor_assert(strlen(secret_cookie) == 24);
+ parsed = tor_malloc_zero(sizeof(rend_service_descriptor_t));
+ if (rend_parse_v2_service_descriptor(parsed, desc) < 0) {
+ log_warn(LD_FS, "Could not parse descriptor");
+ return -1;
+ }
+ if (rend_get_service_id(parsed->pk, query)<0) {
+ log_warn(LD_BUG,"Couldn't compute service ID.");
+ rend_service_descriptor_free(parsed);
+ return -1;
+ }
+ if (rend_decrypt_introduction_points(parsed, secret_cookie) < 0) {
+ log_warn(LD_PROTOCOL,"Couldn't decrypt introduction points.");
+ return -1;
+ }
+ tor_snprintf(key, sizeof(key), "2%s", query);
+ return rend_cache_store_parsed(query, parsed, desc, strlen(desc), key, 0);
+}
+
+/** store a v2 desc in the local cache under its version and descriptor ID,
+ * but don't attempt to decrypt the included list of introduction points
+ * (as we don't have the secret_cookie for it).
+ */
+int
+rend_cache_store_v2_dir(const char *desc)
+{
+ rend_service_descriptor_t *parsed;
+ char key[REND_DESC_ID_V2_LEN+2];
+ tor_assert(rend_cache);
+ parsed = tor_malloc_zero(sizeof(rend_service_descriptor_t));
+ if (rend_parse_v2_service_descriptor(parsed, desc) < 0) {
+ log_warn(LD_FS, "Could not parse descriptor");
+ return -1;
+ }
+ tor_snprintf(key, sizeof(key), "2%s", parsed->desc_id[0]);
+ return rend_cache_store_parsed(parsed->desc_id[0], parsed, desc,
+ strlen(desc), key, 1);
+}
+
/** Called when we get a rendezvous-related relay cell on circuit
* <b>circ</b>. Dispatch on rendezvous relay command. */
void
Modified: tor/branches/114-dist-storage/src/or/rendservice.c
===================================================================
--- tor/branches/114-dist-storage/src/or/rendservice.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/rendservice.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -325,23 +325,13 @@
static int
rend_get_hostname2(rend_service_t *service, char *hostname2)
{
-
char permanent_id_base32[16+1];
- char cookie_base32[24+1];
-
- /* base32-encode service id */
base32_encode(permanent_id_base32, 16+1, service->pk_digest, 10);
-
- /* base32-encode cookie */
- base32_encode(cookie_base32, 24+1, service->secret_cookie, 15);
-
- /* assemble onion address */
if (tor_snprintf(hostname2, 50, "%s.%s.onion\n", permanent_id_base32,
- cookie_base32) < 0) {
+ service->secret_cookie) < 0) {
log_warn(LD_BUG, "could not encode hostname");
return -1;
}
-
return 0;
}
@@ -357,10 +347,8 @@
char buf[128];
for (i=0; i < smartlist_len(rend_service_list); ++i) {
-
char hostname2[50];
char fname2[512];
-
s = smartlist_get(rend_service_list,i);
if (s->private_key)
continue;
@@ -391,8 +379,10 @@
return -1;
}
if (!(s->secret_cookie = read_file_to_str(fname, 0, NULL))) {
- s->secret_cookie = tor_malloc_zero(16);
- crypto_rand(s->secret_cookie, 16);
+ char secret_cookie_bin[15];
+ crypto_rand(secret_cookie_bin, 15);
+ s->secret_cookie = tor_malloc_zero(24+1);
+ base32_encode(s->secret_cookie, 24+1, secret_cookie_bin, 15);
if (write_str_to_file(fname, s->secret_cookie, 0) < 0) {
log_warn(LD_CONFIG, "Could not write secret_cookie file:"
" \"%s\".", s->directory);
@@ -983,31 +973,63 @@
return NULL;
}
-/** Encode and sign an up-to-date service descriptor for <b>service</b>,
- * and upload it to all the dirservers.
+/** Encode and sign up-to-date v0/1 and/or v2 service descriptors for
+ * <b>service</b>, and upload it to all the dirservers/to the responsible
+ * hidden service directories.
*/
static void
-upload_service_descriptor(rend_service_t *service, int version)
+upload_service_descriptor(rend_service_t *service)
{
- char *desc;
- size_t desc_len;
-
/* Update the descriptor. */
rend_service_update_descriptor(service);
- if (rend_encode_service_descriptor(service->desc,
- version,
- service->private_key,
- &desc, &desc_len)<0) {
- log_warn(LD_BUG, "Internal error: couldn't encode service descriptor; "
- "not uploading.");
- return;
+
+ /* Upload v0 or v1 descriptor? */
+ if (get_options()->PublishHidServDescriptors) {
+ char *desc;
+ size_t desc_len;
+
+ /* Encode the descriptor. */
+ if (rend_encode_service_descriptor(service->desc,
+ 0, // TODO114 is 0 correct?
+ service->private_key,
+ &desc, &desc_len)<0) {
+ log_warn(LD_BUG, "Internal error: couldn't encode service descriptor; "
+ "not uploading.");
+ return;
+ }
+
+log_warn(LD_REND, "KL6 this is our v1 descriptor of len %d: %s", desc_len, hex_str(desc, desc_len));
+
+ /* Post it to the dirservers */
+ directory_post_to_dirservers(DIR_PURPOSE_UPLOAD_RENDDESC,
+ HIDSERV_AUTHORITY, desc, desc_len, 0);
+ tor_free(desc);
}
- /* Post it to the dirservers */
- directory_post_to_dirservers(DIR_PURPOSE_UPLOAD_RENDDESC,
- HIDSERV_AUTHORITY, desc, desc_len, 0);
- tor_free(desc);
+ /* Upload v2 descriptor? */
+ if (get_options()->PublishV2HidServDescriptors) {
+ time_t now = time(NULL);
+ char current_desc[4000];
+ char next_desc[4000];
+ /* Encode the current and the next descriptor. */
+ if (rend_encode_v2_descriptor(current_desc, next_desc, service->desc, now,
+ service->secret_cookie) < 0) {
+ log_warn(LD_BUG, "Internal error: couldn't encode service descriptor; "
+ "not uploading.");
+ return;
+ }
+
+ /* Post the current descriptor to the hidden service directories. */
+ directory_post_to_hs_dir(service->desc->desc_id[0],
+ current_desc);
+
+ /* Post also the next descriptor, if necessary. TODO make 6 configurable */
+ if (service->desc->seconds_valid < 6)
+ directory_post_to_hs_dir(service->desc->desc_id[1],
+ next_desc);
+ }
+
service->desc_is_dirty = 0;
}
@@ -1125,62 +1147,6 @@
smartlist_free(exclude_routers);
}
-/* Stores the current and possibly next rendezvous service descriptor for
- * <b>service</b> at time <b>now</b> to one or two files with file name
- * "<corresponding descriptor id>.onion" to the data directory. Returns 0 if
- * successful, -1 otherwise. */
-static int
-rend_store_service_descriptor_to_disk(rend_service_t *service, time_t now)
-{
-
- char *current_desc;
- char *next_desc;
- size_t flen; // len of file name
- char *fname; // file name
- char descriptor_id_base32[32+1];
- int i, n = 1;
-
- /* TODO make 4000 configurable */
- current_desc = tor_malloc_zero(4000);
- next_desc = tor_malloc_zero(4000);
-
- /* TODO might be better to prevent this from being executed multiple times */
- rend_service_update_descriptor(service);
-
- /* encode descriptor */
- if (rend_encode_v2_descriptor(current_desc, next_desc, service->desc, now,
- service->secret_cookie) < 0) {
- log_warn(LD_DIR, "could not encode descriptor");
- return -1;
- }
-
- /* write to file */
- flen = strlen(get_options()->DataDirectory) + 40;
- fname = tor_malloc(flen);
-
- /* write one or two descriptors to disk? TODO make 6 configurable */
- if (service->desc->seconds_valid < 6) n = 2;
-
- for (i = 0; i < 2; i++) {
-
- /* base32-encode descriptor id */
- base32_encode(descriptor_id_base32, 32+1, service->desc->desc_id[i], 20);
-
- /* determine file name */
- tor_snprintf(fname, flen, "%s"PATH_SEPARATOR"%s.rsd",
- get_options()->DataDirectory, descriptor_id_base32);
-
- /* write string to file */
- write_str_to_file(fname,(i == 0 ? current_desc : next_desc),0);
- }
-
- tor_free(current_desc);
- tor_free(next_desc);
- tor_free(fname);
-
- return 0;
-}
-
/** Regenerate and upload rendezvous service descriptors for all
* services, if necessary. If the descriptor has been dirty enough
* for long enough, definitely upload; else only upload when the
@@ -1197,8 +1163,7 @@
int rendpostperiod = get_options()->RendPostPeriod;
if (!get_options()->PublishHidServDescriptors &&
- !get_options()->PublishV2HidServDescriptors &&
- !get_options()->StoreV2HidServDescriptorsToDisk)
+ !get_options()->PublishV2HidServDescriptors)
return;
for (i=0; i < smartlist_len(rend_service_list); ++i) {
@@ -1213,18 +1178,7 @@
/* if it's time, or if the directory servers have a wrong service
* descriptor and ours has been stable for 30 seconds, upload a
* new one of each format. */
- if (get_options()->StoreV2HidServDescriptorsToDisk) {
- if (rend_store_service_descriptor_to_disk(service, now) < 0) {
- log_warn(LD_DIR, "could not store descriptor to disk");
- }
- log_warn(LD_DIR, "stored descriptor to disk");
- }
- if (get_options()->PublishHidServDescriptors) {
- upload_service_descriptor(service, 0);
- }
-
- /* TODO publishing v2 hidden service descriptors not implemented yet! */
-
+ upload_service_descriptor(service);
service->next_upload_time = now + rendpostperiod;
}
}
Modified: tor/branches/114-dist-storage/src/or/router.c
===================================================================
--- tor/branches/114-dist-storage/src/or/router.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/router.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1451,7 +1451,7 @@
"opt extra-info-digest %s\n%s"
"onion-key\n%s"
"signing-key\n%s"
- "%s%s%s",
+ "%s%s%s%s",
router->nickname,
router->address,
router->or_port,
@@ -1467,7 +1467,9 @@
options->DownloadExtraInfo ? "opt caches-extra-info\n" : "",
onion_pkey, identity_pkey,
family_line, bandwidth_usage,
- we_are_hibernating() ? "opt hibernating 1\n" : "");
+ we_are_hibernating() ? "opt hibernating 1\n" : "",
+ options->HSDir ? "opt hidden-service-dir\n" : "");
+
tor_free(family_line);
tor_free(onion_pkey);
tor_free(identity_pkey);
Modified: tor/branches/114-dist-storage/src/or/routerlist.c
===================================================================
--- tor/branches/114-dist-storage/src/or/routerlist.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/routerlist.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1723,6 +1723,7 @@
routerlist->desc_digest_map = sdmap_new();
routerlist->desc_by_eid_map = sdmap_new();
routerlist->extra_info_map = eimap_new();
+ routerlist->hs_dirs = smartlist_create();
}
return routerlist;
}
@@ -1867,14 +1868,17 @@
static INLINE int
_routerlist_find_elt(smartlist_t *sl, void *ri, int idx)
{
- if (idx < 0 || smartlist_get(sl, idx) != ri) {
+ if (idx < 0) {
idx = -1;
SMARTLIST_FOREACH(sl, routerinfo_t *, r,
if (r == ri) {
idx = r_sl_idx;
break;
});
- }
+ } else {
+ tor_assert(idx < smartlist_len(sl));
+ tor_assert(smartlist_get(sl, idx) == ri);
+ };
return idx;
}
@@ -2106,8 +2110,10 @@
* it to rl->old_routers. */
static void
routerlist_replace(routerlist_t *rl, routerinfo_t *ri_old,
- routerinfo_t *ri_new, int idx, int make_old)
+ routerinfo_t *ri_new)
{
+ int idx;
+
routerinfo_t *ri_tmp;
extrainfo_t *ei_tmp;
routerlist_check_bug_417();
@@ -2119,12 +2125,16 @@
tor_assert(ri_old != ri_new);
tor_assert(ri_new->routerlist_index == -1);
- idx = _routerlist_find_elt(rl->routers, ri_old, idx);
+ idx = ri_old->routerlist_index;
+ tor_assert(0 <= idx && idx < smartlist_len(rl->routers));
+ tor_assert(smartlist_get(rl->routers, idx) == ri_old);
+
router_dir_info_changed();
if (idx >= 0) {
smartlist_set(rl->routers, idx, ri_new);
ri_old->routerlist_index = -1;
ri_new->routerlist_index = idx;
+ tor_assert( _routerlist_find_elt(rl->routers, ri_old, -1) == -1 );
} else {
log_warn(LD_BUG, "Appending entry from routerlist_replace.");
routerlist_insert(rl, ri_new);
@@ -2147,7 +2157,7 @@
&ri_new->cache_info);
}
- if (make_old && get_options()->DirPort &&
+ if (get_options()->DirPort &&
ri_old->purpose == ROUTER_PURPOSE_GENERAL) {
signed_descriptor_t *sd = signed_descriptor_from_routerinfo(ri_old);
smartlist_add(rl->old_routers, sd);
@@ -2350,6 +2360,8 @@
int authdir = authdir_mode(get_options());
int authdir_believes_valid = 0;
routerinfo_t *old_router;
+ /* This has side effects, so do it before we start the real work */
+ int have_dir_info = router_have_minimum_dir_info();
routerlist_check_bug_417();
tor_assert(msg);
@@ -2419,9 +2431,6 @@
old_router = rimap_get(routerlist->identity_map,
router->cache_info.identity_digest);
if (old_router) {
- int pos = old_router->routerlist_index;
- tor_assert(smartlist_get(routerlist->routers, pos) == old_router);
-
if (router->cache_info.published_on <=
old_router->cache_info.published_on) {
/* Same key, but old */
@@ -2448,9 +2457,8 @@
old_router->num_unreachable_notifications;
}
if (authdir && !from_cache && !from_fetch &&
- router_have_minimum_dir_info() &&
- dirserv_thinks_router_is_blatantly_unreachable(router,
- time(NULL))) {
+ have_dir_info &&
+ dirserv_thinks_router_is_blatantly_unreachable(router, time(NULL))) {
if (router->num_unreachable_notifications >= 3) {
unreachable = 1;
log_notice(LD_DIR, "Notifying server '%s' that it's unreachable. "
@@ -2465,7 +2473,7 @@
router->num_unreachable_notifications++;
}
}
- routerlist_replace(routerlist, old_router, router, pos, 1);
+ routerlist_replace(routerlist, old_router, router);
if (!from_cache) {
signed_desc_append_to_journal(&router->cache_info, router->purpose);
}
@@ -5369,3 +5377,34 @@
return info;
}
+/** Determines the router that is responsible for <b>desc_id</b> and returns
+ * that router's routerstatus_t.
+ *
+ * TODO114 At the moment we build up the router list in routerlist->hs_dirs for
+ * every invocation of this function; it should be possible to maintain an
+ * up-to-date router list.
+ *
+ * TODO114 When adding replication this function should return an array of
+ * responsible routers.
+ */
+routerstatus_t *
+get_responsible_hs_dir(const char *id)
+{
+ const char *digest;
+ char id_binary[20];
+ tor_assert(id);
+ tor_assert(strlen(id) == 32);
+ base32_decode(id_binary, 20, id, 32);
+ SMARTLIST_FOREACH(routerlist->routers, routerinfo_t *, ri,
+ {
+ if (ri->is_hs_dir) {
+ smartlist_add(routerlist->hs_dirs, ri->cache_info.identity_digest);
+ }
+ });
+ smartlist_sort_digests(routerlist->hs_dirs);
+ digest = smartlist_digest_next_circular(routerlist->hs_dirs, id);
+ local_routerstatus_t *router = router_get_combined_status_by_digest(digest);
+ routerstatus_t *status = &(router->status);
+ return status;
+}
+
Modified: tor/branches/114-dist-storage/src/or/routerparse.c
===================================================================
--- tor/branches/114-dist-storage/src/or/routerparse.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/routerparse.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -59,6 +59,7 @@
K_EXTRA_INFO,
K_EXTRA_INFO_DIGEST,
K_CACHES_EXTRA_INFO,
+ K_HIDDEN_SERVICE_DIR,
K_DIR_KEY_CERTIFICATE_VERSION,
K_DIR_IDENTITY_KEY,
@@ -207,6 +208,7 @@
T01("read-history", K_READ_HISTORY, ARGS, NO_OBJ ),
T01("write-history", K_WRITE_HISTORY, ARGS, NO_OBJ ),
T01("extra-info-digest", K_EXTRA_INFO_DIGEST, GE(1), NO_OBJ ),
+ T01("hidden-service-dir", K_HIDDEN_SERVICE_DIR, NO_ARGS, NO_OBJ ),
T01("family", K_FAMILY, ARGS, NO_OBJ ),
T01("caches-extra-info", K_CACHES_EXTRA_INFO, NO_ARGS, NO_OBJ ),
@@ -302,7 +304,7 @@
END_OF_TABLE
};
-/** List of tokens allowable in rendezvous service derscriptors */
+/** List of tokens allowable in rendezvous service descriptors */
static token_rule_t desc_token_table[] = {
T1("rendezvous-service-descriptor", R_RENDEZVOUS_SERVICE_DESCRIPTOR, EQ(1), \
NO_OBJ),
@@ -1177,6 +1179,10 @@
}
}
+ if ((tok = find_first_by_keyword(tokens, K_HIDDEN_SERVICE_DIR))) {
+ router->is_hs_dir = 1;
+ }
+
tok = find_first_by_keyword(tokens, K_ROUTER_SIGNATURE);
tor_assert(tok);
note_crypto_pk_op(VERIFY_RTR);
@@ -1576,6 +1582,8 @@
rs->is_bad_directory = 1;
else if (!strcmp(tok->args[i], "Authority"))
rs->is_authority = 1;
+ else if (!strcmp(tok->args[i], "HSDir"))
+ rs->is_hs_dir = 1;
}
}
if ((tok = find_first_by_keyword(tokens, K_V))) {
@@ -2856,7 +2864,7 @@
int protocols;
smartlist_t *tokens = NULL;
directory_token_t *tok;
- uint8_t version;
+ //uint8_t version;
/* point 'end' to a point immediately after the final newline. */
end = s + strlen(s);
@@ -2898,11 +2906,11 @@
if (parse_iso_time(tok->args[0], &result->timestamp) < 0)
goto err;
- /* parse version */
+ /* parse descriptor version */
tok = find_first_by_keyword(tokens, R_VERSION);
- version = atoi(tok->args[0]);
- if (version != 2) {
- log_warn(LD_DIR, "wrong descriptor version: %d", version);
+ result->version = atoi(tok->args[0]);
+ if (result->version != 2) {
+ log_warn(LD_DIR, "wrong descriptor version: %d", result->version);
goto err;
}
@@ -2915,22 +2923,13 @@
/* parse secret ID part */
tok = find_first_by_keyword(tokens, R_SECRET_ID_PART);
tor_assert(tok->n_args == 1);
- if (base32_decode(result->secret_id_part[0], 20, tok->args[0],
- strlen(tok->args[0]))) {
- log_warn(LD_DIR, "Couldn't decode secret id part %s",
- escaped(tok->args[0]));
- goto err;
- }
+ strlcpy(result->secret_id_part[0], tok->args[0],
+ sizeof(result->secret_id_part[0]));
/* parse descriptor ID */
tok = find_first_by_keyword(tokens, R_RENDEZVOUS_SERVICE_DESCRIPTOR);
tor_assert(tok->n_args == 1);
- if (base32_decode(result->desc_id[0], 20, tok->args[0],
- strlen(tok->args[0]))) {
- log_warn(LD_DIR, "Couldn't decode desc id %s",
- escaped(tok->args[0]));
- goto err;
- }
+ strlcpy(result->desc_id[0], tok->args[0], sizeof(result->desc_id[0]));
/* parse encrypted introduction points */
tok = find_first_by_keyword(tokens, R_INTRODUCTION_POINTS);
@@ -2938,13 +2937,8 @@
tok->object_body = NULL; /* Prevent free */
result->intro_points_encrypted_size = tok->object_size;
- /* parse descriptor version */
- tok = find_first_by_keyword(tokens, R_PROTOCOL_VERSIONS);
- log_warn(LD_DIR, "parsed protocol versions: %s", tok->args[0]);
- /* for the moment, use version 1 to keep things working TODO change */
- result->version = 1;
-
/* parse protocol versions */
+ tok = find_first_by_keyword(tokens, R_PROTOCOL_VERSIONS);
protocols = atoi(tok->args[0]);
result->protocols = protocols;
@@ -2979,13 +2973,16 @@
const char **s2; /* currently investigated intro point string */
int i;
+tor_assert(secret_cookie);
+
/* create smartlists */
tokens = smartlist_create();
intropoints = smartlist_create();
/* decrypt introduction points and store info in parsed */
ipos_decrypted = tor_malloc_zero(parsed->intro_points_encrypted_size);
- memcpy(key, secret_cookie, 15);
+ base32_decode(key, 15, secret_cookie, 24);
+ //memcpy(key, secret_cookie, 15);
key[15] = 0;
unenclen = crypto_cipher_decrypt_cbc(key, ipos_decrypted,
parsed->intro_points_encrypted,
Modified: tor/branches/114-dist-storage/src/or/test.c
===================================================================
--- tor/branches/114-dist-storage/src/or/test.c 2007-07-09 14:54:48 UTC (rev 10773)
+++ tor/branches/114-dist-storage/src/or/test.c 2007-07-09 15:13:19 UTC (rev 10774)
@@ -1368,18 +1368,24 @@
static void _thread_test_func(void* _s) ATTR_NORETURN;
+static int t1_count = 0;
+static int t2_count = 0;
+
static void
_thread_test_func(void* _s)
{
char *s = _s;
- int i;
+ int i, *count;
tor_mutex_t *m;
char buf[64];
char *cp;
- if (!strcmp(s, "thread 1"))
+ if (!strcmp(s, "thread 1")) {
m = _thread_test_start1;
- else
+ count = &t1_count;
+ } else {
m = _thread_test_start2;
+ count = &t2_count;
+ }
tor_mutex_acquire(m);
tor_snprintf(buf, sizeof(buf), "%lu", tor_get_thread_id());
@@ -1388,6 +1394,7 @@
for (i=0; i<100000; ++i) {
tor_mutex_acquire(_thread_test_mutex);
strmap_set(_thread_test_strmap, "last to run", cp);
+ ++*count;
tor_mutex_release(_thread_test_mutex);
}
tor_mutex_acquire(_thread_test_mutex);
@@ -1403,7 +1410,8 @@
test_threads(void)
{
char *s1, *s2;
- int done = 0;
+ int done = 0, timedout = 0;
+ time_t started;
#ifndef TOR_IS_MULTITHREADED
/* Skip this test if we aren't threading. We should be threading most
* everywhere by now. */
@@ -1422,16 +1430,27 @@
spawn_func(_thread_test_func, s2);
tor_mutex_release(_thread_test_start2);
tor_mutex_release(_thread_test_start1);
+ started = time(NULL);
while (!done) {
tor_mutex_acquire(_thread_test_mutex);
strmap_assert_ok(_thread_test_strmap);
if (strmap_get(_thread_test_strmap, "thread 1") &&
- strmap_get(_thread_test_strmap, "thread 2"))
+ strmap_get(_thread_test_strmap, "thread 2")) {
done = 1;
+ } else if (time(NULL) > started + 10) {
+ timedout = done = 1;
+ }
tor_mutex_release(_thread_test_mutex);
}
tor_mutex_free(_thread_test_mutex);
+ if (timedout) {
+ printf("\nTimed out: %d %d", t1_count, t2_count);
+ test_assert(strmap_get(_thread_test_strmap, "thread 1"));
+ test_assert(strmap_get(_thread_test_strmap, "thread 2"));
+ test_assert(!timedout);
+ }
+
/* different thread IDs. */
test_assert(strcmp(strmap_get(_thread_test_strmap, "thread 1"),
strmap_get(_thread_test_strmap, "thread 2")));
@@ -2775,16 +2794,26 @@
test_crypto();
test_crypto_dh();
test_crypto_s2k();
- puts("\n========================= Util ============================");
+ puts("\n========================= Util ============================"
+ "\n--IPv6");
test_ip6_helpers();
+ puts("\n--gzip");
test_gzip();
+ puts("\n--util");
test_util();
+ puts("\n--smartlist");
test_smartlist();
+ puts("\n--mempool");
test_mempool();
+ puts("\n--strmap");
test_strmap();
+ puts("\n--control formats");
test_control_formats();
+ puts("\n--pqueue");
test_pqueue();
+ puts("\n--mmap");
test_mmap();
+ puts("\n--threads");
test_threads();
puts("\n========================= Onion Skins =====================");
test_onion_handshake();
More information about the tor-commits
mailing list