[tor-commits] [torspec/master] faster headless consensus bootstrapping is now proposal 210

Thu Oct 11 14:38:33 UTC 2012

commit 7dd452336e4df3e4bcc66b40f7cbbd56416bd6a2
Author: Nick Mathewson <nickm at torproject.org>
Date:   Thu Oct 11 10:36:31 2012 -0400

    faster headless consensus bootstrapping is now proposal 210
---
 proposals/000-index.txt                            |    2 +
 .../210-faster-headless-consensus-bootstrap.txt    |  113 ++++++++++++++++++++
 .../xxx-faster-headless-consensus-bootstrap.txt    |  112 -------------------
 3 files changed, 115 insertions(+), 112 deletions(-)

diff --git a/proposals/000-index.txt b/proposals/000-index.txt
index 2863d79..64aea49 100644
--- a/proposals/000-index.txt
+++ b/proposals/000-index.txt
@@ -131,6 +131,7 @@ Proposals by number:
 208  IPv6 Exits Redux [OPEN]
 209  Limit reported bandwidth of unmeasured nodes [OPEN]
 209  Tuning the Parameters for the Path Bias Defense [OPEN]
+210  Faster Headless Consensus Bootstrapping [OPEN]
 
 
 Proposals by status:
@@ -178,6 +179,7 @@ Proposals by status:
    208  IPv6 Exits Redux [for 0.2.4.x]
    209  Limit reported bandwidth of unmeasured nodes [for 0.2.4.x]
    209  Tuning the Parameters for the Path Bias Defense [for 0.2.4.x+]
+   210  Faster Headless Consensus Bootstrapping [for 0.2.4.x+]
  ACCEPTED:
    117  IPv6 exits [for 0.2.4.x]
    140  Provide diffs between consensuses
diff --git a/proposals/210-faster-headless-consensus-bootstrap.txt b/proposals/210-faster-headless-consensus-bootstrap.txt
new file mode 100644
index 0000000..6b1502b
--- /dev/null
+++ b/proposals/210-faster-headless-consensus-bootstrap.txt
@@ -0,0 +1,113 @@
+Filename: 210-faster-headless-consensus-bootstrap.txt
+Title: Faster Headless Consensus Bootstrapping
+Author: Mike Perry
+Created: 01-10-2012
+Status: Open
+Target: 0.2.4.x+
+
+
+Overview and Motiviation
+
+ This proposal describes a way for clients to fetch the initial
+ consensus more quickly in situations where some or all of the directory
+ authorities are unreachable. This proposal is meant to describe a
+ solution for bug #4483.
+
+Design: Bootstrap Process Changes
+
+ The core idea is to attempt to establish bootstrap connections in
+ parallel during the bootstrap process, and download the consensus from
+ the first connection that completes.
+
+ Connection attempts will be done in batches of three. Only one
+ connection will be performed to one of the canonical directory
+ authorities. Two connections will be performed to randomly chosen hard
+ coded directory mirrors.
+
+ If no connections complete within 5 seconds, another batch of three
+ connections will be launched. Otherwise, the first connection to
+ complete will be used to download the consensus document and the others
+ will be closed, after which bootstrapping will proceed as normal.
+
+ If at any time, the total outstanding bootstrap connection attempts
+ exceeds 15, no new connection attempts are to be launched until existing
+ connection attempts experience full timeout.
+
+Design: Fallback Dir Mirror Selection
+
+ The set of hard coded directory mirrors from #572 shall be chosen using
+ the 100 Guard nodes with the longest uptime.
+
+ The fallback weights will be set using each mirror's fraction of
+ consensus bandwidth out of the total of all 100 mirrors.
+
+ This list of fallback dir mirrors should be updated with every
+ major Tor release. In future releases, the number of dir mirrors
+ should be set at 20% of the current Guard nodes, rather than fixed at
+ 100.
+
+Performance: Additional Load with Current Parameter Choices
+
+ This design and the connection count parameters were chosen such that
+ no additional bandwidth load would be placed on the directory
+ authorities. In fact, the directory authorities should experience less
+ load, because they will not need to serve the consensus document for a
+ connection in the event that one of the directory mirrors complete their
+ connection before the directory authority does.
+
+ However, the scheme does place additional TLS connection load on the
+ fallback dir mirrors. Because bootstrapping is rare and all but one of 
+ the TLS connections will be very short-lived and unused, this should not
+ be a substantial issue.
+
+ The dangerous case is in the event of a prolonged consensus failure
+ that induces all clients to enter into the bootstrap process. In this
+ case, the number of initial TLS connections to the fallback dir mirrors
+ would be 2*C/100, or 10,000 for C=500,000 users. If no connections
+ complete before the five retries, this could reach as high as 50,000
+ connection attempts, but this is extremely unlikely to happen in full
+ aggregate.
+
+ However, in the no-consensus scenario today, the directory authorities
+ would already experience C/9 or 55,555 connection attempts. The
+ 5-retry scheme increases their total maximum load to about 275,000
+ connection attempts, but again this is unlikely to be reached
+ in aggregate. Additionally, with this scheme, even if the dirauths
+ are taken down by this load, the dir mirrors should be able to survive
+ it.
+
+Implementation Notes: Code Modifications
+
+ The implementation of the bootstrap process is unfortunately mixed 
+ in with many types of directory activity.
+
+ The process starts in update_consensus_networkstatus_downloads(),
+ which initiates a single directory connection through
+ directory_get_from_dirserver(). Depending on bootstrap state,
+ a single directory server is selected and a connection is
+ eventually made through directory_initiate_command_rend().
+
+ There appear to be a few options for altering this code to perform
+ multiple connections. Without refactoring, one approach would be
+ to make multiple calls to directory_initiate_command_routerstatus()
+ from directory_get_from_dirserver() if the purpose is
+ DIR_PURPOSE_FETCH_CONSENSUS and the only directory servers available
+ are the authorities and the fallback dir mirrors.
+
+ The code in directory_initiate_command_rend() would then need to be
+ altered to maintain a list of the dircons created for this purpose as
+ well as avoid immediately queuing the directory_send_command() request
+ for the DIR_PURPOSE_FETCH_CONSENSUS purpose. A flag would need to be set
+ on the dircon to be checked in connection_dir_finished_connecting().
+
+ The function connection_dir_finished_connecting() would need to be
+ altered to examine the list of pending dircons, determine if this one is
+ the first to complete, and if so, then call directory_send_command() to
+ download the consensus and close the other pending dircons.
+
+ An additional timer would need to be installed to re-call
+ update_consensus_networkstatus_downloads() or a related helper after 5
+ seconds. connection_dir_finished_connecting() would cancel this timer.
+ The helper would check the list of pending connections and ensure it
+ never exceeds 15.
+
diff --git a/proposals/xxx-faster-headless-consensus-bootstrap.txt b/proposals/xxx-faster-headless-consensus-bootstrap.txt
deleted file mode 100644
index 8f5b079..0000000
--- a/proposals/xxx-faster-headless-consensus-bootstrap.txt
+++ /dev/null
@@ -1,112 +0,0 @@
-Title: Faster Headless Consensus Bootstrapping
-Author: Mike Perry
-Created: 01-10-2012
-Status: Open
-Target: 0.2.4.x+
-
-
-Overview and Motiviation
-
- This proposal describes a way for clients to fetch the initial
- consensus more quickly in situations where some or all of the directory
- authorities are unreachable. This proposal is meant to describe a
- solution for bug #4483.
-
-Design: Bootstrap Process Changes
-
- The core idea is to attempt to establish bootstrap connections in
- parallel during the bootstrap process, and download the consensus from
- the first connection that completes.
-
- Connection attempts will be done in batches of three. Only one
- connection will be performed to one of the canonical directory
- authorities. Two connections will be performed to randomly chosen hard
- coded directory mirrors.
-
- If no connections complete within 5 seconds, another batch of three
- connections will be launched. Otherwise, the first connection to
- complete will be used to download the consensus document and the others
- will be closed, after which bootstrapping will proceed as normal.
-
- If at any time, the total outstanding bootstrap connection attempts
- exceeds 15, no new connection attempts are to be launched until existing
- connection attempts experience full timeout.
-
-Design: Fallback Dir Mirror Selection
-
- The set of hard coded directory mirrors from #572 shall be chosen using
- the 100 Guard nodes with the longest uptime.
-
- The fallback weights will be set using each mirror's fraction of
- consensus bandwidth out of the total of all 100 mirrors.
-
- This list of fallback dir mirrors should be updated with every
- major Tor release. In future releases, the number of dir mirrors
- should be set at 20% of the current Guard nodes, rather than fixed at
- 100.
-
-Performance: Additional Load with Current Parameter Choices
-
- This design and the connection count parameters were chosen such that
- no additional bandwidth load would be placed on the directory
- authorities. In fact, the directory authorities should experience less
- load, because they will not need to serve the consensus document for a
- connection in the event that one of the directory mirrors complete their
- connection before the directory authority does.
-
- However, the scheme does place additional TLS connection load on the
- fallback dir mirrors. Because bootstrapping is rare and all but one of 
- the TLS connections will be very short-lived and unused, this should not
- be a substantial issue.
-
- The dangerous case is in the event of a prolonged consensus failure
- that induces all clients to enter into the bootstrap process. In this
- case, the number of initial TLS connections to the fallback dir mirrors
- would be 2*C/100, or 10,000 for C=500,000 users. If no connections
- complete before the five retries, this could reach as high as 50,000
- connection attempts, but this is extremely unlikely to happen in full
- aggregate.
-
- However, in the no-consensus scenario today, the directory authorities
- would already experience C/9 or 55,555 connection attempts. The
- 5-retry scheme increases their total maximum load to about 275,000
- connection attempts, but again this is unlikely to be reached
- in aggregate. Additionally, with this scheme, even if the dirauths
- are taken down by this load, the dir mirrors should be able to survive
- it.
-
-Implementation Notes: Code Modifications
-
- The implementation of the bootstrap process is unfortunately mixed 
- in with many types of directory activity.
-
- The process starts in update_consensus_networkstatus_downloads(),
- which initiates a single directory connection through
- directory_get_from_dirserver(). Depending on bootstrap state,
- a single directory server is selected and a connection is
- eventually made through directory_initiate_command_rend().
-
- There appear to be a few options for altering this code to perform
- multiple connections. Without refactoring, one approach would be
- to make multiple calls to directory_initiate_command_routerstatus()
- from directory_get_from_dirserver() if the purpose is
- DIR_PURPOSE_FETCH_CONSENSUS and the only directory servers available
- are the authorities and the fallback dir mirrors.
-
- The code in directory_initiate_command_rend() would then need to be
- altered to maintain a list of the dircons created for this purpose as
- well as avoid immediately queuing the directory_send_command() request
- for the DIR_PURPOSE_FETCH_CONSENSUS purpose. A flag would need to be set
- on the dircon to be checked in connection_dir_finished_connecting().
-
- The function connection_dir_finished_connecting() would need to be
- altered to examine the list of pending dircons, determine if this one is
- the first to complete, and if so, then call directory_send_command() to
- download the consensus and close the other pending dircons.
-
- An additional timer would need to be installed to re-call
- update_consensus_networkstatus_downloads() or a related helper after 5
- seconds. connection_dir_finished_connecting() would cancel this timer.
- The helper would check the list of pending connections and ensure it
- never exceeds 15.
-



