March 2011 - tor-commits - lists.torproject.org

[tor/master] whitespace fixes to pass 'make check-spaces'
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit c2775196f79b9dba70d517663004e1783873f35c Author: Roger Dingledine <arma(a)torproject.org> Date: Tue Mar 15 16:59:58 2011 -0400 whitespace fixes to pass 'make check-spaces' --- src/or/connection.c | 10 ++++++---- src/or/relay.c | 24 ++++++++++++------------ 2 files changed, 18 insertions(+), 16 deletions(-) diff --git a/src/or/connection.c b/src/or/connection.c index ff23646..b586b43 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -3264,7 +3264,7 @@ _connection_write_to_buf_impl(const char *string, size_t len, * conn->write_event won't be set yet. Otherwise, write data from * this conn as the socket is available. */ if (conn->write_event) { - connection_start_writing(conn); + connection_start_writing(conn); } if (zlib) { conn->outbuf_flushlen += buf_datalen(conn->outbuf) - old_datalen; @@ -3845,9 +3845,11 @@ assert_connection_ok(connection_t *conn, time_t now) * EXIT_CONN_STATE_RESOLVING while the conn is not yet marked to writing. * */ tor_assert((conn->type == CONN_TYPE_EXIT && - conn->state == EXIT_CONN_STATE_RESOLVING) || - connection_is_writing(conn) || conn->write_blocked_on_bw || - (CONN_IS_EDGE(conn) && TO_EDGE_CONN(conn)->edge_blocked_on_circ)); + conn->state == EXIT_CONN_STATE_RESOLVING) || + connection_is_writing(conn) || + conn->write_blocked_on_bw || + (CONN_IS_EDGE(conn) && + TO_EDGE_CONN(conn)->edge_blocked_on_circ)); } if (conn->hold_open_until_flushed) diff --git a/src/or/relay.c b/src/or/relay.c index 17ce7db..81a6497 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -1031,9 +1031,9 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, relay_header_t rh; unsigned domain = layer_hint?LD_APP:LD_EXIT; int reason; - int optimistic_data = 0; /* Set to 1 if we receive data on a stream - that's in the EXIT_CONN_STATE_RESOLVING - or EXIT_CONN_STATE_CONNECTING states.*/ + int optimistic_data = 0; /* Set to 1 if we receive data on a stream + * that's in the EXIT_CONN_STATE_RESOLVING + * or EXIT_CONN_STATE_CONNECTING states. */ tor_assert(cell); tor_assert(circ); @@ -1055,16 +1055,16 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, if (conn && !connection_state_is_open(TO_CONN(conn))) { if (conn->_base.type == CONN_TYPE_EXIT && - (conn->_base.state == EXIT_CONN_STATE_CONNECTING || - conn->_base.state == EXIT_CONN_STATE_RESOLVING) && - rh.command == RELAY_COMMAND_DATA) { - /* We're going to allow DATA cells to be delivered to an exit - * node in state EXIT_CONN_STATE_CONNECTING or - * EXIT_CONN_STATE_RESOLVING. This speeds up HTTP, for example. */ - optimistic_data = 1; + (conn->_base.state == EXIT_CONN_STATE_CONNECTING || + conn->_base.state == EXIT_CONN_STATE_RESOLVING) && + rh.command == RELAY_COMMAND_DATA) { + /* Allow DATA cells to be delivered to an exit node in state + * EXIT_CONN_STATE_CONNECTING or EXIT_CONN_STATE_RESOLVING. + * This speeds up HTTP, for example. */ + optimistic_data = 1; } else { - return connection_edge_process_relay_cell_not_open( - &rh, cell, circ, conn, layer_hint); + return connection_edge_process_relay_cell_not_open( + &rh, cell, circ, conn, layer_hint); } }

1 0

[tor/master] Merge remote branch 'arma/optimistic_server'
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit 415caba967b9b470623ab29230b820b3470a91e6 Merge: 737228e c277519 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 17:20:41 2011 -0400 Merge remote branch 'arma/optimistic_server' changes/prop174_server | 6 ++++++ src/or/connection.c | 19 ++++++++++++++++--- src/or/relay.c | 29 +++++++++++++++++++++++++---- 3 files changed, 47 insertions(+), 7 deletions(-)

1 0

[tor/master] Implement server side of Proposal 174 (optimistic data)
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit bd7721f66ef94d2f3502197bf23a4cf67813eb9f Author: Ian Goldberg <iang(a)cs.uwaterloo.ca> Date: Mon Mar 14 18:05:33 2011 -0400 Implement server side of Proposal 174 (optimistic data) Ian's original message: The current code actually correctly handles queued data at the Exit; if there is queued data in a EXIT_CONN_STATE_CONNECTING stream, that data will be immediately sent when the connection succeeds. If the connection fails, the data will be correctly ignored and freed. The problem with the current server code is that the server currently drops DATA cells on streams in the EXIT_CONN_STATE_CONNECTING state. Also, if you try to queue data in the EXIT_CONN_STATE_RESOLVING state, bad things happen because streams in that state don't yet have conn->write_event set, and so some existing sanity checks (any stream with queued data is at least potentially writable) are no longer sound. The solution is to simply not drop received DATA cells while in the EXIT_CONN_STATE_CONNECTING state. Also do not send SENDME cells in this state, so that the OP cannot send more than one window's worth of data to be queued at the Exit. Finally, patch the sanity checks so that streams in the EXIT_CONN_STATE_RESOLVING state that have buffered data can pass. [...] Here is a simple patch. It seems to work with both regular streams and hidden services, but there may be other corner cases I'm not aware of. (Do streams used for directory fetches, hidden services, etc. take a different code path?) --- src/or/connection.c | 15 +++++++++++++-- src/or/relay.c | 28 +++++++++++++++++++++++----- 2 files changed, 36 insertions(+), 7 deletions(-) diff --git a/src/or/connection.c b/src/or/connection.c index 874e724..ff23646 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -3259,7 +3259,13 @@ _connection_write_to_buf_impl(const char *string, size_t len, return; } - connection_start_writing(conn); + /* If we receive optimistic data in the EXIT_CONN_STATE_RESOLVING + * state, we don't want to try to write it right away, since + * conn->write_event won't be set yet. Otherwise, write data from + * this conn as the socket is available. */ + if (conn->write_event) { + connection_start_writing(conn); + } if (zlib) { conn->outbuf_flushlen += buf_datalen(conn->outbuf) - old_datalen; } else { @@ -3835,7 +3841,12 @@ assert_connection_ok(connection_t *conn, time_t now) tor_assert(conn->s < 0); if (conn->outbuf_flushlen > 0) { - tor_assert(connection_is_writing(conn) || conn->write_blocked_on_bw || + /* With optimistic data, we may have queued data in + * EXIT_CONN_STATE_RESOLVING while the conn is not yet marked to writing. + * */ + tor_assert((conn->type == CONN_TYPE_EXIT && + conn->state == EXIT_CONN_STATE_RESOLVING) || + connection_is_writing(conn) || conn->write_blocked_on_bw || (CONN_IS_EDGE(conn) && TO_EDGE_CONN(conn)->edge_blocked_on_circ)); } diff --git a/src/or/relay.c b/src/or/relay.c index ec8e9ac..19f8d41 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -1031,6 +1031,9 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, relay_header_t rh; unsigned domain = layer_hint?LD_APP:LD_EXIT; int reason; + int optimistic_data = 0; /* Set to 1 if we receive data on a stream + that's in the EXIT_CONN_STATE_RESOLVING + or EXIT_CONN_STATE_CONNECTING states.*/ tor_assert(cell); tor_assert(circ); @@ -1050,9 +1053,20 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, /* either conn is NULL, in which case we've got a control cell, or else * conn points to the recognized stream. */ - if (conn && !connection_state_is_open(TO_CONN(conn))) - return connection_edge_process_relay_cell_not_open( - &rh, cell, circ, conn, layer_hint); + if (conn && !connection_state_is_open(TO_CONN(conn))) { + if (conn->_base.type == CONN_TYPE_EXIT && + (conn->_base.state == EXIT_CONN_STATE_CONNECTING || + conn->_base.state == EXIT_CONN_STATE_RESOLVING) && + rh.command == RELAY_COMMAND_DATA) { + /* We're going to allow DATA cells to be delivered to an exit + * node in state EXIT_CONN_STATE_CONNECTING or + * EXIT_CONN_STATE_RESOLVING. This speeds up HTTP, for example. */ + optimistic_data = 1; + } else { + return connection_edge_process_relay_cell_not_open( + &rh, cell, circ, conn, layer_hint); + } + } switch (rh.command) { case RELAY_COMMAND_DROP: @@ -1102,7 +1116,9 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, log_debug(domain,"circ deliver_window now %d.", layer_hint ? layer_hint->deliver_window : circ->deliver_window); - circuit_consider_sending_sendme(circ, layer_hint); + if (!optimistic_data) { + circuit_consider_sending_sendme(circ, layer_hint); + } if (!conn) { log_info(domain,"data cell dropped, unknown stream (streamid %d).", @@ -1119,7 +1135,9 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, stats_n_data_bytes_received += rh.length; connection_write_to_buf((char*)(cell->payload + RELAY_HEADER_SIZE), rh.length, TO_CONN(conn)); - connection_edge_consider_sending_sendme(conn); + if (!optimistic_data) { + connection_edge_consider_sending_sendme(conn); + } return 0; case RELAY_COMMAND_END: reason = rh.length > 0 ?

1 0

[tor/master] Revise prop174 patch: make the circuit sendme calls unconditional.
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit fe9de9231acdd50dcaac5c7908ea9d29f2b21b48 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Mar 14 18:08:27 2011 -0400 Revise prop174 patch: make the circuit sendme calls unconditional. The circuit_consider_sending_sendme call should be unconditional, since the circuit windows can move independent of the state of streams. --- src/or/relay.c | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/src/or/relay.c b/src/or/relay.c index 19f8d41..17ce7db 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -1116,9 +1116,7 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, log_debug(domain,"circ deliver_window now %d.", layer_hint ? layer_hint->deliver_window : circ->deliver_window); - if (!optimistic_data) { - circuit_consider_sending_sendme(circ, layer_hint); - } + circuit_consider_sending_sendme(circ, layer_hint); if (!conn) { log_info(domain,"data cell dropped, unknown stream (streamid %d).", @@ -1135,9 +1133,14 @@ connection_edge_process_relay_cell(cell_t *cell, circuit_t *circ, stats_n_data_bytes_received += rh.length; connection_write_to_buf((char*)(cell->payload + RELAY_HEADER_SIZE), rh.length, TO_CONN(conn)); + if (!optimistic_data) { - connection_edge_consider_sending_sendme(conn); + /* Only send a SENDME if we're not getting optimistic data; otherwise + * a SENDME could arrive before the CONNECTED. + */ + connection_edge_consider_sending_sendme(conn); } + return 0; case RELAY_COMMAND_END: reason = rh.length > 0 ?

1 0

[tor/master] Merge remote branch 'origin/maint-0.2.2'
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit 737228ed8e81de6619a29bd039d08d959d4164da Merge: 4dde69a c5ffd44 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 17:18:29 2011 -0400 Merge remote branch 'origin/maint-0.2.2' changes/bug2756 | 11 +++++++++++ src/or/connection.c | 2 ++ src/or/connection_edge.c | 17 +++++++++++++++++ src/or/connection_edge.h | 1 + src/or/relay.c | 5 +++-- 5 files changed, 34 insertions(+), 2 deletions(-)

1 0

[tor/master] Merge branch 'bug2756_relay' into maint-0.2.2
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit c5ffd44ccd3c60780143eb384938276522a173c1 Merge: ab41844 a86e27c Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 17:13:07 2011 -0400 Merge branch 'bug2756_relay' into maint-0.2.2 changes/bug2756 | 11 +++++++++++ src/or/connection.c | 2 ++ src/or/connection_edge.c | 17 +++++++++++++++++ src/or/connection_edge.h | 1 + src/or/relay.c | 5 +++-- 5 files changed, 34 insertions(+), 2 deletions(-)

1 0

[tor/master] Consider sending stream-level SENDME cells on partial flushes.
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit 1d36a8e9ae766f67097f4c5e522c33ff0c4a693f Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Mar 14 17:48:45 2011 -0400 Consider sending stream-level SENDME cells on partial flushes. Right now, we only consider sending stream-level SENDME cells when we have completely flushed a connection_edge's outbuf, or when it sends us a DATA cell. Neither of these is ideal for throughput. This patch changes the behavior so we now call connection_edge_consider_sending_sendme when we flush _some_ data from an edge outbuf. Fix for bug 2756; bugfix on svn r152. --- changes/bug2756 | 11 +++++++++++ src/or/connection.c | 2 ++ src/or/connection_edge.c | 17 +++++++++++++++++ src/or/connection_edge.h | 1 + 4 files changed, 31 insertions(+), 0 deletions(-) diff --git a/changes/bug2756 b/changes/bug2756 new file mode 100644 index 0000000..0cad515 --- /dev/null +++ b/changes/bug2756 @@ -0,0 +1,11 @@ + o Minor bugfixes (spec conformance, performance): + - We now ask the other side of a stream (the client or the exit) + for more data on that stream when the amount of queued data on + that stream dips low enough. Previously, we wouldn't ask the + other side for more data until either it sent us more data + (which it wasn't supposed to do if it had exhausted its + window!) or until we had completely flushed all our queued + data. Fixing this should improve throughput. Fixes bug 2756; + bugfix on the earliest released versions of Tor (svn commit + r152). + diff --git a/src/or/connection.c b/src/or/connection.c index fd30ac8..86ed45e 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -3245,6 +3245,8 @@ connection_flushed_some(connection_t *conn) r = connection_dirserv_flushed_some(TO_DIR_CONN(conn)); } else if (conn->type == CONN_TYPE_OR) { r = connection_or_flushed_some(TO_OR_CONN(conn)); + } else if (CONN_IS_EDGE(conn)) { + r = connection_edge_flushed_some(TO_EDGE_CONN(conn)); } conn->in_flushed_some = 0; return r; diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index f02479f..85a5225 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -301,6 +301,23 @@ connection_edge_end_errno(edge_connection_t *conn) return connection_edge_end(conn, reason); } +/** We just wrote some data to conn; act appropriately. + * + * (That is, if it's open, consider sending a stream-level sendme cell if we + * have just flushed enough.) + */ +int +connection_edge_flushed_some(edge_connection_t *conn) +{ + switch (conn->_base.state) { + case AP_CONN_STATE_OPEN: + case EXIT_CONN_STATE_OPEN: + connection_edge_consider_sending_sendme(conn); + break; + } + return 0; +} + /** Connection conn has finished writing and has no bytes left on * its outbuf. * diff --git a/src/or/connection_edge.h b/src/or/connection_edge.h index fa5f91c..0b08dd0 100644 --- a/src/or/connection_edge.h +++ b/src/or/connection_edge.h @@ -23,6 +23,7 @@ int connection_edge_process_inbuf(edge_connection_t *conn, int connection_edge_destroy(circid_t circ_id, edge_connection_t *conn); int connection_edge_end(edge_connection_t *conn, uint8_t reason); int connection_edge_end_errno(edge_connection_t *conn); +int connection_edge_flushed_some(edge_connection_t *conn); int connection_edge_finished_flushing(edge_connection_t *conn); int connection_edge_finished_connecting(edge_connection_t *conn);

1 0

[tor/master] Fix comment for connection_edge_consider_sending_sendme
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit a86e27cf00153f4d926ac2b0974d2cceb543d673 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 15:35:11 2011 -0400 Fix comment for connection_edge_consider_sending_sendme --- src/or/relay.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/or/relay.c b/src/or/relay.c index a6c2506..2510242 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -1409,8 +1409,9 @@ connection_edge_package_raw_inbuf(edge_connection_t *conn, int package_partial, goto repeat_connection_edge_package_raw_inbuf; } -/** Called when we've just received a relay data cell, or when - * we've just finished flushing all bytes to stream conn. +/** Called when we've just received a relay data cell, when + * we've just finished flushing all bytes to stream conn, + * or when we've flushed *some* bytes to the stream conn. * * If conn->outbuf is not too full, and our deliver window is * low, send back a suitable number of stream-level sendme cells.

1 0

[torspec/master] Give a proposal number to pluggable transport.
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit 57c77a2110616a6a5d0bbdee260936c1c13cad9a Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 17:15:37 2011 -0400 Give a proposal number to pluggable transport. --- proposals/000-index.txt | 2 + proposals/180-pluggable-transport.txt | 530 +++++++++++++++++++++++++++ proposals/ideas/xxx-pluggable-transport.txt | 530 --------------------------- 3 files changed, 532 insertions(+), 530 deletions(-) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index be9b4e2..f55afc7 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -100,6 +100,7 @@ Proposals by number: 177 Abstaining from votes on individual flags [OPEN] 178 Require majority of authorities to vote for consensus parameters [OPEN] 179 TLS certificate and parameter normalization [DRAFT] +180 Pluggable transports for circumvention [OPEN] Proposals by status: @@ -132,6 +133,7 @@ Proposals by status: 176 Proposed version-3 link handshake for Tor [for 0.2.3] 177 Abstaining from votes on individual flags 178 Require majority of authorities to vote for consensus parameters + 180 Pluggable transports for circumvention ACCEPTED: 110 Avoiding infinite length circuits [for 0.2.1.x] [in 0.2.1.3-alpha] 117 IPv6 exits [for 0.2.1.x] diff --git a/proposals/180-pluggable-transport.txt b/proposals/180-pluggable-transport.txt new file mode 100644 index 0000000..2df0ced --- /dev/null +++ b/proposals/180-pluggable-transport.txt @@ -0,0 +1,530 @@ +Filename: 180-pluggable-transport.txt +Title: Pluggable transports for circumvention +Author: Jacob Appelbaum, Nick Mathewson +Created: 15-Oct-2010 +Status: Open + +Overview + + This proposal describes a way to decouple protocol-level obfuscation + from the core Tor protocol in order to better resist client-bridge + censorship. Our approach is to specify a means to add pluggable + transport implementations to Tor clients and bridges so that they can + negotiate a superencipherment for the Tor protocol. + +Scope + + This is a document about transport plugins; it does not cover + discovery improvements, or bridgedb improvements. While these + requirements might be solved by a program that also functions as a + transport plugin, this proposal only covers the requirements and + operation of transport plugins. + +Motivation + + Frequently, people want to try a novel circumvention method to help + users connect to Tor bridges. Some of these methods are already + pretty easy to deploy: if the user knows an unblocked VPN or open + SOCKS proxy, they can just use that with the Tor client today. + + Less easy to deploy are methods that require participation by both the + client and the bridge. In order of increasing sophistication, we + might want to support: + + 1. A protocol obfuscation tool that transforms the output of a TLS + connection into something that looks like HTTP as it leaves the + client, and back to TLS as it arrives at the bridge. + 2. An additional authentication step that a client would need to + perform for a given bridge before being allowed to connect. + 3. An information passing system that uses a side-channel in some + existing protocol to convey traffic between a client and a bridge + without the two of them ever communicating directly. + 4. A set of clients to tunnel client->bridge traffic over an existing + large p2p network, such that the bridge is known by an identifier + in that network rather than by an IP address. + + We could in theory support these almost fine with Tor as it stands + today: every Tor client can take a SOCKS proxy to use for its outgoing + traffic, so a suitable client proxy could handle the client's traffic + and connections on its behalf, while a corresponding program on the + bridge side could handle the bridge's side of the protocol + transformation. Nevertheless, there are some reasons to add support + for transportation plugins to Tor itself: + + 1. It would be good for bridges to have a standard way to advertise + which transports they support, so that clients can have multiple + local transport proxies, and automatically use the right one for + the right bridge. + + 2. There are some changes to our architecture that we'll need for a + system like this to work. For testing purposes, if a bridge blocks + off its regular ORPort and instead has an obfuscated ORPort, the + bridge authority has no way to test it. Also, unless the bridge + has some way to tell that the bridge-side proxy at 127.0.0.1 is not + the origin of all the connections it is relaying, it might decide + that there are too many connections from 127.0.0.1, and start + paring them down to avoid a DoS. + + 3. Censorship and anticensorship techniques often evolve faster than + the typical Tor release cycle. As such, it's a good idea to + provide ways to test out new anticensorship mechanisms on a more + rapid basis. + + 4. Transport obfuscation is a relatively distinct problem + from the other privacy problems that Tor tries to solve, and it + requires a fairly distinct skill-set from hacking the rest of Tor. + By decoupling transport obfuscation from the Tor core, we hope to + encourage people working on transport obfuscation who would + otherwise not be interested in hacking Tor. + + 5. Finally, we hope that defining a generic transport obfuscation plugin + mechanism will be useful to other anticensorship projects. + +Non-Goals + + We're not going to talk about automatic verification of plugin + correctness and safety via sandboxing, proof-carrying code, or + whatever. + + We need to do more with discovery and distribution, but that's not + what this proposal is about. We're pretty convinced that the problems + are sufficiently orthogonal that we should be fine so long as we don't + preclude a single program from implementing both transport and + discovery extensions. + + This proposal is not about what transport plugins are the best ones + for people to write. We do, however, make some general + recommendations for plugin authors in an appendix. + + We've considered issues involved with completely replacing Tor's TLS + with another encryption layer, rather than layering it inside the + obfuscation layer. We describe how to do this in an appendix to the + current proposal, though we are not currently sure whether it's a good + idea to implement. + + We deliberately reject any design that would involve linking more code + into Tor's process space. + +Design overview + + To write a new transport protocol, an implementer must provide two + pieces: a "Client Proxy" to run at the initiator side, and a "Server + Proxy" to run at the server side. These two pieces may or may not be + implemented by the same program. + + Each client may run any number of Client Proxies. Each one acts like + a SOCKS proxy that accepts connections on localhost. Each one + runs on a different port, and implements one or more transport + methods. If the protocol has any parameters, they are passed from Tor + inside the regular username/password parts of the SOCKS protocol. + + Bridges (and maybe relays) may run any number of Server Proxies: these + programs provide an interface like stunnel: they get connections from the + network (typically by listening for connections on the network) and relay + them to the Bridge's real ORPort. + + To configure one of these programs, it should be sufficient simply to + list it in your torrc. The program tells Tor which transports it + provides. The Tor consensus should carry a new approved version number that + is specific for pluggable transport; this will allow Tor to know when a + particular transport is known to be unsafe safe or non-functional. + + Bridges (and maybe relays) report in their descriptors which transport + protocols they support. This information can be copied into bridge + lines. Bridges using a transport protocol may have multiple bridge + lines. + + Any methods that are wildly successful, we can bake into Tor. + +Specifications: Client behavior + + We extend the bridge line format to allow you to say which method + to use to connect to a bridge. + + The new format is: + "bridge method address:port [[keyid=]id-fingerprint] [k=v] [k=v] [k=v]" + + To connect to such a bridge, the Tor program needs to know which + local SOCKS proxy will support the transport called "method". It + then connects to this proxy, and asks it to connect to + address:port. If [id-fingerprint] is provided, Tor should expect + the public identity key on the TLS connection to match the digest + provided in [id-fingerprint]. If any [k=v] items are provided, + they are configuration parameters for the proxy: Tor should + separate them with semicolons and put them in the user and + password fields of the request, splitting them across the fields + as necessary. If a key or value value must contain a semicolon or + a backslash, it is escaped with a backslash. + + The "id-fingerprint" field is always provided in a field named + "keyid", if it was given. Method names must be C identifiers. + + Example: if the bridge line is "bridge trebuchet www.example.com:3333 + rocks=20 height=5.6m" AND if the Tor client knows that the + 'trebuchet' method is provided by a SOCKS5 proxy on + 127.0.0.1:19999, the client should connect to that proxy, ask it to + connect to www.example.com, and provide the string + "rocks=20;height=5.6m" as the username, the password, or split + across the username and password. + + There are two ways to tell Tor clients about protocol proxies: + external proxies and managed proxies. An external proxy is configured + with + ClientTransportPlugin method socks4 address:port [auth=X] + or + ClientTransportPlugin method socks5 address:port [username=X] [password=Y] + as in + "ClientTransportPlugin trebuchet socks5 127.0.0.1:9999". + This example tells Tor that another program is already running to handle + 'trubuchet' connections, and Tor doesn't need to worry about it. + + A managed proxy is configured with + ClientTransportPlugin <method> exec <path> [options] + as in + "ClientTransportPlugin trebuchet exec /usr/libexec/trebuchet --managed" + This example tells Tor to launch an external program to provide a + socks proxy for 'trebuchet' connections. The Tor client only + launches one instance of each external program with a given set of + options, even if the same executable and options are listed for + more than one method. + + If instead of a transport name, the torrc lists "*" for a managed proxy, + tor uses that proxy for all transports that it supports. So + "ClientTransportPlugin * exec /usr/libexec/tor/foobar" tells Tor + that it should use the foobar plugin for everything that it supports. + + If two proxies support the same method, Tor should use whichever + one is listed first. + + The same program can implement a managed or an external proxy: it just + needs to take an argument saying which one to be. + + See "Managed proxy behavior" for more information on the managed + proxy interface. + +Server behavior + + Server proxies are configured similarly to client proxies. When + launching a proxy, the server must tell it what ORPort it has + configured, and what address (if any) it can listen on. The + server must tell the proxy which (if any) methods it should + provide if it can; the proxy needs to tell the server which + methods it is actually providing, and on what ports. + + When a client connects to the proxy, the proxy may need a way to + tell the server some identifier for the client address. It does + this in-band. + + As before, the server lists proxies in its torrc. These can be + external proxies that run on their own, or managedproxies that Tor + launches. + + An external server proxy is configured as + ServerTransportPlugin method proxy address:port param=val.. + as in + ServerTransportPlugin trebuchet proxy 127.0.0.1:999 rocks=heavy + The param=val pairs and the address are used to make the bridge + configuration information that we'll tell users. + + A managed proxy is configured as + ServerTransportPlugin method exec /path/to/binary [options] + or + ServerTransportPlugin * exec /path/to/binary [options] + + When possible, Tor should launch only one binary of each binary/option + pair configured. So if the torrc contains + + ClientTransportPlugin foo exec /usr/bin/megaproxy --foo + ClientTransportPlugin bar exec /usr/bin/megaproxy --bar + ServerTransportPlugin * exec /usr/bin/megaproxy --foo + + then Tor will launch the megaproxy binary twice: once with the option + --foo and once with the option --bar. + +Managed proxy interface + + When the Tor client launches a client proxy from the command + line, it communicates via environment variables. At a minimum, + it sets: + + {Client and server} + HOME, PATH -- as you'd expect. + + "STATE_LOCATION" -- a directory where the proxy should store + state if it wants to. This directory is not required to + exist, but the proxy SHOULD be able to create it if it + doesn't. The proxy SHOULD NOT store state elsewhere. + + "MANAGED_TRANSPORT_VER=1" -- To tell the proxy which versions + of this configuration protocol Tor supports. Future versions + will give a comma-separated list. Clients MUST accept + comma-separated lists containing any version that they + recognize, and MUST work correctly even if some of the + versions they don't recognize are non-numeric. + + {Client only} + + "CLIENT_TRANSPORTS" -- a comma-separated list of which methods + this client should enable, or * if all methods should be + enabled. The proxy SHOULD ignore methods that it doesn't + recognize. + + {Server only} + + "EXT_SERVER_PORT=addr:portnum" -- A port (probably on localhost) that + speaks the extended server protocol. + + "ORPORT=addr:portnum" -- Our regular ORPort in a form suitable + for local connections. + + "BINDADDR=addr" -- An address on which to listen for local + connections. This might be the advertised address, or might + be a local address that Tor will forward ports to. It MUST + be an address that will work with bind(). + + "SERVER_TRANSPORTS=..." -- A comma-separated list of server + methods that the proxy should support, or * + + The transport proxy replies by writing NL-terminated lines to + stdout. The metaformat is + + Keyword OptArgs NL + OptArgs = Args | + Args = SP ArgChar | Args ArgChar + ArgChar = Any character but NUL or NL + Keyword = KeywordChar | Keyword KeywordChar + KeyWordChar = All alphanumeric characters, dash, and underscore. + + Tor MUST ignore lines with keywords that it doesn't recognize. + + First, the proxy writes "VERSION 1" to say that it supports this + protocol. It must either pick a version that Tor told it about, or + pick no version at all, and say "ERROR no-version\n" and exit. + + The proxy should then open its ports. If running as a client + proxy, it should not use fixed ports; instead it should autoselect + ports to avoid conflicts. A client proxy should by default only + listen on localhost for connections. + + A server proxy SHOULD try listen at a consistent port, though it + SHOULD pick a different one if the port it last used is now allocated. + + A client or server proxy then should tell which methods it has + made available and how. It does this by printing zero or more + CMETHOD and SMETHOD lines to its stdout. These lines look like: + + CMETHOD methodname SOCKS4/SOCKS5 address:port [ARGS=arglist] \ + [OPT-ARGS=arglist] + + as in + + CMETHOD trebuchet SOCKS5 127.0.0.1:19999 ARGS=rocks,height \ + OPT-ARGS=tensile-strength + + The ARGS field lists mandatory parameters that must appear in + every bridge line for this method. The OPT-ARGS field lists + optional parameters. If no ARGS or OPT-ARGS field is provided, + Tor should not check the parameters in bridge lines for this + method. + + The proxy should print a single "CMETHODS DONE" line after it is + finished telling Tor about the client methods it provides. If it + tries to supply a client method but can't for some reason, it + should say: + CMETHOD-ERROR methodname "Message" + + A proxy should tell Tor about the server methods it is providing + by printing zero or more SMETHOD lines. These lines look like: + + SMETHOD methodname address:port [Options] + + If there's an error setting up a configured server method, the + proxy should say: + SMETHOD-ERROR methodname "message" + + The 'address:port' part of an SMETHOD line is the address to put + in the bridge line. The ARGS: part is a list of key-value pairs + that the client needs to know. The Options part is a list of + space-separated K:V flags that Tor should know about. Recognized + options are: + + - FORWARD:1 + + If this option is set, and address:port is not a publicly + accessible address, then the bridge needs to forward some + other address:port to address:port via upnp-helper. + + - ARGS:k=v,k=v,k=v + + If this option is set, the K=V arguments are added to the + extrainfo document. + + - DECLARE:K=V,... + + If this option is set, all the K=V options should be + added as extension entries to the router descriptor. (See + below) + + - USE-EXTPORT:1 + + If this option is set, the server plugin is using the + extended server port. + + SMETHOD and CMETHOD lines may be interspersed. After the list + SMETHOD line, the proxy says "SMETHODS DONE" + + The proxy SHOULD NOT tell Tor about a server or client method + unless it is actually open and ready to use. + + Tor clients SHOULD NOT use any method from a client proxy or + advertise any method from a server proxy UNLESS it is listed as a + possible method for that proxy in torrc, and it is listed by the + proxy as a method it supports. + + Proxies should respond to a single INT signal by closing their + listener ports and not accepting any new connections, but keeping + all connections open, then terminating when connections are all + closed. Proxies should respond to a second INT signal by shutting + down cleanly. + +The extended ORPort protocol. + + Server transports may need to connect to the bridge and pass + additional information about client connections that the bridge + would ordinarily receive from the kernel's TCP stack. To to this, + they connect to the "extended server port" as given in + SERVER_PORT, sent a short amount of information, wait for a + response, and then send the user traffic on that port. + + The extended server port protocol is as follows: + + COMMAND [2 bytes, big-endian] + BODYLEN [2 bytes, big-endian] + BODY [Bodylen bytes] + + Commands sent from the transport to the server are: + + [0x0000] DONE: There is no more information to give. (body ignored) + + [0x0001] USERADDR: an address:port string that represents the user's + address. If the transport doesn't actually do addresses, + this shouldn't be sent. + + Replies sent from tor to the proxy are: + + [0x1001] OKAY: Send the user's traffic. (body ignored) + + [0x1002] DENY: Tor would prefer not to get more traffic from + this address for a while. (body ignored) + + [We could also use an out-of-band signalling method to tell Tor + about client addresses, but that's a historically error-prone way + to go about annotating connections.] + +Advertising bridge methods: + + Bridges put the 'method' lines in their extra-info documents. + + method SP methodname SP address:port SP arglist NL + + The address:port parse are as returned from an SMETHOD line. The + arglist is a K=V,... list as retuned in the ARGS part of the + SMETHOD line. + + If the SMETHOD line includes a DECLARE: part, the routerinfo gets + a new line: + + method-info SP methodname SP arglist NL + +Bridge authority behavior + + We need to specify a way to test different transport methods that + bridges claim to support. We should test as many as possible. We + should NOT require that we have a way to tra + +Bridgedb behavior: + + Bridgedb can, given a set of router descriptors and their + corresponding extrainfo documents, generate a set of bridge lines + for each descriptor. Bridgedb may want to avoid handing out + methods that seem to get bridges blocked quickly. + +Implementation plan + + First, we should implement per-bridge socks settings (as + described above in "manually configuring a client proxy for a + bridge") and the extended-server-port mechanism. This will let + bridges run transport proxies such that they can hand-generate + bridge lines to give to clients for testing. + + Once that's done, we can improve usability a little bit by + implementing external proxies. Once that's done, we can see if we + need any managed proxies, or if the whole idea there is silly. + + If we do, the next most important part seems to be getting + the client-side automatic part written. And once that's done, we + can evaluate how much of the server side is easy for people to do + and how much is hard. + + The "obfsproxy" obfuscating proxy is a likely candidate for an + initial transport, as is Steven Murdoch's http thing or something + similar. + +Notes on plugins to write: + + We should ship a couple of null plugin implementations in one or two + popular, portable languages so that people get an idea of how to + write the stuff. + + 1. We should have one that's just a proof of concept that does + nothing but transfer bytes back and forth. + + 1. We should not do a rot13 one. + + 2. We should implement a basic proxy that does not transform the bytes at all + + 1. We should implement DNS or HTTP using other software (as goodesll + did years ago with DNS) as an example of wrapping existing code into + our plugin model. + + 2. The obfuscated-ssh superencipherment is pretty trivial and pretty + useful. It makes the protocol stringwise unfingerprintable. + + 1. Nick needs to be told firmly not to bikeshed the obfuscated-ssh + superencipherment too badly + + 1. Go ahead, bikeshed my day + + 1. If we do a raw-traffic proxy, openssh tunnels would be the logical choice. + +Appendix: recommendations for transports + + Be free/open-source software. Also, if you think your code might + someday do so well at circumvention that it should be implemented + inside Tor, it should use the same license as Tor. + + Use libraries that Tor already requires. (You can rely on openssl and + libevent being present if current Tor is present.) + + Be portable: most Tor users are on Windows, and most Tor developers + are not, so designing your code for just one of these platforms will + make it either get a small userbase, or poor auditing. + + Think secure: if your code is in a C-like language, and it's hard to + read it and become convinced it's safe, then it's probably not safe. + + Think small: we want to minimize the bytes that a Windows user needs + to download for a transport client. + + Avoid security-through-obscurity if possible. Specify. + + Resist trivial fingerprinting: There should be no good string or regex + to search for to distinguish your protocol from protocols permitted by + censors. + + Imitate a real profile: There are many ways to implement most + protocols -- and in many cases, most possible variants of a given + protocol won't actually exist in the wild. + + + diff --git a/proposals/ideas/xxx-pluggable-transport.txt b/proposals/ideas/xxx-pluggable-transport.txt deleted file mode 100644 index 201f0ac..0000000 --- a/proposals/ideas/xxx-pluggable-transport.txt +++ /dev/null @@ -1,530 +0,0 @@ -Filename: xxx-pluggable-transport.txt -Title: Pluggable transports for circumvention -Author: Jacob Appelbaum, Nick Mathewson -Created: 15-Oct-2010 -Status: Draft - -Overview - - This proposal describes a way to decouple protocol-level obfuscation - from the core Tor protocol in order to better resist client-bridge - censorship. Our approach is to specify a means to add pluggable - transport implementations to Tor clients and bridges so that they can - negotiate a superencipherment for the Tor protocol. - -Scope - - This is a document about transport plugins; it does not cover - discovery improvements, or bridgedb improvements. While these - requirements might be solved by a program that also functions as a - transport plugin, this proposal only covers the requirements and - operation of transport plugins. - -Motivation - - Frequently, people want to try a novel circumvention method to help - users connect to Tor bridges. Some of these methods are already - pretty easy to deploy: if the user knows an unblocked VPN or open - SOCKS proxy, they can just use that with the Tor client today. - - Less easy to deploy are methods that require participation by both the - client and the bridge. In order of increasing sophistication, we - might want to support: - - 1. A protocol obfuscation tool that transforms the output of a TLS - connection into something that looks like HTTP as it leaves the - client, and back to TLS as it arrives at the bridge. - 2. An additional authentication step that a client would need to - perform for a given bridge before being allowed to connect. - 3. An information passing system that uses a side-channel in some - existing protocol to convey traffic between a client and a bridge - without the two of them ever communicating directly. - 4. A set of clients to tunnel client->bridge traffic over an existing - large p2p network, such that the bridge is known by an identifier - in that network rather than by an IP address. - - We could in theory support these almost fine with Tor as it stands - today: every Tor client can take a SOCKS proxy to use for its outgoing - traffic, so a suitable client proxy could handle the client's traffic - and connections on its behalf, while a corresponding program on the - bridge side could handle the bridge's side of the protocol - transformation. Nevertheless, there are some reasons to add support - for transportation plugins to Tor itself: - - 1. It would be good for bridges to have a standard way to advertise - which transports they support, so that clients can have multiple - local transport proxies, and automatically use the right one for - the right bridge. - - 2. There are some changes to our architecture that we'll need for a - system like this to work. For testing purposes, if a bridge blocks - off its regular ORPort and instead has an obfuscated ORPort, the - bridge authority has no way to test it. Also, unless the bridge - has some way to tell that the bridge-side proxy at 127.0.0.1 is not - the origin of all the connections it is relaying, it might decide - that there are too many connections from 127.0.0.1, and start - paring them down to avoid a DoS. - - 3. Censorship and anticensorship techniques often evolve faster than - the typical Tor release cycle. As such, it's a good idea to - provide ways to test out new anticensorship mechanisms on a more - rapid basis. - - 4. Transport obfuscation is a relatively distinct problem - from the other privacy problems that Tor tries to solve, and it - requires a fairly distinct skill-set from hacking the rest of Tor. - By decoupling transport obfuscation from the Tor core, we hope to - encourage people working on transport obfuscation who would - otherwise not be interested in hacking Tor. - - 5. Finally, we hope that defining a generic transport obfuscation plugin - mechanism will be useful to other anticensorship projects. - -Non-Goals - - We're not going to talk about automatic verification of plugin - correctness and safety via sandboxing, proof-carrying code, or - whatever. - - We need to do more with discovery and distribution, but that's not - what this proposal is about. We're pretty convinced that the problems - are sufficiently orthogonal that we should be fine so long as we don't - preclude a single program from implementing both transport and - discovery extensions. - - This proposal is not about what transport plugins are the best ones - for people to write. We do, however, make some general - recommendations for plugin authors in an appendix. - - We've considered issues involved with completely replacing Tor's TLS - with another encryption layer, rather than layering it inside the - obfuscation layer. We describe how to do this in an appendix to the - current proposal, though we are not currently sure whether it's a good - idea to implement. - - We deliberately reject any design that would involve linking more code - into Tor's process space. - -Design overview - - To write a new transport protocol, an implementer must provide two - pieces: a "Client Proxy" to run at the initiator side, and a "Server - Proxy" to run at the server side. These two pieces may or may not be - implemented by the same program. - - Each client may run any number of Client Proxies. Each one acts like - a SOCKS proxy that accepts connections on localhost. Each one - runs on a different port, and implements one or more transport - methods. If the protocol has any parameters, they are passed from Tor - inside the regular username/password parts of the SOCKS protocol. - - Bridges (and maybe relays) may run any number of Server Proxies: these - programs provide an interface like stunnel: they get connections from the - network (typically by listening for connections on the network) and relay - them to the Bridge's real ORPort. - - To configure one of these programs, it should be sufficient simply to - list it in your torrc. The program tells Tor which transports it - provides. The Tor consensus should carry a new approved version number that - is specific for pluggable transport; this will allow Tor to know when a - particular transport is known to be unsafe safe or non-functional. - - Bridges (and maybe relays) report in their descriptors which transport - protocols they support. This information can be copied into bridge - lines. Bridges using a transport protocol may have multiple bridge - lines. - - Any methods that are wildly successful, we can bake into Tor. - -Specifications: Client behavior - - We extend the bridge line format to allow you to say which method - to use to connect to a bridge. - - The new format is: - "bridge method address:port [[keyid=]id-fingerprint] [k=v] [k=v] [k=v]" - - To connect to such a bridge, the Tor program needs to know which - local SOCKS proxy will support the transport called "method". It - then connects to this proxy, and asks it to connect to - address:port. If [id-fingerprint] is provided, Tor should expect - the public identity key on the TLS connection to match the digest - provided in [id-fingerprint]. If any [k=v] items are provided, - they are configuration parameters for the proxy: Tor should - separate them with semicolons and put them in the user and - password fields of the request, splitting them across the fields - as necessary. If a key or value value must contain a semicolon or - a backslash, it is escaped with a backslash. - - The "id-fingerprint" field is always provided in a field named - "keyid", if it was given. Method names must be C identifiers. - - Example: if the bridge line is "bridge trebuchet www.example.com:3333 - rocks=20 height=5.6m" AND if the Tor client knows that the - 'trebuchet' method is provided by a SOCKS5 proxy on - 127.0.0.1:19999, the client should connect to that proxy, ask it to - connect to www.example.com, and provide the string - "rocks=20;height=5.6m" as the username, the password, or split - across the username and password. - - There are two ways to tell Tor clients about protocol proxies: - external proxies and managed proxies. An external proxy is configured - with - ClientTransportPlugin method socks4 address:port [auth=X] - or - ClientTransportPlugin method socks5 address:port [username=X] [password=Y] - as in - "ClientTransportPlugin trebuchet socks5 127.0.0.1:9999". - This example tells Tor that another program is already running to handle - 'trubuchet' connections, and Tor doesn't need to worry about it. - - A managed proxy is configured with - ClientTransportPlugin <method> exec <path> [options] - as in - "ClientTransportPlugin trebuchet exec /usr/libexec/trebuchet --managed" - This example tells Tor to launch an external program to provide a - socks proxy for 'trebuchet' connections. The Tor client only - launches one instance of each external program with a given set of - options, even if the same executable and options are listed for - more than one method. - - If instead of a transport name, the torrc lists "*" for a managed proxy, - tor uses that proxy for all transports that it supports. So - "ClientTransportPlugin * exec /usr/libexec/tor/foobar" tells Tor - that it should use the foobar plugin for everything that it supports. - - If two proxies support the same method, Tor should use whichever - one is listed first. - - The same program can implement a managed or an external proxy: it just - needs to take an argument saying which one to be. - - See "Managed proxy behavior" for more information on the managed - proxy interface. - -Server behavior - - Server proxies are configured similarly to client proxies. When - launching a proxy, the server must tell it what ORPort it has - configured, and what address (if any) it can listen on. The - server must tell the proxy which (if any) methods it should - provide if it can; the proxy needs to tell the server which - methods it is actually providing, and on what ports. - - When a client connects to the proxy, the proxy may need a way to - tell the server some identifier for the client address. It does - this in-band. - - As before, the server lists proxies in its torrc. These can be - external proxies that run on their own, or managedproxies that Tor - launches. - - An external server proxy is configured as - ServerTransportPlugin method proxy address:port param=val.. - as in - ServerTransportPlugin trebuchet proxy 127.0.0.1:999 rocks=heavy - The param=val pairs and the address are used to make the bridge - configuration information that we'll tell users. - - A managed proxy is configured as - ServerTransportPlugin method exec /path/to/binary [options] - or - ServerTransportPlugin * exec /path/to/binary [options] - - When possible, Tor should launch only one binary of each binary/option - pair configured. So if the torrc contains - - ClientTransportPlugin foo exec /usr/bin/megaproxy --foo - ClientTransportPlugin bar exec /usr/bin/megaproxy --bar - ServerTransportPlugin * exec /usr/bin/megaproxy --foo - - then Tor will launch the megaproxy binary twice: once with the option - --foo and once with the option --bar. - -Managed proxy interface - - When the Tor client launches a client proxy from the command - line, it communicates via environment variables. At a minimum, - it sets: - - {Client and server} - HOME, PATH -- as you'd expect. - - "STATE_LOCATION" -- a directory where the proxy should store - state if it wants to. This directory is not required to - exist, but the proxy SHOULD be able to create it if it - doesn't. The proxy SHOULD NOT store state elsewhere. - - "MANAGED_TRANSPORT_VER=1" -- To tell the proxy which versions - of this configuration protocol Tor supports. Future versions - will give a comma-separated list. Clients MUST accept - comma-separated lists containing any version that they - recognize, and MUST work correctly even if some of the - versions they don't recognize are non-numeric. - - {Client only} - - "CLIENT_TRANSPORTS" -- a comma-separated list of which methods - this client should enable, or * if all methods should be - enabled. The proxy SHOULD ignore methods that it doesn't - recognize. - - {Server only} - - "EXT_SERVER_PORT=addr:portnum" -- A port (probably on localhost) that - speaks the extended server protocol. - - "ORPORT=addr:portnum" -- Our regular ORPort in a form suitable - for local connections. - - "BINDADDR=addr" -- An address on which to listen for local - connections. This might be the advertised address, or might - be a local address that Tor will forward ports to. It MUST - be an address that will work with bind(). - - "SERVER_TRANSPORTS=..." -- A comma-separated list of server - methods that the proxy should support, or * - - The transport proxy replies by writing NL-terminated lines to - stdout. The metaformat is - - Keyword OptArgs NL - OptArgs = Args | - Args = SP ArgChar | Args ArgChar - ArgChar = Any character but NUL or NL - Keyword = KeywordChar | Keyword KeywordChar - KeyWordChar = All alphanumeric characters, dash, and underscore. - - Tor MUST ignore lines with keywords that it doesn't recognize. - - First, the proxy writes "VERSION 1" to say that it supports this - protocol. It must either pick a version that Tor told it about, or - pick no version at all, and say "ERROR no-version\n" and exit. - - The proxy should then open its ports. If running as a client - proxy, it should not use fixed ports; instead it should autoselect - ports to avoid conflicts. A client proxy should by default only - listen on localhost for connections. - - A server proxy SHOULD try listen at a consistent port, though it - SHOULD pick a different one if the port it last used is now allocated. - - A client or server proxy then should tell which methods it has - made available and how. It does this by printing zero or more - CMETHOD and SMETHOD lines to its stdout. These lines look like: - - CMETHOD methodname SOCKS4/SOCKS5 address:port [ARGS=arglist] \ - [OPT-ARGS=arglist] - - as in - - CMETHOD trebuchet SOCKS5 127.0.0.1:19999 ARGS=rocks,height \ - OPT-ARGS=tensile-strength - - The ARGS field lists mandatory parameters that must appear in - every bridge line for this method. The OPT-ARGS field lists - optional parameters. If no ARGS or OPT-ARGS field is provided, - Tor should not check the parameters in bridge lines for this - method. - - The proxy should print a single "CMETHODS DONE" line after it is - finished telling Tor about the client methods it provides. If it - tries to supply a client method but can't for some reason, it - should say: - CMETHOD-ERROR methodname "Message" - - A proxy should tell Tor about the server methods it is providing - by printing zero or more SMETHOD lines. These lines look like: - - SMETHOD methodname address:port [Options] - - If there's an error setting up a configured server method, the - proxy should say: - SMETHOD-ERROR methodname "message" - - The 'address:port' part of an SMETHOD line is the address to put - in the bridge line. The ARGS: part is a list of key-value pairs - that the client needs to know. The Options part is a list of - space-separated K:V flags that Tor should know about. Recognized - options are: - - - FORWARD:1 - - If this option is set, and address:port is not a publicly - accessible address, then the bridge needs to forward some - other address:port to address:port via upnp-helper. - - - ARGS:k=v,k=v,k=v - - If this option is set, the K=V arguments are added to the - extrainfo document. - - - DECLARE:K=V,... - - If this option is set, all the K=V options should be - added as extension entries to the router descriptor. (See - below) - - - USE-EXTPORT:1 - - If this option is set, the server plugin is using the - extended server port. - - SMETHOD and CMETHOD lines may be interspersed. After the list - SMETHOD line, the proxy says "SMETHODS DONE" - - The proxy SHOULD NOT tell Tor about a server or client method - unless it is actually open and ready to use. - - Tor clients SHOULD NOT use any method from a client proxy or - advertise any method from a server proxy UNLESS it is listed as a - possible method for that proxy in torrc, and it is listed by the - proxy as a method it supports. - - Proxies should respond to a single INT signal by closing their - listener ports and not accepting any new connections, but keeping - all connections open, then terminating when connections are all - closed. Proxies should respond to a second INT signal by shutting - down cleanly. - -The extended ORPort protocol. - - Server transports may need to connect to the bridge and pass - additional information about client connections that the bridge - would ordinarily receive from the kernel's TCP stack. To to this, - they connect to the "extended server port" as given in - SERVER_PORT, sent a short amount of information, wait for a - response, and then send the user traffic on that port. - - The extended server port protocol is as follows: - - COMMAND [2 bytes, big-endian] - BODYLEN [2 bytes, big-endian] - BODY [Bodylen bytes] - - Commands sent from the transport to the server are: - - [0x0000] DONE: There is no more information to give. (body ignored) - - [0x0001] USERADDR: an address:port string that represents the user's - address. If the transport doesn't actually do addresses, - this shouldn't be sent. - - Replies sent from tor to the proxy are: - - [0x1001] OKAY: Send the user's traffic. (body ignored) - - [0x1002] DENY: Tor would prefer not to get more traffic from - this address for a while. (body ignored) - - [We could also use an out-of-band signalling method to tell Tor - about client addresses, but that's a historically error-prone way - to go about annotating connections.] - -Advertising bridge methods: - - Bridges put the 'method' lines in their extra-info documents. - - method SP methodname SP address:port SP arglist NL - - The address:port parse are as returned from an SMETHOD line. The - arglist is a K=V,... list as retuned in the ARGS part of the - SMETHOD line. - - If the SMETHOD line includes a DECLARE: part, the routerinfo gets - a new line: - - method-info SP methodname SP arglist NL - -Bridge authority behavior - - We need to specify a way to test different transport methods that - bridges claim to support. We should test as many as possible. We - should NOT require that we have a way to tra - -Bridgedb behavior: - - Bridgedb can, given a set of router descriptors and their - corresponding extrainfo documents, generate a set of bridge lines - for each descriptor. Bridgedb may want to avoid handing out - methods that seem to get bridges blocked quickly. - -Implementation plan - - First, we should implement per-bridge socks settings (as - described above in "manually configuring a client proxy for a - bridge") and the extended-server-port mechanism. This will let - bridges run transport proxies such that they can hand-generate - bridge lines to give to clients for testing. - - Once that's done, we can improve usability a little bit by - implementing external proxies. Once that's done, we can see if we - need any managed proxies, or if the whole idea there is silly. - - If we do, the next most important part seems to be getting - the client-side automatic part written. And once that's done, we - can evaluate how much of the server side is easy for people to do - and how much is hard. - - The "obfsproxy" obfuscating proxy is a likely candidate for an - initial transport, as is Steven Murdoch's http thing or something - similar. - -Notes on plugins to write: - - We should ship a couple of null plugin implementations in one or two - popular, portable languages so that people get an idea of how to - write the stuff. - - 1. We should have one that's just a proof of concept that does - nothing but transfer bytes back and forth. - - 1. We should not do a rot13 one. - - 2. We should implement a basic proxy that does not transform the bytes at all - - 1. We should implement DNS or HTTP using other software (as goodesll - did years ago with DNS) as an example of wrapping existing code into - our plugin model. - - 2. The obfuscated-ssh superencipherment is pretty trivial and pretty - useful. It makes the protocol stringwise unfingerprintable. - - 1. Nick needs to be told firmly not to bikeshed the obfuscated-ssh - superencipherment too badly - - 1. Go ahead, bikeshed my day - - 1. If we do a raw-traffic proxy, openssh tunnels would be the logical choice. - -Appendix: recommendations for transports - - Be free/open-source software. Also, if you think your code might - someday do so well at circumvention that it should be implemented - inside Tor, it should use the same license as Tor. - - Use libraries that Tor already requires. (You can rely on openssl and - libevent being present if current Tor is present.) - - Be portable: most Tor users are on Windows, and most Tor developers - are not, so designing your code for just one of these platforms will - make it either get a small userbase, or poor auditing. - - Think secure: if your code is in a C-like language, and it's hard to - read it and become convinced it's safe, then it's probably not safe. - - Think small: we want to minimize the bytes that a Windows user needs - to download for a transport client. - - Avoid security-through-obscurity if possible. Specify. - - Resist trivial fingerprinting: There should be no good string or regex - to search for to distinguish your protocol from protocols permitted by - censors. - - Imitate a real profile: There are many ways to implement most - protocols -- and in many cases, most possible variants of a given - protocol won't actually exist in the wild. - - -

1 0

[tor/maint-0.2.2] Consider sending stream-level SENDME cells on partial flushes.
by nickm＠torproject.org 15 Mar '11

15 Mar '11

commit 1d36a8e9ae766f67097f4c5e522c33ff0c4a693f Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Mar 14 17:48:45 2011 -0400 Consider sending stream-level SENDME cells on partial flushes. Right now, we only consider sending stream-level SENDME cells when we have completely flushed a connection_edge's outbuf, or when it sends us a DATA cell. Neither of these is ideal for throughput. This patch changes the behavior so we now call connection_edge_consider_sending_sendme when we flush _some_ data from an edge outbuf. Fix for bug 2756; bugfix on svn r152. --- changes/bug2756 | 11 +++++++++++ src/or/connection.c | 2 ++ src/or/connection_edge.c | 17 +++++++++++++++++ src/or/connection_edge.h | 1 + 4 files changed, 31 insertions(+), 0 deletions(-) diff --git a/changes/bug2756 b/changes/bug2756 new file mode 100644 index 0000000..0cad515 --- /dev/null +++ b/changes/bug2756 @@ -0,0 +1,11 @@ + o Minor bugfixes (spec conformance, performance): + - We now ask the other side of a stream (the client or the exit) + for more data on that stream when the amount of queued data on + that stream dips low enough. Previously, we wouldn't ask the + other side for more data until either it sent us more data + (which it wasn't supposed to do if it had exhausted its + window!) or until we had completely flushed all our queued + data. Fixing this should improve throughput. Fixes bug 2756; + bugfix on the earliest released versions of Tor (svn commit + r152). + diff --git a/src/or/connection.c b/src/or/connection.c index fd30ac8..86ed45e 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -3245,6 +3245,8 @@ connection_flushed_some(connection_t *conn) r = connection_dirserv_flushed_some(TO_DIR_CONN(conn)); } else if (conn->type == CONN_TYPE_OR) { r = connection_or_flushed_some(TO_OR_CONN(conn)); + } else if (CONN_IS_EDGE(conn)) { + r = connection_edge_flushed_some(TO_EDGE_CONN(conn)); } conn->in_flushed_some = 0; return r; diff --git a/src/or/connection_edge.c b/src/or/connection_edge.c index f02479f..85a5225 100644 --- a/src/or/connection_edge.c +++ b/src/or/connection_edge.c @@ -301,6 +301,23 @@ connection_edge_end_errno(edge_connection_t *conn) return connection_edge_end(conn, reason); } +/** We just wrote some data to conn; act appropriately. + * + * (That is, if it's open, consider sending a stream-level sendme cell if we + * have just flushed enough.) + */ +int +connection_edge_flushed_some(edge_connection_t *conn) +{ + switch (conn->_base.state) { + case AP_CONN_STATE_OPEN: + case EXIT_CONN_STATE_OPEN: + connection_edge_consider_sending_sendme(conn); + break; + } + return 0; +} + /** Connection conn has finished writing and has no bytes left on * its outbuf. * diff --git a/src/or/connection_edge.h b/src/or/connection_edge.h index fa5f91c..0b08dd0 100644 --- a/src/or/connection_edge.h +++ b/src/or/connection_edge.h @@ -23,6 +23,7 @@ int connection_edge_process_inbuf(edge_connection_t *conn, int connection_edge_destroy(circid_t circ_id, edge_connection_t *conn); int connection_edge_end(edge_connection_t *conn, uint8_t reason); int connection_edge_end_errno(edge_connection_t *conn); +int connection_edge_flushed_some(edge_connection_t *conn); int connection_edge_finished_flushing(edge_connection_t *conn); int connection_edge_finished_connecting(edge_connection_t *conn);

1 0