- tor-commits - lists.torproject.org

[torspec/master] Clarify and revise the pluggable transport proposal.
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit 2bd447f1ce3603c6ca7b23294d5f196dea9a0769 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 17:05:46 2011 -0400 Clarify and revise the pluggable transport proposal. Most notably, combine the managed plugin sections. Also, the "raw socks4/socks5" thing is dumb; take it out. Clients can do that just find with external plugins. Instead of saying bridge socks4 addr:port keyid proxy=127.0.0.1:9999 , you can just say bridge obfs addr addr:port keyid ClientTransportPlugin obfs socks4 127.0.0.1:9999 Which is actually way cleaner. --- proposals/ideas/xxx-pluggable-transport.txt | 337 +++++++++++++++------------ 1 files changed, 183 insertions(+), 154 deletions(-) diff --git a/proposals/ideas/xxx-pluggable-transport.txt b/proposals/ideas/xxx-pluggable-transport.txt index 92c14ef..201f0ac 100644 --- a/proposals/ideas/xxx-pluggable-transport.txt +++ b/proposals/ideas/xxx-pluggable-transport.txt @@ -119,10 +119,9 @@ Design overview inside the regular username/password parts of the SOCKS protocol. Bridges (and maybe relays) may run any number of Server Proxies: these - programs provide an interface like stunnel-server (or whatever the - option is): they get connections from the network (typically by - listening for connections on the network) and relay them to the - Bridge's real ORPort. + programs provide an interface like stunnel: they get connections from the + network (typically by listening for connections on the network) and relay + them to the Bridge's real ORPort. To configure one of these programs, it should be sufficient simply to list it in your torrc. The program tells Tor which transports it @@ -139,18 +138,23 @@ Design overview Specifications: Client behavior - Bridge lines can now follow the extended format "bridge method - address:port [[keyid=]id-fingerprint] [k=v] [k=v] [k=v]". To connect - to such a bridge, a client must open a local connection to the SOCKS - proxy for "method", and ask it to connect to address:port. If - [id-fingerprint] is provided, it should expect the public identity key - on the TLS connection to match the digest provided in - [id-fingerprint]. If any [k=v] items are provided, they are - configuration parameters for the proxy: Tor should separate them with - semicolons and put them in the user and password fields of the request, - splitting them across the fields as necessary. If a key or value - value must contain a semicolon or a backslash, it is escaped with a - backslash. + We extend the bridge line format to allow you to say which method + to use to connect to a bridge. + + The new format is: + "bridge method address:port [[keyid=]id-fingerprint] [k=v] [k=v] [k=v]" + + To connect to such a bridge, the Tor program needs to know which + local SOCKS proxy will support the transport called "method". It + then connects to this proxy, and asks it to connect to + address:port. If [id-fingerprint] is provided, Tor should expect + the public identity key on the TLS connection to match the digest + provided in [id-fingerprint]. If any [k=v] items are provided, + they are configuration parameters for the proxy: Tor should + separate them with semicolons and put them in the user and + password fields of the request, splitting them across the fields + as necessary. If a key or value value must contain a semicolon or + a backslash, it is escaped with a backslash. The "id-fingerprint" field is always provided in a field named "keyid", if it was given. Method names must be C identifiers. @@ -165,101 +169,38 @@ Specifications: Client behavior There are two ways to tell Tor clients about protocol proxies: external proxies and managed proxies. An external proxy is configured - with "ClientTransportPlugin trebuchet socks5 127.0.0.1:9999". This - tells Tor that another program is already running to handle - 'trubuchet' connections, and Tor doesn't need to worry about it. A - managed proxy is configured with "ClientTransportPlugin trebuchet - exec /usr/libexec/tor-proxies/trebuchet [options]", and tells Tor to launch - an external program on-demand to provide a socks proxy for 'trebuchet' - connections. The Tor client only launches one instance of each - external program with a given set of options, even if the same - executable and options are listed for more than one method. + with + ClientTransportPlugin method socks4 address:port [auth=X] + or + ClientTransportPlugin method socks5 address:port [username=X] [password=Y] + as in + "ClientTransportPlugin trebuchet socks5 127.0.0.1:9999". + This example tells Tor that another program is already running to handle + 'trubuchet' connections, and Tor doesn't need to worry about it. + + A managed proxy is configured with + ClientTransportPlugin <method> exec <path> [options] + as in + "ClientTransportPlugin trebuchet exec /usr/libexec/trebuchet --managed" + This example tells Tor to launch an external program to provide a + socks proxy for 'trebuchet' connections. The Tor client only + launches one instance of each external program with a given set of + options, even if the same executable and options are listed for + more than one method. If instead of a transport name, the torrc lists "*" for a managed proxy, - tor uses that proxy for all transports that it supports. + tor uses that proxy for all transports that it supports. So + "ClientTransportPlugin * exec /usr/libexec/tor/foobar" tells Tor + that it should use the foobar plugin for everything that it supports. + + If two proxies support the same method, Tor should use whichever + one is listed first. The same program can implement a managed or an external proxy: it just needs to take an argument saying which one to be. -Client proxy behavior - - When the Tor client launches a client proxy from the command line, it - sets the environment variable - "CLIENT_TRANSPORT_VER=1" - to tell the proxy which versions of this configuration protocol - it supports. Future versions will give a comma-separated list. - - The Tor client also sets the environment variable - CLIENT_TRANSPORTS to a comma-separated list of which methods this - client should enable, or * if all methods should be enabled. - - The Tor client also sets STATE_LOCATION to a directory where - where the proxy should store state, if it wants to. - - The transport proxy replies by printing "VERSION: 1\n" to its - stdout to say that it supports this protocol. It must either - pick a version that Tor told it about, or pick no version at all, - and say "ERROR: no-version\n" and exit. - - It then needs to tell Tor which methods and ports it - supports. It does this by printing zero or more CMETHOD: lines - to its stdout. These look like - - CMETHOD: trebuchet SOCKS5 127.0.0.1:19999 ARGS=rocks,height \ - OPT-ARGS=tensile-strength - - The ARGS field lists mandatory parameters that must appear in every - bridge line for this method. The OPT-ARGS field lists optional - parameters. If no ARGS or OPT-ARGS field is provided, Tor should not - check the parameters in bridge lines for this method. - - The proxy should print a single "CMETHODS: DONE" line after it is - finished telling Tor about the methods it provides. - - The transport proxy MUST exit cleanly when it receives a SIGTERM from - Tor. - - The Tor client MUST ignore lines beginning with a keyword and a colon - if it does not recognize the keyword. - - In the future, if we need a control mechanism, we can use the - stdin/stdout from Tor to the transport proxy. - - A transport proxy MUST handle SOCKS connect requests using the SOCKS - version it advertises. - - Tor clients SHOULD NOT use any method from a client proxy unless it - is both listed as a possible method for that proxy in torrc, and it - is listed by the proxy as a method it supports. - -Manually configuring a client proxy for a bridge - - All clients will support the methods "socks4" and "socks5". Users can use - these to configure a local proxy that doesn't support this plug-in - infrastructure method; developers can use them to test new proxies before - they have added support for this plug-in in. - - A bridge configured with these methods looks like: - - bridge socks4 www.example.com:8888 keyid=(fingerprint) \ - proxy=127.0.0.1:9999 auth=xyz - - or - - bridge socks5 www.example.com:8888 keyid=(fingerprint) \ - proxy=127.0.0.1:9999 user=x password=y - - The "proxy" argument for these methods is mandatory: it specifies a proxy - to use when talking to the bridge. The auth or user/password arguments for - these methods are optional: they are passed to the proxy either as its - authentication part (for socks4) or its username/password part (for - socks5). - - The socks4 method uses SOCKS4 if the bridge is given as an IP - address, and SOCKS4A if the bridge is given as a hostname. - - [We'll want to implement this part first, since it lets us test out - per-bridge proxies, albeit with a user-unfriendly manner.] + See "Managed proxy behavior" for more information on the managed + proxy interface. Server behavior @@ -278,17 +219,17 @@ Server behavior external proxies that run on their own, or managedproxies that Tor launches. - An external proxy is configured as - - ServerTransportPlugin trebuchet address:port [options] -- param=val param=val... + An external server proxy is configured as + ServerTransportPlugin method proxy address:port param=val.. + as in + ServerTransportPlugin trebuchet proxy 127.0.0.1:999 rocks=heavy + The param=val pairs and the address are used to make the bridge + configuration information that we'll tell users. A managed proxy is configured as - - ServerTransportPlugin trebuchet exec /path/to/binary [options] + ServerTransportPlugin method exec /path/to/binary [options] or ServerTransportPlugin * exec /path/to/binary [options] - The param=val pairs in the external proxy configuration, and the address, - are advertised to make our bridge configuration. When possible, Tor should launch only one binary of each binary/option pair configured. So if the torrc contains @@ -300,52 +241,106 @@ Server behavior then Tor will launch the megaproxy binary twice: once with the option --foo and once with the option --bar. - When the server launches managed binaries, it sets these environment - variables: - SERVER_TRANSPORT_VER=1 - (As CLIENT_TRANPORT_VER) +Managed proxy interface + + When the Tor client launches a client proxy from the command + line, it communicates via environment variables. At a minimum, + it sets: + + {Client and server} + HOME, PATH -- as you'd expect. + + "STATE_LOCATION" -- a directory where the proxy should store + state if it wants to. This directory is not required to + exist, but the proxy SHOULD be able to create it if it + doesn't. The proxy SHOULD NOT store state elsewhere. + + "MANAGED_TRANSPORT_VER=1" -- To tell the proxy which versions + of this configuration protocol Tor supports. Future versions + will give a comma-separated list. Clients MUST accept + comma-separated lists containing any version that they + recognize, and MUST work correctly even if some of the + versions they don't recognize are non-numeric. + + {Client only} + + "CLIENT_TRANSPORTS" -- a comma-separated list of which methods + this client should enable, or * if all methods should be + enabled. The proxy SHOULD ignore methods that it doesn't + recognize. + + {Server only} + + "EXT_SERVER_PORT=addr:portnum" -- A port (probably on localhost) that + speaks the extended server protocol. + + "ORPORT=addr:portnum" -- Our regular ORPort in a form suitable + for local connections. + + "BINDADDR=addr" -- An address on which to listen for local + connections. This might be the advertised address, or might + be a local address that Tor will forward ports to. It MUST + be an address that will work with bind(). - EXT_SERVER_PORT=addr:portnum - (A port on localhost that speaks the extended server protocol) + "SERVER_TRANSPORTS=..." -- A comma-separated list of server + methods that the proxy should support, or * - ORPORT=addr:portnum - (Our regular orport in a form suitable for local connections) + The transport proxy replies by writing NL-terminated lines to + stdout. The metaformat is - BINDADDR=addr - (An address on which to listen to connections. This might be the - advertised address, or might be a local address that Tor will - forward ports to.) + Keyword OptArgs NL + OptArgs = Args | + Args = SP ArgChar | Args ArgChar + ArgChar = Any character but NUL or NL + Keyword = KeywordChar | Keyword KeywordChar + KeyWordChar = All alphanumeric characters, dash, and underscore. - SERVER_TRANSPORTS=... - (A comma-separated list of server methods that the proxy - should support, or *). + Tor MUST ignore lines with keywords that it doesn't recognize. - STATE_LOCATION=... - (A directory where where the proxy should store state, if it - wants to.) + First, the proxy writes "VERSION 1" to say that it supports this + protocol. It must either pick a version that Tor told it about, or + pick no version at all, and say "ERROR no-version\n" and exit. - It also opens an extending server port as described below. + The proxy should then open its ports. If running as a client + proxy, it should not use fixed ports; instead it should autoselect + ports to avoid conflicts. A client proxy should by default only + listen on localhost for connections. -Server proxy behavior + A server proxy SHOULD try listen at a consistent port, though it + SHOULD pick a different one if the port it last used is now allocated. - The server proxy communicates with the server as the client does. - Both start with a version line to indicate which protocol they - have chosen (or an error line if it supports no version in common - with tor), then it lists a number of SMETHOD lines. + A client or server proxy then should tell which methods it has + made available and how. It does this by printing zero or more + CMETHOD and SMETHOD lines to its stdout. These lines look like: - Each SMETHOD line is of the form: + CMETHOD methodname SOCKS4/SOCKS5 address:port [ARGS=arglist] \ + [OPT-ARGS=arglist] - SMETHOD: methodname address:port ARGS:k=v,k=v,k=v [Options] + as in - Or: + CMETHOD trebuchet SOCKS5 127.0.0.1:19999 ARGS=rocks,height \ + OPT-ARGS=tensile-strength + + The ARGS field lists mandatory parameters that must appear in + every bridge line for this method. The OPT-ARGS field lists + optional parameters. If no ARGS or OPT-ARGS field is provided, + Tor should not check the parameters in bridge lines for this + method. - SMETHOD-ERROR: methodname message + The proxy should print a single "CMETHODS DONE" line after it is + finished telling Tor about the client methods it provides. If it + tries to supply a client method but can't for some reason, it + should say: + CMETHOD-ERROR methodname "Message" - SMETHOD and CMETHOD lines may be interspersed. + A proxy should tell Tor about the server methods it is providing + by printing zero or more SMETHOD lines. These lines look like: - After the list SMETHOD line, the proxy says "SMETHODS: DONE" + SMETHOD methodname address:port [Options] - Each SMETHOD lime is a configured and working server method. + If there's an error setting up a configured server method, the + proxy should say: + SMETHOD-ERROR methodname "message" The 'address:port' part of an SMETHOD line is the address to put in the bridge line. The ARGS: part is a list of key-value pairs @@ -359,18 +354,47 @@ Server proxy behavior accessible address, then the bridge needs to forward some other address:port to address:port via upnp-helper. + - ARGS:k=v,k=v,k=v + + If this option is set, the K=V arguments are added to the + extrainfo document. + - DECLARE:K=V,... If this option is set, all the K=V options should be added as extension entries to the router descriptor. (See below) + - USE-EXTPORT:1 + + If this option is set, the server plugin is using the + extended server port. + + SMETHOD and CMETHOD lines may be interspersed. After the list + SMETHOD line, the proxy says "SMETHODS DONE" + + The proxy SHOULD NOT tell Tor about a server or client method + unless it is actually open and ready to use. + + Tor clients SHOULD NOT use any method from a client proxy or + advertise any method from a server proxy UNLESS it is listed as a + possible method for that proxy in torrc, and it is listed by the + proxy as a method it supports. + + Proxies should respond to a single INT signal by closing their + listener ports and not accepting any new connections, but keeping + all connections open, then terminating when connections are all + closed. Proxies should respond to a second INT signal by shutting + down cleanly. + +The extended ORPort protocol. + Server transports may need to connect to the bridge and pass - additional information about client connections that the bridge would - ordinarily receive from . To to this, they connect to the - "extended server port" as given in SERVER_PORT, sent a short amount of - information, wait for a response, and then send the user traffic - on that port. + additional information about client connections that the bridge + would ordinarily receive from the kernel's TCP stack. To to this, + they connect to the "extended server port" as given in + SERVER_PORT, sent a short amount of information, wait for a + response, and then send the user traffic on that port. The extended server port protocol is as follows: @@ -384,7 +408,7 @@ Server proxy behavior [0x0001] USERADDR: an address:port string that represents the user's address. If the transport doesn't actually do addresses, - this shouldn't besent. + this shouldn't be sent. Replies sent from tor to the proxy are: @@ -431,11 +455,16 @@ Implementation plan described above in "manually configuring a client proxy for a bridge") and the extended-server-port mechanism. This will let bridges run transport proxies such that they can hand-generate - bridge lines to give to clients for testing. Once that's done, - the next most important part seems to be getting the client-side - automatic part written. And once that's done, we can evaluate how - much of the server side is easy for people to do and how much is - hard. + bridge lines to give to clients for testing. + + Once that's done, we can improve usability a little bit by + implementing external proxies. Once that's done, we can see if we + need any managed proxies, or if the whole idea there is silly. + + If we do, the next most important part seems to be getting + the client-side automatic part written. And once that's done, we + can evaluate how much of the server side is easy for people to do + and how much is hard. The "obfsproxy" obfuscating proxy is a likely candidate for an initial transport, as is Steven Murdoch's http thing or something

1 0

[torspec/master] abandon a missing sentence in xxx-ipv6-plan
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit e83442ee76c18e0bc9f5814f6fa817300d9a43c2 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 17:09:20 2011 -0400 abandon a missing sentence in xxx-ipv6-plan --- proposals/ideas/xxx-ipv6-plan.txt | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/ideas/xxx-ipv6-plan.txt b/proposals/ideas/xxx-ipv6-plan.txt index c59dcd4..26a0359 100644 --- a/proposals/ideas/xxx-ipv6-plan.txt +++ b/proposals/ideas/xxx-ipv6-plan.txt @@ -47,8 +47,9 @@ What needs to change: Designs that we will need to do: For IPv6-only clients, we'll need to specify that routers can have - multiple addresses and ORPorts, and allow secondary addresses/ports - that[...? XXX]. There is an old proposal (118) to try to allow multiple + multiple addresses and ORPorts. + + There is an old proposal (118) to try to allow multiple ORPorts per router. It's been accepted; it needs to be checked for correctness, updated to track other changes in more recent Tor versions, and updated to work with the new microdescriptor designs.

1 0

[tor/master] Check newconn, not conn, before marking newconn for close
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit 06f026de6624ea0591b8d08da9c5c69ffa44568c Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Mar 14 18:52:15 2011 -0400 Check newconn, not conn, before marking newconn for close This fixes bug 2757. It's a bugfix on 85da676108f0d. --- changes/bug2757 | 6 ++++++ src/or/connection.c | 2 +- 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/changes/bug2757 b/changes/bug2757 new file mode 100644 index 0000000..f947afa --- /dev/null +++ b/changes/bug2757 @@ -0,0 +1,6 @@ + - Minor bugfixes + o Avoid a double-mark-for-free warning when failing to attach a + transparent proxy connection. (We thought we had fixed this in + 0.2.2.23-alpha, but it turns out our fix was checking the wrong + connection.) Fixes bug 2757; bugfix on 0.1.2.1-alpha (the original + bug) and 0.2.2.23-alpha (the incorrect fix). diff --git a/src/or/connection.c b/src/or/connection.c index fd30ac8..1894bc7 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -1178,7 +1178,7 @@ connection_handle_listener_read(connection_t *conn, int new_type) } if (connection_init_accepted_conn(newconn, conn->type) < 0) { - if (! conn->marked_for_close) + if (! newconn->marked_for_close) connection_mark_for_close(newconn); return 0; }

1 0

[tor/master] Merge remote branch 'origin/maint-0.2.2'
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit 4dde69a6d003682217c2bf6327b5923c7b811974 Merge: c42bd52 ab41844 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 15:19:38 2011 -0400 Merge remote branch 'origin/maint-0.2.2' changes/bug2757 | 6 ++++++ src/or/connection.c | 2 +- 2 files changed, 7 insertions(+), 1 deletions(-)

1 0

[tor/master] Merge branch 'bug2757' into maint-0.2.2
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit ab418447eb9838a8a6f5588e374910d753740d0f Merge: 5938804 06f026d Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 15:18:15 2011 -0400 Merge branch 'bug2757' into maint-0.2.2 changes/bug2757 | 6 ++++++ src/or/connection.c | 2 +- 2 files changed, 7 insertions(+), 1 deletions(-)

1 0

[tor/maint-0.2.2] Check newconn, not conn, before marking newconn for close
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit 06f026de6624ea0591b8d08da9c5c69ffa44568c Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Mar 14 18:52:15 2011 -0400 Check newconn, not conn, before marking newconn for close This fixes bug 2757. It's a bugfix on 85da676108f0d. --- changes/bug2757 | 6 ++++++ src/or/connection.c | 2 +- 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/changes/bug2757 b/changes/bug2757 new file mode 100644 index 0000000..f947afa --- /dev/null +++ b/changes/bug2757 @@ -0,0 +1,6 @@ + - Minor bugfixes + o Avoid a double-mark-for-free warning when failing to attach a + transparent proxy connection. (We thought we had fixed this in + 0.2.2.23-alpha, but it turns out our fix was checking the wrong + connection.) Fixes bug 2757; bugfix on 0.1.2.1-alpha (the original + bug) and 0.2.2.23-alpha (the incorrect fix). diff --git a/src/or/connection.c b/src/or/connection.c index fd30ac8..1894bc7 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -1178,7 +1178,7 @@ connection_handle_listener_read(connection_t *conn, int new_type) } if (connection_init_accepted_conn(newconn, conn->type) < 0) { - if (! conn->marked_for_close) + if (! newconn->marked_for_close) connection_mark_for_close(newconn); return 0; }

1 0

[tor/maint-0.2.2] Merge branch 'bug2757' into maint-0.2.2
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit ab418447eb9838a8a6f5588e374910d753740d0f Merge: 5938804 06f026d Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 15:18:15 2011 -0400 Merge branch 'bug2757' into maint-0.2.2 changes/bug2757 | 6 ++++++ src/or/connection.c | 2 +- 2 files changed, 7 insertions(+), 1 deletions(-)

1 0

[tor/master] Fix a compile warning when using clang
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit 30b3f0cdb148f85330e2c93ebb1c00f3cfc8655a Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Wed Mar 9 16:39:04 2011 +0100 Fix a compile warning when using clang Issue noticed by Steven Murdoch; fixes bug 2689. The cast didn't do anything, and we don't need to look at the return value of the function here. --- src/or/rendclient.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/or/rendclient.c b/src/or/rendclient.c index af91099..51306b3 100644 --- a/src/or/rendclient.c +++ b/src/or/rendclient.c @@ -412,7 +412,7 @@ directory_get_from_hs_dir(const char *desc_id, const rend_data_t *rend_query) tor_assert(rend_query); /* Determine responsible dirs. Even if we can't get all we want, * work with the ones we have. If it's empty, we'll notice below. */ - (int) hid_serv_get_responsible_directories(responsible_dirs, desc_id); + hid_serv_get_responsible_directories(responsible_dirs, desc_id); base32_encode(desc_id_base32, sizeof(desc_id_base32), desc_id, DIGEST_LEN);

1 0

[tor/master] Merge remote branch 'origin/maint-0.2.1' into maint-0.2.2
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit 5938804d17f6b458fcec9d479cd734d2e0256631 Merge: 98abe14 30b3f0c Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 15:14:12 2011 -0400 Merge remote branch 'origin/maint-0.2.1' into maint-0.2.2 src/or/rendclient.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)

1 0

[tor/master] Merge remote branch 'origin/maint-0.2.2'
by nickm＠torproject.org 16 Mar '11

16 Mar '11

commit c42bd529dea474db1a1af4432b9fe5552c0ca07e Merge: 84f27eb 5938804 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Mar 15 15:14:28 2011 -0400 Merge remote branch 'origin/maint-0.2.2' src/or/rendclient.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)

1 0