- tor-dev - lists.torproject.org

First draft: Thandy Package Format spec
by Erinn Clark 31 Mar '11

31 Mar '11

Also available here: https://gitweb.torproject.org/erinn/thandy.git/blob/refs/heads/thp-draft-sp… Title: Specification for a TBB/Thandy package format. Status: Draft Authors: nickm, erinn Started-On: 9 Feb 2011 Finished-On: Introduction On some platforms, in some environments, Thandy can use existing platform-provided mechanisms for packages. But for the Tor Browser Bundle, and for Windows, we can't use built-in packaging systems (because they put data in a database or registry, because they require root). Requirements, Goals: Thandy has these requirements for a packaging system: - It needs to be able to install/upgrade packages. - It needs to be able to check which version of a package is installed For use in TBB, we we need a few more features: - It needs to be able to remove packages - It needs to be able to leave cofiguration files alone when upgrading To use this right, Thandy needs these features: - download packages as-needed in the background - report when packages are ready to install - Have a way to upgrade the TBB as it restarts To avoid big problems, package installation needs these features: - Idempotence - Validatability - Spending as little time as possible in non-functional states. (We can't get true atomicity on some OSs, but we should get as much as reasonable as we can.) Nongoals: This is a packagesystem for TBB and similar things for use with Thandy. It is not a more general system, a replacement for rpm/deb, an all-purpose software distribution mechanism, or a dessert topping. This is not the spec for Thandy. This is not a spec for interfaces to Thandy. It lists some interfaces that are necessary but it doesn't explain how they work. This is not the spec for any modes of operation for thandy, interfaces to thandy, or a thandy net installer. We need specs for those, of course. Though this document has suggestions on how to make good packages and install them well, there may be additional requirements for a high-quality installer not listed here. We're trying to design this system to _support_ being the most solid tool it can be, but we're not trying to figure out every possible implementation detail here. Mode of operation: While TBB is running, Vidalia should periodically launch Thandy, telling it, "Fetch packages as needed" or "Tell me if you could fetch packages." While TBB is running, Vidalia should periodically launch Thandy, asking it, "Are there packages downloaded and ready to install?" If so, it should tell the user. When TBB first starts, it should start a launcher program that asks Thandy, "Are there packages downloaded and ready to install?" If so, it should tell Thandy to install them. To make Thandy and the launcher self-upgrade, here's a two-step process: the install process installs new stuff into a new directory to the side of the old directory, and then launches a new "replacer" program to move the new stuff over the old stuff. The replacer, when it's done, re-launches the launcher. "It's not pretty, and you can't dance to it." - Frank Zappa The "thandy installable" file format: The file metaformat is a zip file, with the file extension .thp. It has these directories: content/ meta/ The "content" tree has the actual package contents, in a directory layout mirroring the layout of the installed package. The "meta" tree has information about the package. The meta tree has one required file, "package.json". Its contents are a single json object, with the following required fields: 'format-version' The number 1. An installer SHOULD NOT try to install a package with a format-version that it doesn't recognize. 'manifest' A list of files relative to the thp's content directory. Each file is an object with these fields: 'name': the name of the file relative to thp's content directory 'digest': optionally, a 2-tuple of a digest algorithm and a hexadecimal digest value. The following algorithms are supported: SHA256. 'length': optionally, the length of the file in bytes 'isconfig': optionally, a boolean. If it is present and true, this file is considered "configuration". 'package-name' The name of this package. There shouldn't be two packages with the same name installed at once in the same hierarchy. Ex: "Tor". 'package-version' The version of the package as a human-readable string. This should include both the version of the thing being packaged, and the version of of the package itself. Ex: "0.2.2.35-rc-7" is the seventh release of a thp file for Tor 0.2.2.35-rc. 'package-version-tuple' The version of the package as a list of numbers and strings such that a lexical comparison of two package version tuples is a correct version comparison. Ex: [ 0, 2, 2, 35, "rc", 7 ] 'timestamp' The time when this thp was generated, as a YYYY-MM-DD HH:MM:SS string, relative to UTC. Ex: "2011-03-02 17:33:07" 'additional-files' Optional: A list of files or file sets relative to the install root, to decribe files that the package is responsible for, even if they're not distributed with the package. This is used to kill off temporary and cache files on uninstall. Each file is an object with these fields: 'name': The name of the file relative to the install root. This name may contain "*" and "**" file-globbing patterns. 'isconfig': as for manifest. 'install-order' Optional: number between 0 and 100 inclusive to indicate that this package must be installed before or after others. Defaults to "50". 'options' Optional: a map from option strings to values. Known options are: 'cycle-install': This package should be installed to a separate install root from the currently installed package, then moved over. 'platform' Optional. The OS and CPU type that this package is for. 'require-features' Optional. A list of strings naming installer features that the installer needs to support to install this package correctly. An installer SHOULD NOT try to install the package if it does not recognize and support all members of this list. Ex: [ "pythonscripts" ] 'require-packages' Optional. A list of objects for all packages that must be installed before this package can be installed. Each object has these fields: 'package-name' 'min-version-tuple' 'scripts' Optional: a map from scripting language to set of scripts. Supported scripting languages are python2, sh, none. (We can add more later.) If the 'scripts' field is present, the installer SHOULD NOT install the package unless it supports one or more of the scripting languages. Each set of scripts contains one or more of the following fields: 'checkinst' 'preinst' 'postinst' 'prerm' 'postrm' Each names a file relative to meta/scripts in the thp zip file. Implementations SHOULD NOT generate other files or subdirectories of the main zip root directory; implementations MUST ignore files and subdirectories that they do not recognize. The package database: The installer keeps a directory that contains files describing the status of the packages we have installed. It has a subdirectory: "pkg-status". "pkg-status" has, for each package, two files: packagename.json, and packagename.status. Optionally, it has a packagename.json.new file. The packagename.json file contains a copy of the package.json file from the most recent successfully installed version of the the package. The packagename.status file contains a json object containing at least the field 'status' set to one of the following: "INSTALLED", "IN-PROGRESS". If a package install or upgrade is in-progress, packagename.json.new has the package.json file from the new version of the package. Scripts: Each script should be callable by a language- and platform-specific calling convention for invoking a script with named arguments. The arguments to the script are: THP_PACKAGE_NAME THP_OLD_VERSION (absent if we are doing a fresh installation) THP_NEW_VERSION THP_OLD_INSTALL_ROOT THP_INSTALL_ROOT THP_JSON_FILE THP_VERBOSE (flag: "1" if the script should log verbosely.) THP_PURGE (flag: "1" if we are doing a remove and we want to get rid of absolutely everything.) THP_TEMP_DIR For sh and python2 scripts, these arguments are passed as environment variables. Scripts need a way to signal success or failure. For sh and python2 scripts, this is done via return value. The 'none' script type must never have any scripts. Installers should never choose it if they support any other script type: it only exists to tell the installer that the scripts are optional. Directories: All installation happens relative to a single "install root". Thandy maintains a cache directory of its own, containing (among other things) downloaded thp files. The thp installer has a database directory explaining package status. Steps of operation: Checking for and downloading packages is done by thandy, and out-of-scope here, except inasmuch as Thandy needs the thp installer to say which version (if any) of package X is installed. The installer can do this by looking at the X.status file and the X.json file. To validate a package, the thp installer verifies that all files listed in the manifest are in fact installed with the sizes and digests listed, unless they are config files. {THIS NEXT PART IS A DRAFT AND NEEDS MORE THOUGHT! IT COULD BE WAY MORE ATOMIC} Installing and updating a downloaded package is done by Thandy teling the thp installer, "install/update these packages" with a list of thp files. To do this, the installer: * Grabs a lockfile. * Finds the current version, if any, of all of the packages. * Runs the checkinst script if present for every package that might need installing or updating. If any fails, the update can't happen. * For each package, sorted in topological order by 'requires' relationships, with ties broken by 'install-order' fields: * Overwrite the status file for the package in the database, changing the status to "IN-PROGRESS". Copy the package's package.json file to packagename.json.new in the database. * Run the package's preinst script. If that fails, abort. * Unpack all files in the package's content fork to their target locations. If files tagged 'config' are present, ignore them. * If any files are listed in the manifest for the old version of the package but not for the new version, and they are not config files, remove them. (Have an option to override this?) * Run the postinst script * Replace the packagename.json file with the packagename.json.new file. * Change the package status to "INSTALLED". CHANGES TO MAKE TO THE INSTALL PROCESS ABOVE: - Instead of overwrite-as-we-go, perhaps have overwrite as the very last step? - Perhaps checkpoint all files first for easy rollback? - Specify how to do the cycle-install feature. "Is it... atomic?" "Yes! VERY atomic!" -- The 5000 Fingers of Dr T Future directions and open questions: F.1 Configuration file updates We need a smart way to handle configuration file updates and changes in the future. There are several three-way merge tools available that we can model our behavior on, or re-use the code of, such as Debian's ucf tool. F.1 Binary patching There are auto-update tools (courgette for Chrome, as an example) which do smart binary patching, thus often drastically reducing the download time of updates. When the tool is more mature, we should look into ways to do this.

2 1

Designing and implementing improved circuit-setup protocol [was: GSoC 2011]
by Nick Mathewson 31 Mar '11

31 Mar '11

Quoting the whole message in my reply, since we're moving this to tor-dev where it belongs. On Wed, Mar 23, 2011 at 1:36 PM, Robert Ransom <rransom.8774(a)gmail.com> wrote: > On Wed, 23 Mar 2011 12:54:17 -0400 > Nick Mathewson <nickm(a)freehaven.net> wrote: > >> On Wed, Mar 23, 2011 at 8:10 AM, George Kadianakis <desnacked(a)gmail.com> wrote: >> > Hey Robert and Nick, >> > >> > I hate bureaucratic procedures and yet I have to send my GSoC >> > application 'till the 8th of April; so I want to get done with it. >> > >> > My application should have "a project proposal, why you'd like to >> > execute on this particular project, and the reason you're the best >> > individual to do so." (that was taken from GSoC FAQ). >> > >> > So regarding the project proposal, I talked with Robert yesterday and he >> > told me: >> > 02:18:24 <rransom> asn: Implementing the new circuit protocol will be >> > a good GSoC project for you. I think identity key >> > migration (and thus link protocol migration) will >> > require too much beyond the new circuit protocol. >> > I suppose that by "new circuit protocol" you mean the section 3.2 of >> > xxx-crypto-migration.txt, Robert. >> > >> > How do you feel about this, Nick? >> >> It could be helpful. I don't have a good sense of how much work this >> would be, since I don't have a good sense of what the new protocol >> would actually be. Robert -- how would you think this would break >> down into a summer's worth of work? > > The first step in the Great Tor Crypto Migration is to add new CREATE2 > and EXTEND2 RELAY cell types. They can be used with the existing > circuit-extension handshake and link protocol initially, but will be > extensible to support new ones. > > Further steps, all independent of each other: > > * Add 128-byte and 2048-byte RELAY cells and a circuit-configuration > cell, initially to allow the client to change the cell size to be > used on a circuit. > > * Refactor Tor's cryptographic primitive abstractions to accommodate > public-key encryption primitives, public-key signature primitives, > symmetric authenticated encryption, symmetric block encryption, and > hashes. > > * Implement one or more new link protocols that do not constrain a > relay's choice of identity key cryptosystem. > > Further mutually independent steps building on those above: > > * Modify the directory protocol and implementation to support relays > with multiple identity keys. > > * Implement a new circuit-extension handshake (the part that involves > ‘onionskins’). > > * Implement a new circuit ciphersuite (the part that mangles cell data > so that relay A can't see what data relay C sees). Hm. These steps all stretch pretty far beyond what's just described in 3.2 of xxx-crypto-migration. I think they're probably more than we can promise to design before summer, and possibly more than a typical gsoc scope all put together. >> I'm also a little concerned about the interaction of 3.2 and 3.3 >> ("Relay crypto") : I'll be surprised if it turns out that we can >> design a good circuit extend protocol without thinking about the >> countours of a new relay protocol. (Not that you'd have to build both >> at once, but we should think about them all as we design.) > > It's actually the other way around -- we need a new EXTEND cell before > we can use a new link protocol. (Otherwise, we would have to build in > a covert channel (i.e. a backdoor for people who want to block Tor by > handshake) in the new link protocol to indicate client and server link > protocol versions, and that really *really* sucks.) I'm talking about the stuff in 3.3: the relay protocol, where we process cells. Link protocol stuff is 3.1. Also, I'm talking about *design order*, not *implementation order*: The different parts of the Tor protocol are not sufficiently orthogonal that we can do them independently. Thus, we need to get most of the design changes sketched out before we can be reasonably say that any part of the redesign does what the other parts need. >> Maybe we should get a protocol sketch together this week if the app is >> due April 8. > > Yes. I have the EXTEND2 cell draft written; I bogged down on writing > explanatory text (I thought I didn't have enough in the draft, but > didn't know what to add). > > >> Either way, I think this might be drifting into design work, so if >> there are no objections, I'd suggest that we follow up on the tor-dev >> mailing list. :) > > Good idea. > > > Robert Ransom >

3 2

(revised) Proposal 180: Pluggable Transports for Circumvention
by Nick Mathewson 21 Mar '11

21 Mar '11

We sent out an earlier version of this, well, earlier. Here it is more complete and more revised. I've tried to incorporate people's comments. There's some worthwhile stuff it doesn't do yet, like provide automatic and easy ways to compose plugins. I think it should work well enough, but more comment is always worthwhile. yrs, -- Nick ===== Filename: 180-pluggable-transport.txt Title: Pluggable transports for circumvention Author: Jacob Appelbaum, Nick Mathewson Created: 15-Oct-2010 Status: Open Overview This proposal describes a way to decouple protocol-level obfuscation from the core Tor protocol in order to better resist client-bridge censorship. Our approach is to specify a means to add pluggable transport implementations to Tor clients and bridges so that they can negotiate a superencipherment for the Tor protocol. Scope This is a document about transport plugins; it does not cover discovery improvements, or bridgedb improvements. While these requirements might be solved by a program that also functions as a transport plugin, this proposal only covers the requirements and operation of transport plugins. Motivation Frequently, people want to try a novel circumvention method to help users connect to Tor bridges. Some of these methods are already pretty easy to deploy: if the user knows an unblocked VPN or open SOCKS proxy, they can just use that with the Tor client today. Less easy to deploy are methods that require participation by both the client and the bridge. In order of increasing sophistication, we might want to support: 1. A protocol obfuscation tool that transforms the output of a TLS connection into something that looks like HTTP as it leaves the client, and back to TLS as it arrives at the bridge. 2. An additional authentication step that a client would need to perform for a given bridge before being allowed to connect. 3. An information passing system that uses a side-channel in some existing protocol to convey traffic between a client and a bridge without the two of them ever communicating directly. 4. A set of clients to tunnel client->bridge traffic over an existing large p2p network, such that the bridge is known by an identifier in that network rather than by an IP address. We could in theory support these almost fine with Tor as it stands today: every Tor client can take a SOCKS proxy to use for its outgoing traffic, so a suitable client proxy could handle the client's traffic and connections on its behalf, while a corresponding program on the bridge side could handle the bridge's side of the protocol transformation. Nevertheless, there are some reasons to add support for transportation plugins to Tor itself: 1. It would be good for bridges to have a standard way to advertise which transports they support, so that clients can have multiple local transport proxies, and automatically use the right one for the right bridge. 2. There are some changes to our architecture that we'll need for a system like this to work. For testing purposes, if a bridge blocks off its regular ORPort and instead has an obfuscated ORPort, the bridge authority has no way to test it. Also, unless the bridge has some way to tell that the bridge-side proxy at 127.0.0.1 is not the origin of all the connections it is relaying, it might decide that there are too many connections from 127.0.0.1, and start paring them down to avoid a DoS. 3. Censorship and anticensorship techniques often evolve faster than the typical Tor release cycle. As such, it's a good idea to provide ways to test out new anticensorship mechanisms on a more rapid basis. 4. Transport obfuscation is a relatively distinct problem from the other privacy problems that Tor tries to solve, and it requires a fairly distinct skill-set from hacking the rest of Tor. By decoupling transport obfuscation from the Tor core, we hope to encourage people working on transport obfuscation who would otherwise not be interested in hacking Tor. 5. Finally, we hope that defining a generic transport obfuscation plugin mechanism will be useful to other anticensorship projects. Non-Goals We're not going to talk about automatic verification of plugin correctness and safety via sandboxing, proof-carrying code, or whatever. We need to do more with discovery and distribution, but that's not what this proposal is about. We're pretty convinced that the problems are sufficiently orthogonal that we should be fine so long as we don't preclude a single program from implementing both transport and discovery extensions. This proposal is not about what transport plugins are the best ones for people to write. We do, however, make some general recommendations for plugin authors in an appendix. We've considered issues involved with completely replacing Tor's TLS with another encryption layer, rather than layering it inside the obfuscation layer. We describe how to do this in an appendix to the current proposal, though we are not currently sure whether it's a good idea to implement. We deliberately reject any design that would involve linking more code into Tor's process space. Design overview To write a new transport protocol, an implementer must provide two pieces: a "Client Proxy" to run at the initiator side, and a "Server Proxy" to run at the server side. These two pieces may or may not be implemented by the same program. Each client may run any number of Client Proxies. Each one acts like a SOCKS proxy that accepts connections on localhost. Each one runs on a different port, and implements one or more transport methods. If the protocol has any parameters, they are passed from Tor inside the regular username/password parts of the SOCKS protocol. Bridges (and maybe relays) may run any number of Server Proxies: these programs provide an interface like stunnel: they get connections from the network (typically by listening for connections on the network) and relay them to the Bridge's real ORPort. To configure one of these programs, it should be sufficient simply to list it in your torrc. The program tells Tor which transports it provides. The Tor consensus should carry a new approved version number that is specific for pluggable transport; this will allow Tor to know when a particular transport is known to be unsafe safe or non-functional. Bridges (and maybe relays) report in their descriptors which transport protocols they support. This information can be copied into bridge lines. Bridges using a transport protocol may have multiple bridge lines. Any methods that are wildly successful, we can bake into Tor. Specifications: Client behavior We extend the bridge line format to allow you to say which method to use to connect to a bridge. The new format is: "bridge method address:port [[keyid=]id-fingerprint] [k=v] [k=v] [k=v]" To connect to such a bridge, the Tor program needs to know which local SOCKS proxy will support the transport called "method". It then connects to this proxy, and asks it to connect to address:port. If [id-fingerprint] is provided, Tor should expect the public identity key on the TLS connection to match the digest provided in [id-fingerprint]. If any [k=v] items are provided, they are configuration parameters for the proxy: Tor should separate them with semicolons and put them in the user and password fields of the request, splitting them across the fields as necessary. If a key or value value must contain a semicolon or a backslash, it is escaped with a backslash. The "id-fingerprint" field is always provided in a field named "keyid", if it was given. Method names must be C identifiers. Example: if the bridge line is "bridge trebuchet www.example.com:3333 rocks=20 height=5.6m" AND if the Tor client knows that the 'trebuchet' method is provided by a SOCKS5 proxy on 127.0.0.1:19999, the client should connect to that proxy, ask it to connect to www.example.com, and provide the string "rocks=20;height=5.6m" as the username, the password, or split across the username and password. There are two ways to tell Tor clients about protocol proxies: external proxies and managed proxies. An external proxy is configured with ClientTransportPlugin method socks4 address:port [auth=X] or ClientTransportPlugin method socks5 address:port [username=X] [password=Y] as in "ClientTransportPlugin trebuchet socks5 127.0.0.1:9999". This example tells Tor that another program is already running to handle 'trubuchet' connections, and Tor doesn't need to worry about it. A managed proxy is configured with ClientTransportPlugin <method> exec <path> [options] as in "ClientTransportPlugin trebuchet exec /usr/libexec/trebuchet --managed" This example tells Tor to launch an external program to provide a socks proxy for 'trebuchet' connections. The Tor client only launches one instance of each external program with a given set of options, even if the same executable and options are listed for more than one method. If instead of a transport name, the torrc lists "*" for a managed proxy, tor uses that proxy for all transports that it supports. So "ClientTransportPlugin * exec /usr/libexec/tor/foobar" tells Tor that it should use the foobar plugin for everything that it supports. If two proxies support the same method, Tor should use whichever one is listed first. The same program can implement a managed or an external proxy: it just needs to take an argument saying which one to be. See "Managed proxy behavior" for more information on the managed proxy interface. Server behavior Server proxies are configured similarly to client proxies. When launching a proxy, the server must tell it what ORPort it has configured, and what address (if any) it can listen on. The server must tell the proxy which (if any) methods it should provide if it can; the proxy needs to tell the server which methods it is actually providing, and on what ports. When a client connects to the proxy, the proxy may need a way to tell the server some identifier for the client address. It does this in-band. As before, the server lists proxies in its torrc. These can be external proxies that run on their own, or managedproxies that Tor launches. An external server proxy is configured as ServerTransportPlugin method proxy address:port param=val.. as in ServerTransportPlugin trebuchet proxy 127.0.0.1:999 rocks=heavy The param=val pairs and the address are used to make the bridge configuration information that we'll tell users. A managed proxy is configured as ServerTransportPlugin method exec /path/to/binary [options] or ServerTransportPlugin * exec /path/to/binary [options] When possible, Tor should launch only one binary of each binary/option pair configured. So if the torrc contains ClientTransportPlugin foo exec /usr/bin/megaproxy --foo ClientTransportPlugin bar exec /usr/bin/megaproxy --bar ServerTransportPlugin * exec /usr/bin/megaproxy --foo then Tor will launch the megaproxy binary twice: once with the option --foo and once with the option --bar. Managed proxy interface When the Tor client launches a client proxy from the command line, it communicates via environment variables. At a minimum, it sets: {Client and server} HOME, PATH -- as you'd expect. "STATE_LOCATION" -- a directory where the proxy should store state if it wants to. This directory is not required to exist, but the proxy SHOULD be able to create it if it doesn't. The proxy SHOULD NOT store state elsewhere. "MANAGED_TRANSPORT_VER=1" -- To tell the proxy which versions of this configuration protocol Tor supports. Future versions will give a comma-separated list. Clients MUST accept comma-separated lists containing any version that they recognize, and MUST work correctly even if some of the versions they don't recognize are non-numeric. {Client only} "CLIENT_TRANSPORTS" -- a comma-separated list of which methods this client should enable, or * if all methods should be enabled. The proxy SHOULD ignore methods that it doesn't recognize. {Server only} "EXT_SERVER_PORT=addr:portnum" -- A port (probably on localhost) that speaks the extended server protocol. "ORPORT=addr:portnum" -- Our regular ORPort in a form suitable for local connections. "BINDADDR=addr" -- An address on which to listen for local connections. This might be the advertised address, or might be a local address that Tor will forward ports to. It MUST be an address that will work with bind(). "SERVER_TRANSPORTS=..." -- A comma-separated list of server methods that the proxy should support, or * The transport proxy replies by writing NL-terminated lines to stdout. The metaformat is Keyword OptArgs NL OptArgs = Args | Args = SP ArgChar | Args ArgChar ArgChar = Any character but NUL or NL Keyword = KeywordChar | Keyword KeywordChar KeyWordChar = All alphanumeric characters, dash, and underscore. Tor MUST ignore lines with keywords that it doesn't recognize. First, the proxy writes "VERSION 1" to say that it supports this protocol. It must either pick a version that Tor told it about, or pick no version at all, and say "ERROR no-version\n" and exit. The proxy should then open its ports. If running as a client proxy, it should not use fixed ports; instead it should autoselect ports to avoid conflicts. A client proxy should by default only listen on localhost for connections. A server proxy SHOULD try listen at a consistent port, though it SHOULD pick a different one if the port it last used is now allocated. A client or server proxy then should tell which methods it has made available and how. It does this by printing zero or more CMETHOD and SMETHOD lines to its stdout. These lines look like: CMETHOD methodname SOCKS4/SOCKS5 address:port [ARGS=arglist] \ [OPT-ARGS=arglist] as in CMETHOD trebuchet SOCKS5 127.0.0.1:19999 ARGS=rocks,height \ OPT-ARGS=tensile-strength The ARGS field lists mandatory parameters that must appear in every bridge line for this method. The OPT-ARGS field lists optional parameters. If no ARGS or OPT-ARGS field is provided, Tor should not check the parameters in bridge lines for this method. The proxy should print a single "CMETHODS DONE" line after it is finished telling Tor about the client methods it provides. If it tries to supply a client method but can't for some reason, it should say: CMETHOD-ERROR methodname "Message" A proxy should tell Tor about the server methods it is providing by printing zero or more SMETHOD lines. These lines look like: SMETHOD methodname address:port [Options] If there's an error setting up a configured server method, the proxy should say: SMETHOD-ERROR methodname "message" The 'address:port' part of an SMETHOD line is the address to put in the bridge line. The ARGS: part is a list of key-value pairs that the client needs to know. The Options part is a list of space-separated K:V flags that Tor should know about. Recognized options are: - FORWARD:1 If this option is set, and address:port is not a publicly accessible address, then the bridge needs to forward some other address:port to address:port via upnp-helper. - ARGS:k=v,k=v,k=v If this option is set, the K=V arguments are added to the extrainfo document. - DECLARE:K=V,... If this option is set, all the K=V options should be added as extension entries to the router descriptor. (See below) - USE-EXTPORT:1 If this option is set, the server plugin is using the extended server port. SMETHOD and CMETHOD lines may be interspersed. After the list SMETHOD line, the proxy says "SMETHODS DONE" The proxy SHOULD NOT tell Tor about a server or client method unless it is actually open and ready to use. Tor clients SHOULD NOT use any method from a client proxy or advertise any method from a server proxy UNLESS it is listed as a possible method for that proxy in torrc, and it is listed by the proxy as a method it supports. Proxies should respond to a single INT signal by closing their listener ports and not accepting any new connections, but keeping all connections open, then terminating when connections are all closed. Proxies should respond to a second INT signal by shutting down cleanly. The extended ORPort protocol. Server transports may need to connect to the bridge and pass additional information about client connections that the bridge would ordinarily receive from the kernel's TCP stack. To to this, they connect to the "extended server port" as given in SERVER_PORT, sent a short amount of information, wait for a response, and then send the user traffic on that port. The extended server port protocol is as follows: COMMAND [2 bytes, big-endian] BODYLEN [2 bytes, big-endian] BODY [Bodylen bytes] Commands sent from the transport to the server are: [0x0000] DONE: There is no more information to give. (body ignored) [0x0001] USERADDR: an address:port string that represents the user's address. If the transport doesn't actually do addresses, this shouldn't be sent. Replies sent from tor to the proxy are: [0x1001] OKAY: Send the user's traffic. (body ignored) [0x1002] DENY: Tor would prefer not to get more traffic from this address for a while. (body ignored) [We could also use an out-of-band signalling method to tell Tor about client addresses, but that's a historically error-prone way to go about annotating connections.] Advertising bridge methods: Bridges put the 'method' lines in their extra-info documents. method SP methodname SP address:port SP arglist NL The address:port parse are as returned from an SMETHOD line. The arglist is a K=V,... list as retuned in the ARGS part of the SMETHOD line. If the SMETHOD line includes a DECLARE: part, the routerinfo gets a new line: method-info SP methodname SP arglist NL Bridge authority behavior We need to specify a way to test different transport methods that bridges claim to support. We should test as many as possible. We should NOT require that we have a way to tra Bridgedb behavior: Bridgedb can, given a set of router descriptors and their corresponding extrainfo documents, generate a set of bridge lines for each descriptor. Bridgedb may want to avoid handing out methods that seem to get bridges blocked quickly. Implementation plan First, we should implement per-bridge socks settings (as described above in "manually configuring a client proxy for a bridge") and the extended-server-port mechanism. This will let bridges run transport proxies such that they can hand-generate bridge lines to give to clients for testing. Once that's done, we can improve usability a little bit by implementing external proxies. Once that's done, we can see if we need any managed proxies, or if the whole idea there is silly. If we do, the next most important part seems to be getting the client-side automatic part written. And once that's done, we can evaluate how much of the server side is easy for people to do and how much is hard. The "obfsproxy" obfuscating proxy is a likely candidate for an initial transport, as is Steven Murdoch's http thing or something similar. Notes on plugins to write: We should ship a couple of null plugin implementations in one or two popular, portable languages so that people get an idea of how to write the stuff. 1. We should have one that's just a proof of concept that does nothing but transfer bytes back and forth. 1. We should not do a rot13 one. 2. We should implement a basic proxy that does not transform the bytes at all 1. We should implement DNS or HTTP using other software (as goodesll did years ago with DNS) as an example of wrapping existing code into our plugin model. 2. The obfuscated-ssh superencipherment is pretty trivial and pretty useful. It makes the protocol stringwise unfingerprintable. 1. Nick needs to be told firmly not to bikeshed the obfuscated-ssh superencipherment too badly 1. Go ahead, bikeshed my day 1. If we do a raw-traffic proxy, openssh tunnels would be the logical choice. Appendix: recommendations for transports Be free/open-source software. Also, if you think your code might someday do so well at circumvention that it should be implemented inside Tor, it should use the same license as Tor. Use libraries that Tor already requires. (You can rely on openssl and libevent being present if current Tor is present.) Be portable: most Tor users are on Windows, and most Tor developers are not, so designing your code for just one of these platforms will make it either get a small userbase, or poor auditing. Think secure: if your code is in a C-like language, and it's hard to read it and become convinced it's safe, then it's probably not safe. Think small: we want to minimize the bytes that a Windows user needs to download for a transport client. Avoid security-through-obscurity if possible. Specify. Resist trivial fingerprinting: There should be no good string or regex to search for to distinguish your protocol from protocols permitted by censors. Imitate a real profile: There are many ways to implement most protocols -- and in many cases, most possible variants of a given protocol won't actually exist in the wild.

3 2

Logging [Enhancement]
by grarpamp 18 Mar '11

18 Mar '11

- An FYI, found while grokking, thanks. - These various log entries could really have their format normalized sometime. - If it's going to tell us why it's giving up in one case, it should do so in all cases. - The fingerprints should be printed instead of the ambigous router names. - Eliminate personal pronouns 'We'. - Remove 'Have'. - Shorten seconds to sec. - Change 'get a connection' to 'build a circuit' - Remove quoting, parenthesis, etc. Or create, and consistently use, parseable entries. We tried for 15 seconds to connect to '[scrubbed]' using exit 'Unnamed'. Retrying on a new circuit. Have tried resolving or connecting to address '[scrubbed]' at 3 different places. Giving up. Tried for 123 seconds to get a connection to [scrubbed]:80. Giving up. Tried for 120 seconds to get a connection to [scrubbed]:80. Giving up. (waiting for circuit) Tried for 120 seconds to get a connection to [scrubbed]:0. Giving up. (waiting for socks info)

2 1

DefenestraTor: Throwing out Windows in Tor
by Roger Dingledine 14 Mar '11

14 Mar '11

http://www.cacr.math.uwaterloo.ca/techreports/2011/cacr2011-06.pdf "Tor is the most widely used privacy enhancing technology for achieving online anonymity and resisting censorship. While conventional wisdom dictates that the level of anonymity offered by Tor increases as its user base grows, the most significant obstacle to Tor adoption contin- ues to be its slow performance. We seek to enhance Tor's performance by offering techniques to control congestion and improve flow control, thereby reducing unnecessary delays. To reduce congestion, we first evaluate small fixed-size circuit win- dows and a dynamic circuit window that adaptively resizes in response to perceived congestion. While these solutions improve web page response times and require modification only to exit routers, they generally offer poor flow control and slower downloads relative to Tor's current design. To improve flow control while reducing congestion, we implement N23, an ATM-style per-link algorithm that allows Tor routers to explicitly cap their queue lengths and signal congestion via back-pressure. Our results show that N23 offers better congestion and flow control, resulting in improved web page response times and faster page loads compared to Tor's current design and the other window-based approaches. We also argue that our proposals do not enable any new attacks on Tor users' privacy. " I haven't read it yet, but figured I'd share it with others here in the meantime. --Roger

2 1

Is this "fix" Ok for BSD malloc?
by Steve Snyder 09 Mar '11

09 Mar '11

When configured using the --enable-openbsd-malloc option Tor v0.2.1.30 will build without error on RHEL5, but then won't run. The cause of the runtime problem is that the symbol __libc_enable_secure is defined in OpenBSD_malloc_Linux.c but not actually built into the GLibC used in RHEL5. Can I "fix" this problem my simply removing the reference to __libc_enable_secure? Let me rephrase that. This does actually fix the problem, but I wonder if it creates a different problem, one which has not become apparent in 2 days of run time. So my question is: what functionality have I lost by removing the __libc_enable_secure test? Thanks. -------------------------- --- tor-0.2.1.30/src/common/OpenBSD_malloc_Linux.c.original 2009-05-18 19:47:00.000000000 -0400 +++ tor-0.2.1.30/src/common/OpenBSD_malloc_Linux.c 2011-03-07 11:19:36.000000000 -0500 @@ -77,11 +77,11 @@ static int align = 0; static size_t g_alignment = 0; -extern int __libc_enable_secure; +// extern int __libc_enable_secure; static int issetugid(void) { - if (__libc_enable_secure) return 1; +// if (__libc_enable_secure) return 1; if (getuid() != geteuid()) return 1; if (getgid() != getegid()) return 1; return 0;

1 0

Re: [tor-dev] xxx-triangleboy-transport.txt (+comments on pluggable-transport)
by Mike Perry 03 Mar '11

03 Mar '11

Thus spake Nick Mathewson (nickm(a)freehaven.net) > On Tue, Feb 1, 2011 at 12:08 AM, Mike Perry <mikeperry(a)fscked.org> wrote: > [...] > > > > Appendix: List of key xxx-pluggable-transport.txt shortcomings > > > > 1. This pluggable transport does not need any intelligence or process > > launching on the Client side, aside from a way to tell Tor not to be > > so pedantic about ensuring identity key and IP address consistency. > > What pedanticism exactly do you mean? Matching the IP addr in the > netinfo cell, or something else? I actually don't know the full list there. I think the last time we tried to enumerate this, we decided that "try it and see what actually breaks" was a more efficient approach. > > 2. The relay side needs to be able to detect if it has both the > > permissions and the network ability to send spoofed packets. It needs > > to communicate this fact with the Relay Plugin by responding with the > > appropriate extrainfo lines, or with "METHODS: NONE" to indicate > > error. This relay-side handshake should be specified in the > > pluggable-transport spec. > > To be clear, you mean that the plugin checks whether it can spoof (and > tell Tor "I do nothing!" if it can't), or that Tor needs to find out > whether it can spoof and tell the plugin that whether it can spoof or > not. The first sounds reasonable. The second doesn't so much: most > Tors won't want to try to spoof, so building the checks into Tor > wouldn't make sense to me, unless I'm missing something. Yes, the first is what we want here. > > 3. The relay plugin side needs some way to communicate EXTRAINFO lines > > to be added to its extrainfo descriptor. In this proposal, we use the > > SMETHOD reply to do this. > > Needs spec, not just an example. Ok. Well, does the example I gave look like an instance of what we want to specify as the behavior? > > 4. Is extrainfo really the best place to keep this information? > > Shouldn't it just be in the relay/bridge descriptor? Putting it in > > extrainfo requires our TransportBridges to enable the wasteful > > DownloadExtraInfo=1 torrc setting, which will consume more scarce > > resources and RAM on what will probably be cheap routers with 32-64M > > of RAM. > > Hm. Relay/bridge descriptor should be okay, I guess. I'm not sure > why the TransportBridges are getting those anyway, though: I see why > the client needs it, but not what the transportbridge needs it. The transport bridge should avoid being an open proxy. I suppose it can only properly carry full TCP streams to an endpoint, but without restricting who those endpoints are, it becomes a reflector open to abuse such as portscans. > Also, why are transportbridges even running tor? They don't seem to > be doing anything particularly tor-like. Mostly for the consensus, for reasons above. Not strictly required, if we don't mind being an arbitrary SYN proxy. > > 5. How would we go about chaining an actual obfuscation mechanism with > > this transport? Would we just create new and separate transport called > > TriangleBoyOverHTTP, for example, or is there a better way to chain > > different mechanisms? > > On the client side, socks-over-socks-over-socks is not actually a > terrible way of doing things there. We'll need to distinguish between > which transports are chainable and which aren't, though[*], and maybe > revise the design to make sure there's a way to tell to tell tor to do > said chaining. > > On the server side, you need some way to distinguish which processing > method to do for incoming data. Separate ports seems ok there. We > still need to figure out the server side. > > [*] generally, obfuscation is chainable but transport isn't. This is not a bad generalization. Do we want to make this distinction (between obfuscation vs transport) elsewhere in the spec, I wonder? -- Mike Perry Mad Computer Scientist fscked.org evil labs

2 1

Proposal status, with suggestions for fun stuff to do
by Nick Mathewson 03 Mar '11

03 Mar '11

Hi, everybody! This is a long mail, but if you're interested in Tor development, it could be a good one to read. As promised at the dev meeting, I've spent a while going over all the current Tor proposals in state "Accepted", "Open", and "Draft", plus the documents in proposals/ideas. With any luck, this will help us move design discussion forward, and point out some neglected designs/discussions that people could pick up to specify/improve/implement. I've also summarized the stuff in proposals/ideas: I think we should be more aggressive at putting proposal drafts in "Proposals", and in making a place for design overviews and/or proposal-proposal documents that are not themselves proposals. Of course, if you're looking for a programming project, this isn't the only place to look: you can also head over to the tracker, and look at the tickets, particularly those with keyword "easy". If this is useful to people, I'm hoping to do this once every month or two. (A reminder: We've moved the tor specifications and proposals to their own repository. You can get the latest versions from https://gitweb.torproject.org/torspec.git . For an overview on how the proposal process is supposed to work, see proposal 001, at https://gitweb.torproject.org/torspec.git/HEAD:/proposals/001-process.txt .) ===== ACCEPTED: 110 Avoiding infinite length circuits This proposal solves a potential DoS attack. This has been nearly completely implemented for a while. All we need to do is verify that there are no clients that do not send RELAY_EARLY cells correctly, and if so, turn on the code that drops EXTEND cells not sent via RELAY_EARLY. The hard part here is double-checking that all the clients that send EXTEND commands inside non-RELAY_EARLY cells are really and truly obsolete. 117 IPv6 exits [for 0.2.1.x] This proposal describes how to transmit IPv6 traffic over Tor. It needs updating to work properly with microdescriptors; it also has some open questions about DNS. I think this should move to 'needs-revision'. 118 Advertising multiple ORPorts at once This proposal describes how an OR can advertise more than one address and OR port at a time. It needs to be updated to work with microdescriptors, and to explain how much information can be transmitted in the consensus and how (the original proposal was written before consensus directories were really figured out). I think this should move to 'needs-revision' 140 Provide diffs between consensuses This proposal describes a way to transmit less directory traffic by sending only differences between consensuses, rather than the consensuses themselves. It is mainly languishing for lack of an appropriately licensed, well-written, very small, pure-C implementation of the "diff" and "patch" algorithms. (The good diffs seem to be GPL (which we can't use without changing Tor's license), or spaghetti code, or not easily usable as a library, or not written in C, or very large, or some combination of those.) ACTION: Move to 'needs-revision', or find a suitable library 147 Eliminate the need for v2 directories in generating v3 directories This proposal explains a way that we can phase out the vestigial use of v2 directory documents in keeping authorities well-informed enough to generating the v3 consensus. It's still correct; somebody should implement it before the v2 directory code rots any further. 157 Make certificate downloads specific This proposal added cross-certification and signing-key-specific download URLs for directory authority certificates. It is IIRC mostly implemented; there are just some SHOULDs that we should turn into MUSTS if all sufficiently old authority certificates are now obsolete. 166 Including Network Statistics in Extra-Info Documents I think this one is implemented (or mostly implemented) by Karsten in 0.2.2.x. If so, we just need to make sure it is merged into dir-spec, and closed. Karsten, is there something left here? (NOTE TO SELF: I should remember to ask Karsten if he doesn't notice the above.) 172 GETINFO controller option for circuit information 173 GETINFO Option Expansion These would help controllers (particularly arm) provide more useful information about a running Tor process. They're accepted and some parts of 173 are even implemented: somebody just needs to implement the rest. 174 Optimistic Data for Tor: Server Side This proposal would make streams start faster by allowing clients to send DATA cells immediately, without waiting for a CONNECTED cell. There's a patch under review as trac ticket #1795: it has some issues that need to get fixed before we can merge it. I really want to get it fixed up soon, and merged in the next week or two. ===== OPEN: 143 Improvements of Distributed Storage for Tor Hidden Service Descriptors Here's a proposal from Karsten about making the hidden service DHT more reliable and secure to use. It could use more discussion and analysis. 145 Separate "suitable as a guard" from "suitable as a new guard" Currently, the Guard flag means both "You can use this node as a guard if you need to pick a new guard" and "If this node is currently your guard, you can keep using it as a guard." This proposal tries to separate these two concepts, so that clients can stop picking a router once it is full of existing clients using it as a guard, but the clients currently on it won't all drop it. It's not clear whether this has anonymity issues, and it's not clear whether the imagined performance gains are actually worthwhile. 146 Add new flag to reflect losing-term stability From time to time we get the idea of having clients ship with a reasonably recent consensus (or a list of directory mirrors), so instead of bootstrapping from one of the authorities, they can bootstrap from a regular directory cache. The problem here is that by the time the client is run, most of the directory mirrors will be down or will have changed their IP. This proposal tries to address that. It needs analysis based on behavior of actual routers on the network to see whether it could work, and what parameters might work. 156 Tracking blocked ports on the client side This proposal provides a way for clients to learn which ports they are (and aren't) able to connect to, and connect to the ones that work. It comes with a patch, too. It also lets routers track ports that _they_ can't connect to. I'm a little unconvinced that : most clients that have some ports blocked will need bridges, not just restriction to a smaller set of ports. This could be good behind restrictive firewalls, though. The router-side part is a little iffy: routers that can't connect to each other violate one of our network topology assumptions, and even if we do want to track failed router->router connections, the routers need to be sure that they aren't fooled into trying to connect repeatedly to a series of nonexistent addresses in an attempt to make them believe that (say) they can't reach port 443. This one is a paradigmatic "open" proposal: it needs more discussion. The patch probably also needs to be ported to 0.2.3.x; it touches some code that has changed. 158 Clients download consensus + microdescriptors This proposal provides a way for clients to download authority-provided summaries of router descriptors instead of the descriptors themselves; I'm working on this one; it's going in 0.2.3.x. It needs some cleanup, but it's basically right. (NOTE TO SELF: Clean this up and move it to "Accepted!") 159 Exit Scanning This is an overview of SoaT, with some ideas for how to integrate it into Tor. Mike, is this implemented? Done? Superseded? Still open? (NOTE TO SELF: I should remember to ask Mike if he doesn't notice the above.) 162 Publish the consensus in multiple flavors This proposal describes a way for authorities to migrate from one format of consensus to another without bloating the consensus forever. It is mostly implemented in 0.2.3.x, though some parts of it need to get work. I should clean it up too. (NOTE TO SELF: Clean this up and move it to "Accepted!") 163 Detecting whether a connection comes from a client This one describes a number of ways to distinguish clients from non-clients when rejecting single-hop connections. Some are implemented; others are rejected; others are still discussable. Somebody should figure out what happened here, and reopen discussion as interested. 164 Reporting the status of server votes This proposal explains a way for authorities to provide a slightly more verbose document that relay operators can use to diagnose reasons that their router was or was not listed in the consensus. These documents would be like slightly more verbose versions of the authorities' votes, and would explain *why* the authority voted as it did. It wouldn't be too hard to implement, and would be a fine project for somebody who wants to get to know the directory code. 165 Easy migration for voting authority sets This is a design for how to change the set of authorities without having a flag day where the authority operators all reconfigure their authorities at once. It needs more discussion. One difficulty here is that we aren't talking much about changing the set of authorities, but that may be a chicken-and-egg issue, since changing the set is so onerous. If anybody is interested, it would be great to move the discussion ahead here. 168 Reduce default circuit window This proposal reduces the default window for circuit sendme cells. I think it's implemented, isn't it? If so, we should make sure that tor-spec.txt is updated and close it. (NOTE TO SELF: Ask Roger; This should probably be called "Finished", or "Closed" if we have updated the spec.) 171 Separate streams across circuits by connection metadata This proposal describes a way to prevent cross-application linking of different anonymized sessions by isolating different kinds of streams on different circuits based on their characteristics. I want this one in 0.2.3.x; it would improve client anonymity. A patch would be great; there are efficiency issues to consider, though. ACTION: This should be "accepted", I believe, unless somebody has lingering objections. 176 Proposed version-3 link handshake for Tor Here's my proposal for how to improve our TLS handshake to eliminate renegotiation, become less fingerprintable, and generally make the world a better place. It needs more review by actual crypto people, but I would like to get it into 0.2.3.x. 177 Abstaining from votes on individual flags Here's my proposal for letting authorities have opinions about some (flag,router) combinations without voting on whether _every_ router should have that flag. It's simple, and I think it's basically right. With more discussion and review, somebody could/should build it for 0.2.3.x, I think. ===== DRAFT: 127 Relaying dirport requests to Tor download site / website The idea here was to make it easier to fetch and learn about Tor by making it easy for relays to automatically act as proxies to the Tor website. It needs more discussion, and there are some significant details to work out. It's not at all clear whether this is actually a good idea or not. 132 A Tor Web Service For Verifying Correct Browser Configuration This proposal was meant to give users a way to see if their browser and privoxy (yes, it was a while ago) are correctly configured by running a local webserver on 127.0.0.1. I'm not sure the status here. 133 Incorporate Unreachable ORs into the Tor Network This proposal had an idea for letting ORs that can only make outgoing connections still relay data usefully in the network. It's something we should keep in mind, and it's a pretty neat idea, but it radically changes the network topology. Anybody who wants to analyze new network topologies should definitely have a look. 141 Download server descriptors on demand The idea of this proposal was for clients to only download the consensus, and later ask nodes for their own server descriptors while building the circuit through them. It would make each circuit more time-consuming to build, but make bootstrapping much cheaper. A microdescriptor-based version of this would be even better, and microdescriptors would solve a lot of the problem that this was meant to resolve. It still is not wholly superseded by the microdescriptor system, though. 144 Increase the diversity of circuits by detecting nodes belonging the same provider This is a version of the good idea, "Let's do routing in a way that tries to keep from routing traffic through the same provider too much!" There are complex issues here that the proposal doesn't completely address, but I think it might be a fine idea for somebody to see how much more we know now than we did in 2008, particularly in light of the relevant paper(s) by Matt Edmann and Paul Syverson. 149 Using data from NETINFO cells NETINFO cells currently contain time and address information that could be used to estimate skew and provide yet another way to learn our IP address. This proposal starts to discuss ways that we could try to use that information safely, but doesn't really get anywhere. See trac issues #2628 for some discussion about this stuff. It would be neat if we had a working design here, though I suspect that with the slow and grinding death of older operating systems, systems with radically skewed clocks may become a thing of the past. 170 Configuration options regarding circuit building This is Sebastian's take on how the *Nodes options should work. Roger preferred a more minimal approach that may or may not be right; see the discussion starting at https://trac.torproject.org/projects/tor/ticket/1090#comment:21 for his approach. I'm not going to call this "superseded" or "rejected" until my 1090 code is done, though. :( 175 Automatically promoting Tor clients to nodes Here's Steven's proposal for adding a mode between "client mode" and "relay mode" for "self-test to see if you would be a good relay, and if so become one." It didn't get enough attention when it was posted to the list; more people should review it. 178 Require majority of authorities to vote for consensus parameters This is a proposal for making it a little harder for a small number of authorities to influence the value of a consensus parameter. It needs more discussion; Sebastian has a draft implementation in his "safer_params" branch. (NOTE TO SELF: I think this could fairly become "open"; I should ask Sebastian.) ===== NEEDS-REVISION: 131 Help users to verify they are using Tor Here's a proposal for making a torcheck-like website more reliable. If anybody wants to pick it up (especially somebody working on torcheck) and see whether it should be reopened or rejected, that would be a fine thing. ====== "IDEAS": xxx-bwrate-algs.txt Here's a note about better round-robin calculations for rate limiting. Mike wrote it; I don't understand what's going on with it. xxx-choosing-crypto-in-tor-protocol.txt Here's the start of a considerations document that Marian was writing about how to pick new crypto algorithms to migrate to. xxx-controllers-intercept-extends.txt Here's an old idea from Geoff Goodell about letting controllers intercept EXTEND commands. It is partially superseded by Damian's 172, though there is an additional feature that doesn't make sense for the Tor network, but might make sense for experimental stuff like Geoff was working on. xxx-crypto-migration.txt Here's the document I wrote in December about which parts of our crypto there are to migrate, and what might be involved. This isn't a draft or pre-draft of a design proposal; it is more of a new category of "survey of stuff we need to design and think about", or "proposal for proposals" or something. xxx-crypto-requirements.txt Here's Robert Ransom's draft for what crypto properties, exactly, we're trying to get out of our circuit crypto. It is not a proposal draft or pre-draft: it is again a "design considerations" document, and one worth reading. xxx-draft-spec-for-TLS-normalization.txt Here's Jacob's proposal for certificate normalization. It should get renamed, given a proposal number, and called "Open" or "Draft" depending on how much the details are likely to change. xxx-encrypted-services.txt This is a start-of-a-proposal of Rogers that somebody should pick up and finish some time: The idea is to use the hidden service mechanism to provide a secure naming and encrypted connection facility for hosting sites that do not actually want anonymity themselves. There's been more interest in this topic lately. It might also turn into "exit enclaves done right". xxx-exit-scanning-outline.txt Looks like this was superseded by 159? xxx-geoip-survey-plan.txt Here's an old document I wrote a while ago about tracking usage by country. Probably it should go into the metrics documentation somewhere (if we do this), or get thrown into "old" (if we won't do this), or updated (if we might someday do this). xxx-grand-scaling-plan.txt Here are some notes on scaling that Roger wrote in 2008. There might be some smart ideas here! Have a look some time. xxx-hide-platform.txt Here's a proposal pre-draft that says we should normalize the platform string. Somebody could turn this into a proposal pretty easily. xxx-ipv6-plan.txt Here's a survey of what we need to do for IPv6 support that I started writing. It's not a proposal; it's a survey of proposal and implementation statuses. xxx-pluggable-transport.txt Here's the thing Jacob and I have been working on for pluggable obfuscation techniques. It should get turned into "Draft" status asap, if not "Open". We should finish writing the missing sections, like, yesterday. xxx-port-knocking.txt Here's a pre-proposal from Jacob about using port knocking to make bridges harder to fingerprint. This could be a good idea for somebody to do in terms of the pluggable transport spec. xxx-rate-limit-exits.txt Here are some notes of Roger's from 2008 claiming that we should rate-limit stream creation at exit nodes. It could help avoid port-scans if we do it right, but we would need to be exceedingly careful not to disrupt useful traffic. xxx-using-spdy.txt Here's a document from Steven about opportunistic use of SPDY over Tor. This should be a "Draft" proposal IMO. xxx-what-uses-sha1.txt Here's the beginnings of a survey of where Tor uses SHA1, with an eye to stopping. This really isn't a design proposal; it might fall into a new category of "issue survey" or "information" or something.

3 2

SSL Observatory Observations
by Tim Wilde 25 Feb '11

25 Feb '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 G'morning all! Last weekend's hackfest inspired me to attempt to run some numbers on the EFF SSL Observatory data[1], in particular looking at two things: the commonality of the "Internet Widgits Pty" organization string (a default in OpenSSL's CSR generator) in self-signed certificate subjects, and, further to that, the most commonly seen self-signed certificates in the data. Using the CSV data from the observatory, I found that out of 5,618,558 total certs in the all-certs file, only 8,891 contained subjects that matched the regex /Internet Widg[ie]ts Pty/ (a quick search observed both spellings; my OpenSSL used Widgits, FWIW. I further recorded the subjects of all certs where subject == issuer, and counted each, with the following results for the top 10: ' C=--, ST=SomeState, L=SomeCity, O=SomeOrganization, OU=SomeOrganizationalUnit, CN=localhost.localdomain/emailAddress=root(a)localhost.localdomain': 154867 ' C=US, ST=California, L=Sunnyvale, O=HTTPS Management Certificate for SonicWALL (self-signed), OU=HTTPS Management Certificate for SonicWALL (self-signed), CN=192.168.168.168': 111453 ' C=US, ST=Virginia, L=Herndon, O=Parallels, OU=Plesk, CN=plesk/emailAddress=info(a)plesk.com': 82689 ' CN=Fortinet, O=Fortinet Ltd.': 54975 ' C=US, ST=Virginia, L=Herndon, O=SWsoft, Inc., OU=Plesk, CN=plesk/emailAddress=info(a)plesk.com': 53184 ' C=USA, ST=California, L=Sunnyvale, O=HTTPS Management Certificate for SonicWALL (self-signed), OU=HTTPS Management Certificate for SonicWALL (self-signed), CN=192.168.168.168': 22987 ' CN=SpeedTouch 605, O=THOMSON, OU=DSL Internet Gateway Device': 16790 ' C=US, ST=Someprovince, L=Sometown, O=none, OU=none, CN=localhost/emailAddress=webaster@localhost': 16638 ' CN=SpeedTouch 5x6, O=THOMSON, OU=DSL Internet Gateway Device': 15286 ' C=US, ST=CA, L=Sunnyvale, O=SonicWALL, Inc., OU=SSL-VPN, CN=192.168.200.1': 11760 (Apologies for the lousy wrapping; initial spaces are present in the source data.) The numbers after the colons are, as you may guess, the total number of self-signed certificates with that subject. It may be instructive to dig into the netblocks where these subjects occur and attempt to determine the context in which they occur. For the purposes of making Tor appear unblockable due to collateral damage, my strong suspicion is that these will not be all that helpful, as they are quite likely (and in some cases, clearly) used by CPE devices and the like for internal management, and thus won't really be likely to cause much difficulty for the censors if blocked. My personal thinking is that the methodology in Jake's TLS normalization proposal[2] makes a lot of sense. Perhaps one modification, per the discussion at the hackfest, would be to stick to making the server's presented certificates self-signed, as that is probably more normal than having random unknown issuers, and less provably false than faking a known issuer like GoDaddy. Even ignoring the question of whether or not a censor may be willing to sustain the collateral damage of blocking one of the common self-signed subjects above, by using a commonName that varies per bridge/relay and other certificate fields that also vary widely (and in fact may be randomized to some extent), we don't present any blockable static values in our certificate subject or issuer and can focus on other fun stuff in the certs. :) Hopefully Jake hasn't already made this all a moot point while I was in the other room working with Andrew on Saturday and I just haven't heard about it yet. :) Further thoughts/discussions/flames welcome (but only if the flames come from Mike). Regards, Tim [1] https://www.eff.org/observatory [2] https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/ideas/x… - -- Tim Wilde, Senior Software Engineer, Team Cymru, Inc. twilde(a)cymru.com | +1-630-230-5433 | http://www.team-cymru.org/ -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAk1idPgACgkQluRbRini9tgmZACfSBNRhJ+ozVgIyDIdA4nWbOpI 25oAnRbbXvMkA+ttGkStfB2VFkcal1jd =vU04 -----END PGP SIGNATURE-----

2 1

Re: [tor-dev] Publishing sanitized bridge pool assignments
by Karsten Loesing 25 Feb '11

25 Feb '11

On Thu, Feb 10, 2011 at 12:00:40PM +0100, Karsten Loesing wrote: > So, I implemented the new sanitizing algorithm that replaces bridge IPs > with 10.x.x.x addresses derived from the original bridge IPs. I also > sanitized the descriptors from November 2008 and performed an early > analysis on the results. > > The last remaining step is to sanitize our archives of bridge descriptors > once again and make the sanitized data available. I don't expect to have > new tarballs before next Tuesday. The new tarballs are available here: https://metrics.torproject.org/data.html#bridgedesc > Before publishing the new tarballs, I'd like to invite people to review > the new sanitizing process: > > - A specification-like description of the sanitizing process is in > Section 3 of https://metrics.torproject.org/papers/data-2011-02-10.pdf > > - The early analysis results of bridges changing their IP addresses are > here: https://trac.torproject.org/projects/tor/ticket/2435#comment:3 > > - The tarball with sanitized bridge descriptors from November 2008 is > here (6.2M): > http://freehaven.net/~karsten/volatile/bridge-descriptors-hashed-ips-2008-1… > > Any questions, comments, and concerns are highly appreciated! Feedback still much appreciated! Best, Karsten

1 0