- tor-dev - lists.torproject.org

Request for website commit review
by Robert Ransom 14 Apr '11

14 Apr '11

Please review Tor Subversion commits 24619, 24620, and 24621, then update the website. Regarding r24619: This change should be self-explanatory. Regarding r24620: I confirmed Robert Hogan's OpenPGP fingerprint on <https://trac.torproject.org/projects/tor/ticket/2883>. Regarding r24621: I obtained Tomás Touceda's OpenPGP fingerprint from his Tor LDAP account request on <https://trac.torproject.org/projects/tor/ticket/2621>, and I verified that the key with that fingerprint signed the Git tag vidalia-0.2.12 in <git://git.torproject.org/vidalia.git> and the release tarball at <https://www.torproject.org/dist/vidalia/vidalia-0.2.12.tar.gz>. I believe that these changes are appropriate, but they need to be reviewed before they are put onto The Tor Project's website. Robert Ransom

1 0

Redirecting network traffic to tor
by rp 08 Apr '11

08 Apr '11

Hi, I've become interested in one problem stated on tor web pages which deals with redirecting all traffic to tor (in windows). Redirecting traffic to tor seems to seek the goal of preventing traffic leaks. Openvpn does it's tunneling by rerouting all traffic to virtual network device and by changing dns resolver to the one on the remote tunnel side. Using openvpn's code, there is one possible way to prevent trafic leaks - by initializing fake route for all network traffic except for tor's, so only tor traffic can leave to the network. To try it, I have written simple .so library, that is linked to openvpn and uses openvpn code to manipulate routes (and possibly dns settings). Library gets loaded with tor using LD_PRELOAD and hooks connect and close calls on sockets. It works in the following way: 1) default route is overriden with fake one - now no traffic except dns can leave (assuming dns server is default gateway) 2) with each tor's connect, a route is added so traffic is routed through old route and not fake one, on close call the route is deleted - now only tor traffic and dns can leave 3) dns is set up to use tor's resolver - now only tor traffic will leave (assuming dns is properly redirected) 4) local connections fe. from web browser to local privoxy and from local privoxy to tor socks proxy are working, so this could be in principle used to help user with web browser gain more safety. The drawback is that to manipulate routes, one needs admin privileges. But this shouldn't be a problem - LD_PRELOAD way of doing things was the fastest to try this out. Although this is rather crude, it should prevent traffic leaks. What do you think about this? Could it be to some use? I have already tested all points except number 3), which would need more work, so before it I have to ask for your opinions - whether this is a waste of time or not. Please let me know. Roman PS: sorry for the noise with messages to various lists - i've got somewhat excited that there is a possibility for me to help with something useful..

2 1

Nick+Roger IRC chat time for developers, next Tuesday+Wednesday, 2-6pm eastern
by Nick Mathewson 08 Apr '11

08 Apr '11

Hi, all! Frequently, developers (including Nick and Roger) block on having both Nick and Roger online at the same time. We're trying to solve this by scheduling time to make sure that we're online and available to work with others on stuff that's blocking on is. We'll be on the development IRC channel (#tor-dev on OFTC) on March 8 and 9, from 1400 to 1800 EST (1900-2300 UTC, if I can add right). If you're doing something developmenty that's blocking on one or both of us, please stop by and have a chat. peace, -- Nick

2 1

Improved circuit-setup protocol [was: Re: Designing and implementing improved circuit-setup protocol [was: GSoC 2011]]
by Nick Mathewson 08 Apr '11

08 Apr '11

On Thu, Mar 31, 2011 at 5:52 AM, Robert Ransom <rransom.8774(a)gmail.com> wrote: Hi! I'm going to wait on a full review of your create/extend proposal till it's done, but I though I could potentially answer some questions and offer some comments: 1) I think CREATE cells need to get a field asking for a specific way of handling RELAY cells. If we ever update the relay crypto (that is, the stuff described in 5.5 and 6.1 today) to do something other than AES128_CTR for the encryption, SHA1 for the digest, we'll need a way to ask for it. I suppose that we could do that via adding a new handshake type for a new desired link protocol, but it seems like the two could be more or less orthogonal. 2) I don't get the rationale behind the variable-length type names. Elsewhere in our protocol we'd just use a 1- or 2-byte type field. Do we win a lot by letting these be longer? 3) By convention, everything in our protocol is in "network" (big-endian) byte-order. 4) There needs to be an authenticated way for the client to know which handshake types and link specifier types a given node supports. This could be as simple as a series of statements like "Tor versions xyz and later will support handshake type Y", or as complex as advertising them in router descriptors. 5) I like the idea of separating the link specifiers from the handshake type in EXTENDED cells. 6) I also think it'd be smart to figure out the actual link specifier and handshake types that we want to move to before we decide that we are confident that this format is right. Those can be a different proposal, though. 7) Here's a first cut of what I think might go in a link specifier format: * V4Address -- an ipv4 address, set to 0 if there is no IPv4 address for the node [4 bytes] * V4ORPort [2 bytes] * V6Address -- an ipv6 address, set to 0 if there is no IPv6 address for the node [16 bytes] * V6ORPort [2 bytes] * SHA256 of RSA1024 identity key [32 bytes] When we decide what longer identity keys should look like, we can add a new link specifier format that supports those. 8) This is totally back-of-the-envelope stuff, but it might be a good starting point for crypto discussion. Here's a first cut of what I think might go in an improved RSA handshake: * First 8 bytes of the SHA256 hash of the onion key [8 bytes] (This is here so that onion key rotation can work without having to sometimes try the wrong onion key incorrectly.) * PK-encrypted: * Padding [PK_PAD_LEN bytes] * SHA256 hash of all remaining fields. [32 bytes] * Symmetric key seed [16 bytes] * The first part of g^x. [as much will fit in the PK-encrypted portion] * Symmetrically encrypted: * The rest of g^x. * 0 bytes for padding. Here's a first cut of what I think might go in a hypothetical diffie-hellman based handshake, for use with something like Curve25519. (I'm using g^x and g^y notation here as if this were diffie-hellman in Z_p, since I don't yet trust myself to write ECC stuff correctly. I'm assuming that the node's public onion key is g^x.) * SHA256 of all remaining fields. [32 bytes] * First 8 bytes of the SHA256 hash of the onion key (8 bytes) * g^y1 [DH_LEN bytes] * Encrypted using a symmetric key based on g^(x*y1): * g^y2 [DH_LEN bytes] * 0 bytes for padding In both cases, we'll want a new key derivation function. yrs, -- Nick

4 7

[Patch] src/test.c
by Gisle Vanem 08 Apr '11

08 Apr '11

I don't think it's a good idea to use hard-coded paths. Even in a test source-file. Easy patch: --- Git-latest\src\test\test.c Wed Mar 30 11:58:28 2011 +++ src\test\test.c Thu Mar 31 14:06:14 2011 @@ -86,7 +86,7 @@ #ifdef MS_WINDOWS // XXXX tor_snprintf(temp_dir, sizeof(temp_dir), - "c:\\windows\\temp\\tor_test_%d", (int)getpid()); + "%s\\tor_test_%d", getenv("TEMP"), (int)getpid()); r = mkdir(temp_dir); #else tor_snprintf(temp_dir, sizeof(temp_dir), "/tmp/tor_test_%d", (int) getpid()); ---------- --gv

3 5

Embedding Parrot in Tor as GSoC Project
by Jonathan "Duke" Leto 08 Apr '11

08 Apr '11

Howdy, Parrot recently added a GSoC proposal idea to embed Parrot into Tor. It would be great to get the feedback from Tor developers. Also, this could be under the Parrot or the Tor GSoC project, whichever makes the most sense. The proposal idea could use more detail, but the high-level view that I imagine is: 1) Allow Parrot to talk to libtor (this will use NCI - Native Call Interface) via PIR 2) Ability to create Parrot interpreter objects from within Tor via C 3) Write glue code for a High Level Language (HLL) to talk to libtor There are various HLL's to choose from: Lua, TCL, Perl 6, Winxed, NQP and others. Duke Thanks for your help, Duke [0] http://trac.parrot.org/parrot/wiki/GSoc2011 -- Jonathan "Duke" Leto jonathan(a)leto.net http://leto.net

4 6

GSoC - Torbutton for Thunderbird
by Sukhbir Singh 05 Apr '11

05 Apr '11

Hi! I am interested in participating in Google's Summer of Code this year and implementing the Torbutton equivalent for Thunderbird [0]. I have been discussing the project with Mike Perry and we feel that it is possible to implement this. The first step was removing the hostname Thunderbird puts in the outgoing message header, which we easily fixed. That gave us the initial motivation to pursue this further. There are a number of issue that still remain, however we feel that those can be easily worked out. We have put all probable solutions to Torbutton requirements from [1] in the attachment with this mail and are looking for your suggestions/ comments about this. We have not submitted an application for this to Google yet; I intend to do that after gauging the response towards this proposal from the mailing list. Let's hope that we can make this a success, -- Thanks, Sukhbir Singh [0] - https://www.torproject.org/getinvolved/volunteer.html.en#torbuttonForThunde… [1] - https://www.torproject.org/torbutton/en/design/#requirements Attachment - Plain text

2 1

Torbutton for Thunderbird
by Sukhbir Singh 04 Apr '11

04 Apr '11

Hi! I am interested in participating in Google's Summer of Code this year and implementing the Torbutton equivalent for Thunderbird [0]. I have been discussing the project with Mike Perry and we feel that it is possible to implement this. The first step was removing the hostname Thunderbird puts in the outgoing message header, which we easily fixed. That gave us the initial motivation to pursue this further. There are a number of issue that still remain, however we feel that those can be easily worked out. We have put all probable solutions to Torbutton requirements from [1] in the attachment with this mail and are looking for your suggestions/ comments about this. We have not submitted an application for this to Google yet; I intend to do that after gauging the response towards this proposal from the mailing list. Let's hope that we can make this a success, -- Thanks, Sukhbir Singh [0] - https://www.torproject.org/getinvolved/volunteer.html.en#torbuttonForThunde… [1] - https://www.torproject.org/torbutton/en/design/#requirements Attachment - Plain text

1 0

First draft: Thandy Package Format spec
by Erinn Clark 31 Mar '11

31 Mar '11

Also available here: https://gitweb.torproject.org/erinn/thandy.git/blob/refs/heads/thp-draft-sp… Title: Specification for a TBB/Thandy package format. Status: Draft Authors: nickm, erinn Started-On: 9 Feb 2011 Finished-On: Introduction On some platforms, in some environments, Thandy can use existing platform-provided mechanisms for packages. But for the Tor Browser Bundle, and for Windows, we can't use built-in packaging systems (because they put data in a database or registry, because they require root). Requirements, Goals: Thandy has these requirements for a packaging system: - It needs to be able to install/upgrade packages. - It needs to be able to check which version of a package is installed For use in TBB, we we need a few more features: - It needs to be able to remove packages - It needs to be able to leave cofiguration files alone when upgrading To use this right, Thandy needs these features: - download packages as-needed in the background - report when packages are ready to install - Have a way to upgrade the TBB as it restarts To avoid big problems, package installation needs these features: - Idempotence - Validatability - Spending as little time as possible in non-functional states. (We can't get true atomicity on some OSs, but we should get as much as reasonable as we can.) Nongoals: This is a packagesystem for TBB and similar things for use with Thandy. It is not a more general system, a replacement for rpm/deb, an all-purpose software distribution mechanism, or a dessert topping. This is not the spec for Thandy. This is not a spec for interfaces to Thandy. It lists some interfaces that are necessary but it doesn't explain how they work. This is not the spec for any modes of operation for thandy, interfaces to thandy, or a thandy net installer. We need specs for those, of course. Though this document has suggestions on how to make good packages and install them well, there may be additional requirements for a high-quality installer not listed here. We're trying to design this system to _support_ being the most solid tool it can be, but we're not trying to figure out every possible implementation detail here. Mode of operation: While TBB is running, Vidalia should periodically launch Thandy, telling it, "Fetch packages as needed" or "Tell me if you could fetch packages." While TBB is running, Vidalia should periodically launch Thandy, asking it, "Are there packages downloaded and ready to install?" If so, it should tell the user. When TBB first starts, it should start a launcher program that asks Thandy, "Are there packages downloaded and ready to install?" If so, it should tell Thandy to install them. To make Thandy and the launcher self-upgrade, here's a two-step process: the install process installs new stuff into a new directory to the side of the old directory, and then launches a new "replacer" program to move the new stuff over the old stuff. The replacer, when it's done, re-launches the launcher. "It's not pretty, and you can't dance to it." - Frank Zappa The "thandy installable" file format: The file metaformat is a zip file, with the file extension .thp. It has these directories: content/ meta/ The "content" tree has the actual package contents, in a directory layout mirroring the layout of the installed package. The "meta" tree has information about the package. The meta tree has one required file, "package.json". Its contents are a single json object, with the following required fields: 'format-version' The number 1. An installer SHOULD NOT try to install a package with a format-version that it doesn't recognize. 'manifest' A list of files relative to the thp's content directory. Each file is an object with these fields: 'name': the name of the file relative to thp's content directory 'digest': optionally, a 2-tuple of a digest algorithm and a hexadecimal digest value. The following algorithms are supported: SHA256. 'length': optionally, the length of the file in bytes 'isconfig': optionally, a boolean. If it is present and true, this file is considered "configuration". 'package-name' The name of this package. There shouldn't be two packages with the same name installed at once in the same hierarchy. Ex: "Tor". 'package-version' The version of the package as a human-readable string. This should include both the version of the thing being packaged, and the version of of the package itself. Ex: "0.2.2.35-rc-7" is the seventh release of a thp file for Tor 0.2.2.35-rc. 'package-version-tuple' The version of the package as a list of numbers and strings such that a lexical comparison of two package version tuples is a correct version comparison. Ex: [ 0, 2, 2, 35, "rc", 7 ] 'timestamp' The time when this thp was generated, as a YYYY-MM-DD HH:MM:SS string, relative to UTC. Ex: "2011-03-02 17:33:07" 'additional-files' Optional: A list of files or file sets relative to the install root, to decribe files that the package is responsible for, even if they're not distributed with the package. This is used to kill off temporary and cache files on uninstall. Each file is an object with these fields: 'name': The name of the file relative to the install root. This name may contain "*" and "**" file-globbing patterns. 'isconfig': as for manifest. 'install-order' Optional: number between 0 and 100 inclusive to indicate that this package must be installed before or after others. Defaults to "50". 'options' Optional: a map from option strings to values. Known options are: 'cycle-install': This package should be installed to a separate install root from the currently installed package, then moved over. 'platform' Optional. The OS and CPU type that this package is for. 'require-features' Optional. A list of strings naming installer features that the installer needs to support to install this package correctly. An installer SHOULD NOT try to install the package if it does not recognize and support all members of this list. Ex: [ "pythonscripts" ] 'require-packages' Optional. A list of objects for all packages that must be installed before this package can be installed. Each object has these fields: 'package-name' 'min-version-tuple' 'scripts' Optional: a map from scripting language to set of scripts. Supported scripting languages are python2, sh, none. (We can add more later.) If the 'scripts' field is present, the installer SHOULD NOT install the package unless it supports one or more of the scripting languages. Each set of scripts contains one or more of the following fields: 'checkinst' 'preinst' 'postinst' 'prerm' 'postrm' Each names a file relative to meta/scripts in the thp zip file. Implementations SHOULD NOT generate other files or subdirectories of the main zip root directory; implementations MUST ignore files and subdirectories that they do not recognize. The package database: The installer keeps a directory that contains files describing the status of the packages we have installed. It has a subdirectory: "pkg-status". "pkg-status" has, for each package, two files: packagename.json, and packagename.status. Optionally, it has a packagename.json.new file. The packagename.json file contains a copy of the package.json file from the most recent successfully installed version of the the package. The packagename.status file contains a json object containing at least the field 'status' set to one of the following: "INSTALLED", "IN-PROGRESS". If a package install or upgrade is in-progress, packagename.json.new has the package.json file from the new version of the package. Scripts: Each script should be callable by a language- and platform-specific calling convention for invoking a script with named arguments. The arguments to the script are: THP_PACKAGE_NAME THP_OLD_VERSION (absent if we are doing a fresh installation) THP_NEW_VERSION THP_OLD_INSTALL_ROOT THP_INSTALL_ROOT THP_JSON_FILE THP_VERBOSE (flag: "1" if the script should log verbosely.) THP_PURGE (flag: "1" if we are doing a remove and we want to get rid of absolutely everything.) THP_TEMP_DIR For sh and python2 scripts, these arguments are passed as environment variables. Scripts need a way to signal success or failure. For sh and python2 scripts, this is done via return value. The 'none' script type must never have any scripts. Installers should never choose it if they support any other script type: it only exists to tell the installer that the scripts are optional. Directories: All installation happens relative to a single "install root". Thandy maintains a cache directory of its own, containing (among other things) downloaded thp files. The thp installer has a database directory explaining package status. Steps of operation: Checking for and downloading packages is done by thandy, and out-of-scope here, except inasmuch as Thandy needs the thp installer to say which version (if any) of package X is installed. The installer can do this by looking at the X.status file and the X.json file. To validate a package, the thp installer verifies that all files listed in the manifest are in fact installed with the sizes and digests listed, unless they are config files. {THIS NEXT PART IS A DRAFT AND NEEDS MORE THOUGHT! IT COULD BE WAY MORE ATOMIC} Installing and updating a downloaded package is done by Thandy teling the thp installer, "install/update these packages" with a list of thp files. To do this, the installer: * Grabs a lockfile. * Finds the current version, if any, of all of the packages. * Runs the checkinst script if present for every package that might need installing or updating. If any fails, the update can't happen. * For each package, sorted in topological order by 'requires' relationships, with ties broken by 'install-order' fields: * Overwrite the status file for the package in the database, changing the status to "IN-PROGRESS". Copy the package's package.json file to packagename.json.new in the database. * Run the package's preinst script. If that fails, abort. * Unpack all files in the package's content fork to their target locations. If files tagged 'config' are present, ignore them. * If any files are listed in the manifest for the old version of the package but not for the new version, and they are not config files, remove them. (Have an option to override this?) * Run the postinst script * Replace the packagename.json file with the packagename.json.new file. * Change the package status to "INSTALLED". CHANGES TO MAKE TO THE INSTALL PROCESS ABOVE: - Instead of overwrite-as-we-go, perhaps have overwrite as the very last step? - Perhaps checkpoint all files first for easy rollback? - Specify how to do the cycle-install feature. "Is it... atomic?" "Yes! VERY atomic!" -- The 5000 Fingers of Dr T Future directions and open questions: F.1 Configuration file updates We need a smart way to handle configuration file updates and changes in the future. There are several three-way merge tools available that we can model our behavior on, or re-use the code of, such as Debian's ucf tool. F.1 Binary patching There are auto-update tools (courgette for Chrome, as an example) which do smart binary patching, thus often drastically reducing the download time of updates. When the tool is more mature, we should look into ways to do this.

2 1

Designing and implementing improved circuit-setup protocol [was: GSoC 2011]
by Nick Mathewson 31 Mar '11

31 Mar '11

Quoting the whole message in my reply, since we're moving this to tor-dev where it belongs. On Wed, Mar 23, 2011 at 1:36 PM, Robert Ransom <rransom.8774(a)gmail.com> wrote: > On Wed, 23 Mar 2011 12:54:17 -0400 > Nick Mathewson <nickm(a)freehaven.net> wrote: > >> On Wed, Mar 23, 2011 at 8:10 AM, George Kadianakis <desnacked(a)gmail.com> wrote: >> > Hey Robert and Nick, >> > >> > I hate bureaucratic procedures and yet I have to send my GSoC >> > application 'till the 8th of April; so I want to get done with it. >> > >> > My application should have "a project proposal, why you'd like to >> > execute on this particular project, and the reason you're the best >> > individual to do so." (that was taken from GSoC FAQ). >> > >> > So regarding the project proposal, I talked with Robert yesterday and he >> > told me: >> > 02:18:24 <rransom> asn: Implementing the new circuit protocol will be >> > a good GSoC project for you. I think identity key >> > migration (and thus link protocol migration) will >> > require too much beyond the new circuit protocol. >> > I suppose that by "new circuit protocol" you mean the section 3.2 of >> > xxx-crypto-migration.txt, Robert. >> > >> > How do you feel about this, Nick? >> >> It could be helpful. I don't have a good sense of how much work this >> would be, since I don't have a good sense of what the new protocol >> would actually be. Robert -- how would you think this would break >> down into a summer's worth of work? > > The first step in the Great Tor Crypto Migration is to add new CREATE2 > and EXTEND2 RELAY cell types. They can be used with the existing > circuit-extension handshake and link protocol initially, but will be > extensible to support new ones. > > Further steps, all independent of each other: > > * Add 128-byte and 2048-byte RELAY cells and a circuit-configuration > cell, initially to allow the client to change the cell size to be > used on a circuit. > > * Refactor Tor's cryptographic primitive abstractions to accommodate > public-key encryption primitives, public-key signature primitives, > symmetric authenticated encryption, symmetric block encryption, and > hashes. > > * Implement one or more new link protocols that do not constrain a > relay's choice of identity key cryptosystem. > > Further mutually independent steps building on those above: > > * Modify the directory protocol and implementation to support relays > with multiple identity keys. > > * Implement a new circuit-extension handshake (the part that involves > ‘onionskins’). > > * Implement a new circuit ciphersuite (the part that mangles cell data > so that relay A can't see what data relay C sees). Hm. These steps all stretch pretty far beyond what's just described in 3.2 of xxx-crypto-migration. I think they're probably more than we can promise to design before summer, and possibly more than a typical gsoc scope all put together. >> I'm also a little concerned about the interaction of 3.2 and 3.3 >> ("Relay crypto") : I'll be surprised if it turns out that we can >> design a good circuit extend protocol without thinking about the >> countours of a new relay protocol. (Not that you'd have to build both >> at once, but we should think about them all as we design.) > > It's actually the other way around -- we need a new EXTEND cell before > we can use a new link protocol. (Otherwise, we would have to build in > a covert channel (i.e. a backdoor for people who want to block Tor by > handshake) in the new link protocol to indicate client and server link > protocol versions, and that really *really* sucks.) I'm talking about the stuff in 3.3: the relay protocol, where we process cells. Link protocol stuff is 3.1. Also, I'm talking about *design order*, not *implementation order*: The different parts of the Tor protocol are not sufficiently orthogonal that we can do them independently. Thus, we need to get most of the design changes sketched out before we can be reasonably say that any part of the redesign does what the other parts need. >> Maybe we should get a protocol sketch together this week if the app is >> due April 8. > > Yes. I have the EXTEND2 cell draft written; I bogged down on writing > explanatory text (I thought I didn't have enough in the draft, but > didn't know what to add). > > >> Either way, I think this might be drifting into design work, so if >> there are no objections, I'd suggest that we follow up on the tor-dev >> mailing list. :) > > Good idea. > > > Robert Ransom >

3 2