- tor-dev - lists.torproject.org

Proposal: Optimistic Data for Tor: Client Side
by Ian Goldberg 24 Sep '11

24 Sep '11

Sorry this took so long. As usual, things got inserted ahead of it in the priority queue. :-p Anyway, here's the client-side sibling proposal to the already-implemented 174. It cuts down time-to-first-byte for HTTP requests by 25 to 50 percent, so long as your SOCKS client (e.g. webfetch, polipo, etc.) is patched to support it. (With that kind of speedup, I think it's worth it.) Discuss. ;-) - Ian

4 9

The Torouter project - where are we now?
by Runa A. Sandvik 24 Sep '11

24 Sep '11

Hi everyone, I am trying to figure out how we can move the Torouter project forward. In this email, I will try to summarize the current status of this project. The Torouter project has a total of four bugs in Trac. These are #2334, #2376, #2370 and #2596. #2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look at this and figure out a way to solve it. #2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint. There are a couple of different ways to solve this problem; Karsten suggested that we could modify OpenWrt to stop creating a /tmp partition. This probably means that we will have to ship our own OpenWrt image. I'm thinking that another option would be to modify the Tor-OpenWrt-package to use / as the data directory instead of /tmp. However, I wonder if 64M (no matter how it's partitioned) of space on the Buffalo router is insufficient as long as #2334 remains open. #2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt. #2596: Figure out a better name than "torouter". Andrew thinks we should come up with a better name for this project, one that does not have "Tor" in the name. Suggestions? The Torouter project also has some open questions (some are mentioned on https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter as well): 1. How can we make sure that the version of Tor in OpenWrt is always up to date? Should we set up our own OpenWrt repository? Right now, the version of Tor in OpenWrt is 0.2.1.26. Also, should we offer packages for both stable and unstable? 2. We want to collect statistics, which means that we need to ship a GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts? 3. Do we want one Tor process for bridging and one for the transparent wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too. Have I missed anything important? Are there any other packages that we should include in OpenWrt? Other comments or suggestions? If you have more information or would like to help out, please let me know. Thanks, -- Runa A. Sandvik

8 16

Re: [tor-dev] Proposal 185: Directory caches without DirPort
by Karsten Loesing 23 Sep '11

23 Sep '11

Looks like we lost tor-dev somewhere. Quoting Nick's reply without cutting out stuff I have no comments to. On 9/22/11 10:16 PM, Nick Mathewson wrote: > On Thu, Sep 22, 2011 at 4:49 AM, Karsten Loesing > <karsten.loesing(a)gmx.net> wrote: >> Hi Nick, >> >> and here are a few comments to your proposal 185: >> >> On 9/20/11 9:34 PM, Nick Mathewson wrote: >>> Configuration: >>> >>> Add a new torrc option, "DirCache". Its values can be "0", "1", >>> and "auto". If it is 0, we never act as a directory cache, even >>> if DirPort is set. >> >> Do these relays set "dir-cache 0" in their router descriptors? Or do >> they just not include "dir-cache 1"? > > They just do not include "dir-cache 1". I'll make this more explicit. > >> But if they set a non-zero >> DirPort, how will the directory authorities and clients know that the >> node doesn't want to act as a directory cache? > > If you don't want to act as a directory cache, you don't publish DirPort. > > "dir-cache 0" wouldn't work to disable a dirport published in your > "router" line, since old Tors wouldn't recognize the dir-cache option, > and so wouldn't know that they weren't supposed to use DirPort. > >> >>> If it is 1, then we act as a directory cache >>> according to same rules as those used for nodes that set a >>> DirPort. If it is "auto", then Tor decides whether to act as a >>> directory cache. >> >> What are the rules when setting a DirPort? Successful self-test and >> minimum advertised bandwidth? > > I believe so, except of course you don't need to have a reachable > DirPort in this case. > >> How's "1" different from "auto" if Tor >> decides whether to act as a directory cache in both cases? > > "1" is "I want to be a directory cache"; "auto" is "use me as a > directory cache if that would help". I think that auto might be in > the future be more adaptive, depending on the state of the network and > the number of directory caches. > > I would like "DirCache auto" to be something we're comfortable making > the new default. > >>> Consensus: >>> >>> Authorities should assign a "DirCache" flag to all nodes running >>> as a directory cache that do not set a DirPort. >> >> Would it make sense to have them assign the "DirCache" flag for nodes >> with a non-zero DirPort, too? > > IMO that would just use up space. Is there an argument in favor? Yup, I agree with you that it uses up space. My thinking was that clients wouldn't have to check two things (non-zero Dir port or DirCache flag) to learn whether a relay has directory info for them, but just one thing. And it could lead to some confusion when people try to understand dir-spec.txt or the TorStatus output (once it displays this flag). Not very strong arguments though. I also wasn't certain if the proposal was wrong here and you really meant that the flag would be assigned to all directory caches. If the proposal says what you meant, carry on. :) Best, Karsten

2 1

Proposal 184: Miscellaneous changes for a v3 Tor link protocol
by Nick Mathewson 23 Sep '11

23 Sep '11

Filename: 184-v3-link-protocol.txt Title: Miscellaneous changes for a v3 Tor link protocol Author: Nick Mathewson Created: 19-Sep-2011 Status: Open Target: 0.2.3.x Overview: When proposals 176 and 179 are implemented, Tor will have a new link protocol. I propose two simple improvements for the v3 link protocol: a more partitioned set of which types indicate variable-length cells, and a better way to handle link padding if and when we come up with a decent scheme for it. Motivation: We're getting a new link protocol in 0.2.3.x, thanks (again) to TLS fingerprinting concerns. When we do, it'd be nice to take care of some small issues that require a link protocol version increment. First, our system for introducing new variable-length cell types has required a protocol increment for each one. Unlike fixed-length (512 byte) cells, we can't add new variable-length cells in the existing link protocols and just let older clients ignore them, because unless the recipient knows which cells are variable-length, it will treat them as 512-byte cells and discard too much of the stream or too little. In the past, it's been useful to be able to introduce new cell types without having to increment the link protocol version. Second, once we have our new TLS handshake in place, we will want a good way to address the remaining fingerprinting opportunities. Some of those will likely involve traffic volume. We can't fix that easily with our existing PADDING cell type, since PADDING cells are fixed-length, and wouldn't be so easy to use to break up our TLS record sizes. Design: Indicating variable-length cells. Beginning with the v3 link protocol, we specify that all cell types in the range 128..255 indicate variable-length cells. Cell types in the range 0..127 are still used for 512-byte cells, except that the VERSIONS cell type (7) also indicates a variable-length cell (for backward compatibility). As before, all Tor instances must ignore cells with types that they don't recognize. Design: Variable-length padding. We add a new variable-length cell type, "VPADDING", to be used for padding. All Tor instances may send a DROP cell at any point that a VERSIONS cell is not required; a VPADDING cell's body may be any length; the body of a VPADDING cell MAY have any content. Upon receiving a VPADDING cell, the recipient should drop it, as with a PADDING cell. Interaction with proposal 176: Proposal 176 says that during the v3 handshake, no cells other than VERSIONS, AUTHENTICATE, AUTH_CHALLENGE, CERT, and NETINFO are allowed, and those are only allowed in their standard order. If this proposal is accepted, then VPADDING cells should also be allowed in the handshake at any point after the VERSIONS cell. They should be included when computing the "SLOG" and "CLOG" handshake-digest fields of the AUTHENTICATE cell. Notes on future-proofing: It may be in the future we need a new cell format that is neither the original 512-byte format nor the variable-length format. If we do, we can just increment the link protocol version number again. Right now we have 10 cell types; with this proposal and proposal 176, we will have 14. It's unlikely that we'll run out any time soon, but if we start to approach the number 64 with fixed-length cell types or 196 with var-length cell types, we should consider tweaking the link protocol to have a variable-length cell type encoding.

3 2

Proposal 185: Directory caches without DirPort
by Nick Mathewson 23 Sep '11

23 Sep '11

Filename: 185-dir-without-dirport.txt Title: Directory caches without DirPort Author: Nick Mathewson Created: 20-Sep-2011 Status: Open Overview: Exposing a directory port is no longer necessary for running as a directory cache. This proposal suggests that we eliminate that requirement, and describes how. Motivation: Now that we tunnel directory connections by default, it is no longer necessary to have a DirPort to be a directory cache. In fact, bridges act as directory caches but do not actually have a DirPort exposed. It would be nice and tidy to expand that property to the rest of the network. Configuration: Add a new torrc option, "DirCache". Its values can be "0", "1", and "auto". If it is 0, we never act as a directory cache, even if DirPort is set. If it is 1, then we act as a directory cache according to same rules as those used for nodes that set a DirPort. If it is "auto", then Tor decides whether to act as a directory cache. Advertising cache status: Nodes which are running as a directory cache but which do not have a DirPort set should set the entry "dir-cache 1" in their router descriptors. Consensus: Authorities should assign a "DirCache" flag to all nodes running as a directory cache that do not set a DirPort. This does not require a new version of the consensus algorithm.

3 2

connection_edge_process_relay_cell(): data cell dropped, unknown stream
by Fabian Keil 22 Sep '11

22 Sep '11

While experimenting with Tor v0.2.3.4-alpha's "optimistic data" and thus keeping an eye on the logs, I noticed that I get a lot of the following messages (with and without "optimistic data"): Sep 20 13:47:57.428 [debug] {APP} connection_edge_process_relay_cell(): Now seen 8236 relay cells here (command 2, stream 49411). Sep 20 13:47:57.428 [debug] {APP} connection_edge_process_relay_cell(): circ deliver_window now 911. Sep 20 13:47:57.428 [info] {APP} connection_edge_process_relay_cell(): data cell dropped, unknown stream (streamid 49411). Sep 20 13:47:57.431 [debug] {APP} connection_edge_process_relay_cell(): Now seen 8237 relay cells here (command 2, stream 49411). Sep 20 13:47:57.431 [debug] {APP} connection_edge_process_relay_cell(): circ deliver_window now 910. Sep 20 13:47:57.431 [info] {APP} connection_edge_process_relay_cell(): data cell dropped, unknown stream (streamid 49411). Sep 20 13:47:57.432 [debug] {APP} connection_edge_process_relay_cell(): Now seen 8238 relay cells here (command 2, stream 49411). Sep 20 13:47:57.432 [debug] {APP} connection_edge_process_relay_cell(): circ deliver_window now 909. Sep 20 13:47:57.432 [info] {APP} connection_edge_process_relay_cell(): data cell dropped, unknown stream (streamid 49411). Sep 20 13:47:57.432 [debug] {APP} connection_edge_process_relay_cell(): Now seen 8239 relay cells here (command 2, stream 49411). [...] Sep 20 13:48:01.479 [debug] {APP} connection_edge_process_relay_cell(): Now seen 8379 relay cells here (command 2, stream 49411). Sep 20 13:48:01.479 [debug] {APP} connection_edge_process_relay_cell(): circ deliver_window now 968. Sep 20 13:48:01.479 [info] {APP} connection_edge_process_relay_cell(): data cell dropped, unknown stream (streamid 49411). Sep 20 13:48:01.479 [debug] {APP} relay_lookup_conn(): found conn for stream 49410. Sometimes up to 400-500 dropped cells per unknown stream id. The deliver window seems to go down to 900 and then starts at 999 again. It's my impression that it always happens after the client connection is closed prematurely, something my feed reader unfortunately does frequently: Sep 20 13:47:56.608 [debug] {APP} connection_edge_process_relay_cell(): Now seen 8235 relay cells here (command 2, stream 49411). Sep 20 13:47:56.608 [debug] {APP} connection_edge_process_relay_cell(): circ deliver_window now 912. Sep 20 13:47:56.792 [debug] {APP} circuit_consider_stop_edge_reading(): considering layer_hint->package_window 995 Sep 20 13:47:56.792 [info] {EDGE} connection_edge_reached_eof(): conn (fd 14) reached eof. Closing. I assume it can take several seconds for the exit relay to stop sending data and the Tor client has to continue reading from the stream to keep the circuit usable. Is that correct? Is it intended behaviour for the Tor client to forget a stream id before the stream is actually closed on the remote end? Fabian

2 2

Tor-related postdoc position available
by Ian Goldberg 20 Sep '11

20 Sep '11

In case you're interested but hadn't yet heard, I've got a postdoctoral researcher position available "in the field of privacy-enhancing technologies, preferably on the topic of privacy-preserving communications systems". If you (or someone you know) is finishing up a PhD and would like to work on PETs/Tor stuff with us Waterloo folks, drop me a line (noting "Postdoctoral position" in the subject line). Details are here: http://crysp.uwaterloo.ca/prospective/postdoc/ Thanks, - Ian

1 0

Building an automatic censorship-detection system for Tor
by Karsten Loesing 17 Sep '11

17 Sep '11

In the past years we observed sudden drops in Tor usage in certain countries. Some of these events may be attributed to countries specifically wanting to block Tor, as it enables their citizens to circumvent their censorship infrastructure. Other events likely result from country-wide Internet outages that are unrelated to Tor. Two such events happened in January 2011 in Iran [0] and Egypt [1]. The key point is that we can derive from observed Tor usage when something potentially interesting is going on in a country. We should be able to build a system that keeps track of our daily user estimates per country and automatically sends out a warning whenever there are sudden changes. George Danezis wrote "An anomaly-based censorship-detection system for Tor" [2]. George's detection code [3] uses our daily user estimates [4] as input and gives a list of sudden upturns and downturns per country as output. We integrated the results of his script in the metrics website [5] to demonstrate what events the detector recognizes as possible censorship events. For example, the detector would have warned in January 2011 about the events in Iran [6] and Egypt [7], shown as red dots for downturns. However, as one can see, George's script also detects quite a few false positives. Whenever there's a red or blue dot, the script would have issued a warning for a human to check. It would be neat to reduce these false warnings while still catching the really suspicious events. Want to help us make our censorship-detection system better? Any suggestion to improve George's algorithm or to come up with an alternative approach to detect possible censorship events in our data would be much appreciated! Let us know if we can help you get started. [0] https://metrics.torproject.org/users.html?graph=direct-users&start=2011-01-… [1] https://metrics.torproject.org/users.html?graph=direct-users&start=2011-01-… [2] https://metrics.torproject.org/papers/detector-2011-08-11.pdf [3] https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-2718 [4] https://metrics.torproject.org/csv/direct-users.csv [5] https://metrics.torproject.org/users.html#censorship-events [6] https://metrics.torproject.org/users.html?graph=direct-users&start=2011-01-… [7] https://metrics.torproject.org/users.html?graph=direct-users&start=2011-01-…

3 2

Run Tor relay in background on OS X
by Doug Ransom 04 Sep '11

04 Sep '11

Hi, I was thinking of tweaking TOR and/or Vidalia so users can easily (i.e. w/o being a skilled unix sysadmin) to configure TOR to run at boot. Any thoughts or concerns before I poke at the source code? In the past I have had some success running TOR at boot using Launchd, but its hard and doesnt play nice with Vidalia. Doug Ransom doug.ransom(a)alumni.uvic.ca

1 0

Research problem: better guard rotation parameters
by Roger Dingledine 30 Aug '11

30 Aug '11

The next in my series of research blog posts is up: https://blog.torproject.org/blog/research-problem-better-guard-rotation-par… "What algorithm should we use to assign Guard flags such that a) we assign the flag to as many relays as possible, yet b) we minimize the chance that Alice will use the adversary's node as a guard?" --Roger

1 0