tor-dev August 2011

tor-dev@lists.torproject.org

17 participants
16 discussions

Thandy and what's next
by Tomas Touceda 13 May '12

13 May '12

Hello everyone, Thandy, for those who haven't heard about it by now, in few words is a package system for Tor packages and the like. One of the biggest problems we have right now is keeping users' Tor up to date, so we are trying to make time to get this going. (For more detailed information about Thandy check [1], specially the doc/ and specs/ directories) A couple of months ago, nickm and erinn worked in writing the package format spec (see [2] for more details), and now I'm going to start … [View More]

4 4

New paper by Goldberg, Stebila, and Ostaoglu with proposed circuit handshake
by Nick Mathewson 16 Jan '12

16 Jan '12

Crypto people who have been following threads about the circuit-establishment handshake will be interested in the new paper, "Anonymity and one-way authentication in key-exchange protocols", by Goldberg, Stebila, and Ostaoglu. Here's the version they updated today: http://www.cacr.math.uwaterloo.ca/techreports/2011/cacr2011-11.pdf If we're moving to an improved handshake, this might be a good candidate to consider. The protocol itself is on page 14. Some notes, written by a guy who knows … [View More]

8 29

Sanitizing and publishing our web server logs
by Karsten Loesing 01 Dec '11

01 Dec '11

Hi everyone, we have been discussing sanitizing and publishing our web server logs for quite a while now. The idea is to remove all potentially sensitive parts from the logs, publish them in monthly tarballs on the metrics website, and analyze them for top visited pages, top downloaded packages, etc. See the tickets #1641 and #2489 for details. Here's a suggested sanitizing procedure for our web logs, which are in Apache's combined log format: - Ignore everything except GET requests. - … [View More]Ignore all requests that resulted in a 404 status code. - Rewrite log lines so that they only contain the following fields: - IP address 0.0.0.0 for HTTP request or 0.0.0.1 for HTTPS requests (as logged by our Apache configuration), - the request date (with the time part set to 00:00:00), - the requested URL (cut off at the first encountered "?"), - the HTTP version, - the server's HTTP status code, and - the size of the returned object. - Write all lines from a given virtual host and day to a single output file. - Sort the output file alphanumerically to conceal the original order of requests. Here's a sample sanitized log file for www.torproject.org from May 1, 2009 (462K): http://freehaven.net/~karsten/volatile/www.torproject.org-access.log-200905… Is there still anything sensitive in that log file that we should remove? For example: - Do the logs reveal how many pages were cached already on the requestor's site (e.g. as repeat accesses)? Note that log files are sorted before being published. - Are there other concerns about making these sanitized log files publicly available? Are the decisions to remove parts from the logs reasonable? In particular: - Do we have to take out all requests with 404 status codes? Some of these requests for non-existing URLs contain typos which may not be safe to make public. Should we instead put in some placeholder for the URL part and keep the 404 lines to know how many 404's we have per day? - Is there any good reason to keep the portion of a URL after a "?"? - Is it possible to leave some part of Referers in the logs that helps us figure out where our traffic originates and what search terms people use to find us? - Can we resolve client IP addresses to country codes and include those in the logs instead of our 0.0.0.0/0.0.0.1 code for HTTP/HTTPS? How would we handle countries with only a few users per day, e.g., should there be a threshold below which we consider requests to come from "a country with less than XY users?" Thanks, Karsten [View Less]

6 10

Re: [tor-dev] Proposal 178: Require majority of authorities to vote for consensus parameters
by Nick Mathewson 27 Nov '11

27 Nov '11

On Tue, Feb 22, 2011 at 1:34 AM, Sebastian Hahn <hahn.seb(a)web.de> wrote: > Design: > > When the consensus is generated, the directory authorities ensure that > a param is only included in the list of params if at least half of the > total number of authorities votes for that param. The value chosen is > the low-median of all the votes. We don't mandate that the authorities > have to vote on exactly the same value for it to be included because > some consensus … [View More]

2 7

Survey on Tor Trac usage and how you manage your tasks
by Karsten Loesing 25 Oct '11

25 Oct '11

Hi everyone! At the last annual dev meeting in Waterloo we had a very productive discussion about managing the various channels of Tor communication: https://trac.torproject.org/projects/tor/wiki/org/meetings/2011TorAnnualDev… Related to that discussion, we should talk about how we use Tor's Trac system, or more generally how we manage our Tor tasks. We use Trac as issue tracker, wiki, and for project management. But it seems that everyone uses Trac slightly different and has their own … [View More]expectations how other people use it. This can slow us down. In addition to that, some people may not know how Trac can solve their problems. Knowing how others use Trac might help them manage their tasks more quickly. Let's conduct a survey! The survey below consists of two parts: The first part focuses on Trac's features and whether you're using them or not. The second part is about managing your tasks in general, either with or without Trac. If you are a Tor developer or volunteer or think you have some input on the topic, please fill out this survey until Sunday, August 28, 2011 and send it either to this list or to me. I'll try to summarize what was said and send the survey results to this list, hopefully before the end of the month. Thanks! Karsten 1 Using Trac features 1.1 Which of the reports (stored ticket queries) do you use most often? 1.2 What are typical custom queries that you run? 1.3 How do you use milestones and roadmaps? 1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists? 1.5 Which wiki pages do you read/edit most often? 1.6 How do you search for wiki pages? 1.7 What are typical search terms that you use when using the search features? 1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use? 1.9 How relevant are the following ticket fields for you? 1.9.1 Reported by 1.9.2 Owned by 1.9.3 Priority 1.9.4 Milestone 1.9.5 Component 1.9.6 Version 1.9.7 Keywords 1.9.8 Cc 1.9.9 Parent ID 1.9.10 Points 1.9.11 Actual Points 1.10 How relevant are the following ticket statuses for you? 1.10.1 accepted 1.10.2 assigned 1.10.3 closed 1.10.4 needs_information 1.10.5 needs_review 1.10.6 new 1.10.7 reopened 1.11 What other features do you use in Trac? 1.12 What features are you missing in Trac? 1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)? 2 Solving typical software development tasks Note that the questions below don't just focus on Trac, but on any tools or communication media you use for solving a task. 2.1 How do you decide what to work on, both when fixing bugs or when implementing new features? 2.2 How do you keep track of what things you're supposed to do for sponsors and for when? 2.3 How do you memorize new ideas to work on when they come up? 2.4 How do you coordinate working together with someone on something? 2.5 How do you learn who you could work together with on something? 2.6 What other software development tasks do you have that may be supported by Trac or a Trac-like system? [View Less]

15 35

Proposal: Optimistic Data for Tor: Client Side
by Ian Goldberg 24 Sep '11

24 Sep '11

Sorry this took so long. As usual, things got inserted ahead of it in the priority queue. :-p Anyway, here's the client-side sibling proposal to the already-implemented 174. It cuts down time-to-first-byte for HTTP requests by 25 to 50 percent, so long as your SOCKS client (e.g. webfetch, polipo, etc.) is patched to support it. (With that kind of speedup, I think it's worth it.) Discuss. ;-) - Ian

4 9

The Torouter project - where are we now?
by Runa A. Sandvik 24 Sep '11

24 Sep '11

Hi everyone, I am trying to figure out how we can move the Torouter project forward. In this email, I will try to summarize the current status of this project. The Torouter project has a total of four bugs in Trac. These are #2334, #2376, #2370 and #2596. #2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look … [View More]at this and figure out a way to solve it. #2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint. There are a couple of different ways to solve this problem; Karsten suggested that we could modify OpenWrt to stop creating a /tmp partition. This probably means that we will have to ship our own OpenWrt image. I'm thinking that another option would be to modify the Tor-OpenWrt-package to use / as the data directory instead of /tmp. However, I wonder if 64M (no matter how it's partitioned) of space on the Buffalo router is insufficient as long as #2334 remains open. #2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt. #2596: Figure out a better name than "torouter". Andrew thinks we should come up with a better name for this project, one that does not have "Tor" in the name. Suggestions? The Torouter project also has some open questions (some are mentioned on https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter as well): 1. How can we make sure that the version of Tor in OpenWrt is always up to date? Should we set up our own OpenWrt repository? Right now, the version of Tor in OpenWrt is 0.2.1.26. Also, should we offer packages for both stable and unstable? 2. We want to collect statistics, which means that we need to ship a GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts? 3. Do we want one Tor process for bridging and one for the transparent wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too. Have I missed anything important? Are there any other packages that we should include in OpenWrt? Other comments or suggestions? If you have more information or would like to help out, please let me know. Thanks, -- Runa A. Sandvik [View Less]

8 16

Research problem: better guard rotation parameters
by Roger Dingledine 30 Aug '11

30 Aug '11

The next in my series of research blog posts is up: https://blog.torproject.org/blog/research-problem-better-guard-rotation-par… "What algorithm should we use to assign Guard flags such that a) we assign the flag to as many relays as possible, yet b) we minimize the chance that Alice will use the adversary's node as a guard?" --Roger

1 0

Exit Enclaving and the future of?
by Jacob Appelbaum 29 Aug '11

29 Aug '11

Hi, I'll try to sum up Exit Enclaving in no certain terms and hope that Nick or someone else will chime in to correct my assumptions. This should probably be formalized and put into the path-spec at some point. I was surprised to not see it in there to begin with but I probably just missed it. Anyway, without further delay... When Tor builds a circuit it does so with certain constraints in mind - most of these are described in the path-spec. Exit Enclaving is a set of client circuit building … [View More]constraints that all go together. When a user wishes to connect to a given IP address, Tor will check this IP against all of the Tor nodes in the Tor consensus. If the IP address in question is a Tor node in the consensus, the Tor client will check the selected Tor node's exit policy. If the Tor node's exit policy allows connections to *itself* - that is the IP address of Tor node - we check that the requested port is allowed by that policy. If we find that this set of constraints is matched, we build a *four* hop circuit to the Tor node that has the same IP as our request. The Tor node that matches is the fourth hop and we force exiting to the service offered by that node. The user then exits from the Tor node that met the constraints and they're connected to the service offered by that Tor node. As a result, client security is greatly improved by using Tor as the full path to the services offered on that Tor node. Obviously, with a bit of firewall trickery, it would be possible to offer the services on another physical host but the general point still stands - the service on a given IP address may only be enclaved by an IP address that is a confirmed Tor relay. I think Exit Enclaving is confusing for the uninitiated. It is documented but not in the standard Tor ways and it appears to be one of the lesser understood things in Tor land. Here's the official documentation as far as I've found it : https://trac.torproject.org/projects/tor/wiki/doc/ExitEnclave Here's the important bits from that page: SocksPort 9050 # what port to open for local application connections SocksListenAddress 127.0.0.1 # accept connections only from localhost ORPort 9001 Nickname archivetpo ExitPolicyRejectPrivate 0 ExitPolicy accept 38.229.70.19:443 ExitPolicy reject *:* There are a few references to the term (Exit) Enclave in the torspec.git repo but none of them actually define how it works with absolute authority. At best we've described how to try to become one without actually explaining the essence concretely in a specification. I think we should add a paragraph to the man page as well as the torspec. I'd be happy to write it once we actually hammer out the current and pas client behaviour. There is a problem with the current Exit Enclaving approach and if anything, it's not the worst one to have. If the user attempts to connect with a name, we are unable to try to satisfy the constraints that require an IP address; so we just skip Enclaving entirely for that connection. After the first sucessful connection or the first time Tor resolves the name, we're able to take the above steps in our attempt to opportunistically enclave. Before that - the Tor client only knows the name and thus the IP address is a mystery. Generally this results in the first connection exiting via any other node that allows exiting to the target host. I think this is perhaps an anonymity concern as it shows that the Tor client has likely not connected to the host in question - an evil Exit might take note of this and watch for correlated streams over time, etc. Here's an example of a semi-popular website documenting their enclaving experience: http://www.gabrielweinberg.com/blog/2010/08/duckduckgo-now-operates-a-tor-e… Recently the CCC jabber server became an exit enclave and a hidden service, similar to the DuckDuckGo setup: https://twitter.com/#!/jabbercccde/status/107850926638895104 https://twitter.com/#!/jabbercccde/status/107850540842627072 As I stated previously, I think that the use of Exit Enclaves greatly improves client security. It reduces the chances of a network sniffer getting lucky, it helps to mitigate the severity of services getting crypto stuff wrong, etc. I think it would be nice to make this easier to set-up and configure for server administrators. I'd also like to make this more flexible for Tor clients and if possible, find some methods for making Tor clients automatically enclave on the first connection. There are a few ways that we might see Exit Enclaving happen on the first connection. It is possible to make the first connection enclave if the client is using the TorDNSPort; Tor will have resolved and cached the names used by all clients, connections by names should then Enclave and connections by IP will Enclave as well. Alternatively, having all connections to Tor happen by IP address only would force all clients to Enclave on first connection. I think this is probably dangerous in all cases except a Transparent Tor network. Another possible hack is to set MapAddress for the hosts that matter to you - this doesn't scale very well and it's confusing for most people; it isn't dynamic and it seems that it will likely desynchronise over time. I'd like to advocate for a simple set of options to make this stuff easier to use and more likely to be used by a server admin. Here's a bug where I suggest an option OfferExitEnclave that is either a bool or a port list: https://trac.torproject.org/projects/tor/ticket/800 My thought was that a service provider who wants to offer enhanced service to Tor users could set a single flag and it would create a Tor Exit node that only exits to itself; I imagine that it might make sense to change the advertised bandwidth. It might even be a good idea to set a reasonable bandwidth cap of sorts but I think this is probably not great until we can automatically guess intelligently. I imagine that flipping the hypothetical OfferExitEnclave switch would actually be short hand for the following: ORPort auto Nickname exitenclave MaxAdvertisedBandwidth 20KB ExitPolicyRejectPrivate 0 ExitPolicy accept ORListenConfirmedPublicAddress:* ExitPolicy reject *:* Thus when we want to help someone enclave their entire host - we tell them to flip a single flag like so: OfferExitEnclave 1 If they want to enclave a specific service on their host - we tell them to flip the same flag like so: OfferExitEnclave 1 ExitEnclavePorts 5222 I'm not sold on that set of options but I think offering at least the OfferExitEnclave option as a bool would result in a lot more clarity for a server admin who wants to enhance their services for use with Tor. I think that generally, the nice part about Exit Enclaving is that it works as well as the rest of Tor - it does not really rely on external dependencies at all - no need for DNS or DNSSEC stuff, no need for the authorities to do any extra work on a per node basis, the client can use a different exit if they like, etc. On the client side, I think that would could make some improvements that aren't merely cosmetic but when enabled would improve client security. As I've already stated, Transparent Tor networks with TorDNSPort do this automatically by the nature of how they function. Other than the Torouter and probably TAILS, I'm not sure of other projects that takes advantage of this option. I'm not even certain that it will work in the long term for Torouter because of SRV records and jabber clients. Generally the client situation as it stands today lacks a general switch to force attempts at Exit Enclaving by name. Such an option would force a full round trip. Certainly we lack a general manner that attempts to Exit Enclave for all hosts and I'd like to see that change. I think it would be nice to have a few options and I'm far from sold on any of them. I'll lay them out as if they were already implemented. LearnExitEnclaveNames 0|1 This option could learn what possible names are frequently used and opportunistically resolve them, perhaps even caching some meta-data about exit enclaves that we've used, if it was stable, etc - this is could be something like a mash up TrackHostExits, LongLivedPorts, MapAddress, and something that learns overtime the hosts you use regularly. I think it would need some work to be done in a privacy preserving manner, probably with something like scrypt, if it needed to be cached on disk. This would solve the problem of a host being a known Exit Enclave after the first resovle and then later expiring from Tor's internal DNS cache. I think this is a common problem for IM connections and simply marking that name as something worth the round trip would be a huge gain, even if only for a single Tor run where the results are never cached on disk. My Tor often runs for days at a time and I think that the DNS cache internally expires before my Tor client exits. I'm pretty sure that means that it's up to my IM client or other software to cache the IP or we're back to square zero. AttemptExitEnclaving 0|1 This option could simply force the full round trip for all connections. Simple enough but likely a major performance hit. The default would be zero and keep things as they are today. AttemptExitEnclavingPorts [port list] This option could attempt Enclaving for any connections to any host if the connection is to one of the listed ports. AlwaysAttemptEnclaveNames [list of names] This option could take a list of names where we want to force a full round trip and that we'd like to enclave. Perhaps it would even fail to work if the Enclaving would fail and perhaps it could fail in a helpful manner. I imagine that some people might like this for customised browser bundles or if they're a power user who knows what they want, etc. I know that Nick had some suggestions - perhaps adding known Exit Enclaving names into the consensus or into descriptors or something that the directory authorities handle. I'm not sure that I fully understand his suggestions and I look forward to his reply to this email. :) All the best, Jake [View Less]

1 0

lround() missing in MSVC
by Gisle Vanem 24 Aug '11

24 Aug '11

lround() is missing in MS Visual-C's <math.h>. Not available anywhere. Here is an easy patch: --- Git-latest\src\common\util.c Sun Aug 14 13:48:49 2011 +++ src\common\util.c Tue Aug 23 21:15:25 2011 @@ -335,7 +335,11 @@ long tor_lround(double d) { +#ifdef _MSC_VER + return (long)(d > 0 ? d + 0.5 : ceil(d - 0.5)); +#else return lround(d); +#endif } /** Returns floor(log2(u64)). If u64 is 0, (incorrectly) returns 0. */ ---------------- --gv

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev August 2011