- tor-dev - lists.torproject.org

Proposal: Faster Headless Consensus Bootstrapping
by Mike Perry 11 Oct '12

11 Oct '12

Also at: https://gitweb.torproject.org/user/mikeperry/torspec.git/blob/consensus-boo… ------------------------------------------------------------------------- Title: Faster Headless Consensus Bootstrapping Author: Mike Perry Created: 01-10-2012 Status: Open Target: 0.2.4.x+ Overview and Motiviation This proposal describes a way for clients to fetch the initial consensus more quickly in situations where some or all of the directory authorities are unreachable. This proposal is meant to describe a solution for bug #4483. Design: Bootstrap Process Changes The core idea is to attempt to establish bootstrap connections in parallel during the bootstrap process, and download the consensus from the first connection that completes. Connection attempts will be done in batches of three. Only one connection will be performed to one of the canonical directory authorities. Two connections will be performed to randomly chosen hard coded directory mirrors. If no connections complete within 5 seconds, another batch of three connections will be launched. Otherwise, the first connection to complete will be used to download the consensus document and the others will be closed, after which bootstrapping will proceed as normal. If at any time, the total outstanding bootstrap connection attempts exceeds 15, no new connection attempts are to be launched until existing connection attempts experience full timeout. Design: Fallback Dir Mirror Selection The set of hard coded directory mirrors from #572 shall be chosen using the 100 Guard nodes with the longest uptime. The fallback weights will be set using each mirror's fraction of consensus bandwidth out of the total of all 100 mirrors. This list of fallback dir mirrors should be updated with every major Tor release. In future releases, the number of dir mirrors should be set at 20% of the current Guard nodes, rather than fixed at 100. Performance: Additional Load with Current Parameter Choices This design and the connection count parameters were chosen such that no additional bandwidth load would be placed on the directory authorities. In fact, the directory authorities should experience less load, because they will not need to serve the consensus document for a connection in the event that one of the directory mirrors complete their connection before the directory authority does. However, the scheme does place additional TLS connection load on the fallback dir mirrors. Because bootstrapping is rare and all but one of the TLS connections will be very short-lived and unused, this should not be a substantial issue. The dangerous case is in the event of a prolonged consensus failure that induces all clients to enter into the bootstrap process. In this case, the number of initial TLS connections to the fallback dir mirrors would be 2*C/100, or 10,000 for C=500,000 users. If no connections complete before the five retries, this could reach as high as 50,000 connection attempts, but this is extremely unlikely to happen in full aggregate. However, in the no-consensus scenario today, the directory authorities would already experience C/9 or 55,555 connection attempts. The 5-retry scheme increases their total maximum load to about 275,000 connection attempts, but again this is unlikely to be reached in aggregate. Additionally, with this scheme, even if the dirauths are taken down by this load, the dir mirrors should be able to survive it. Implementation Notes: Code Modifications The implementation of the bootstrap process is unfortunately mixed in with many types of directory activity. The process starts in update_consensus_networkstatus_downloads(), which initiates a single directory connection through directory_get_from_dirserver(). Depending on bootstrap state, a single directory server is selected and a connection is eventually made through directory_initiate_command_rend(). There appear to be a few options for altering this code to perform multiple connections. Without refactoring, one approach would be to make multiple calls to directory_initiate_command_routerstatus() from directory_get_from_dirserver() if the purpose is DIR_PURPOSE_FETCH_CONSENSUS and the only directory servers available are the authorities and the fallback dir mirrors. The code in directory_initiate_command_rend() would then need to be altered to maintain a list of the dircons created for this purpose as well as avoid immediately queuing the directory_send_command() request for the DIR_PURPOSE_FETCH_CONSENSUS purpose. A flag would need to be set on the dircon to be checked in connection_dir_finished_connecting(). The function connection_dir_finished_connecting() would need to be altered to examine the list of pending dircons, determine if this one is the first to complete, and if so, then call directory_send_command() to download the consensus and close the other pending dircons. An additional timer would need to be installed to re-call update_consensus_networkstatus_downloads() or a related helper after 5 seconds. connection_dir_finished_connecting() would cancel this timer. The helper would check the list of pending connections and ensure it never exceeds 15. -- Mike Perry

2 1

Proposal 208: IPv6 Exits Redux
by Nick Mathewson 11 Oct '12

11 Oct '12

Filename: 208-ipv6-exits-redux.txt Title: IPv6 Exits Redux Author: Nick Mathewson Created: 10-Oct-2012 Status: Open Target: 0.2.4.x 1. Obligatory Motivation Section [Insert motivations for IPv6 here. Mention IPv4 address exhaustion. Insert official timeline for official IPv6 adoption here. Insert general desirability of being able to connect to whatever address there is here. Insert profession of firm conviction that eventually there will be something somebody wants to connect to which requires the ability to connect to an IPv6 address.] 2. Proposal Proposal 117 has been there since coderman wrote it in 2007, and it's still mostly right. Rather than replicate it in full, I'll describe this proposal as a patch to it. 2.1. Exit policies Rather than specify IPv6 policies in full, we should move (as we have been moving with IPv4 addresses) to summaries of which IPv6 ports are generally permitted. So let's allow server descriptors to include a list of accepted IPv6 ports, using the same format as the "p" line in microdescriptors, using the "ipv6-policy" keyword. "ipv6-policy" SP ("accept" / "reject") SP PortList NL Exits should still, of course, be able to configure more complex policies, but they should no longer need to tell the whole world about them. After this ipv6-policy line is validated, it should get copied into a "p6" line in microdescriptors. This change breaks the existing exit enclave idea for IPv6, but the exiting exit enclave implementation never worked right in the first place. If we can come up with a good way to support it, we can add that back in. 2.2. Which addresses should we connect to? One issue that's tripped us up a few times is how to decide whether we can use IPv6 addresses. You can't use them with SOCKS4 or SOCKS4a, IIUC. With SOCKS5, there's no way to indicate that you prefer IPv4 or IPv6. It's possible that some SOCKS5 users won't understand IPv6 addresses. With this in mind, I'm going to suggest that with SOCKS4 or SOCKS4a, clients should always require IPv4. With SOCKS5, clients should accept IPv6. If it proves necessary, we can also add per-SOCKSPort configuration flags to override the above default behavior. See also partitioning discussion in Security Notes below. 2.3. Extending BEGIN cells. Prop117 (and the section above) says that clients should prefer one address or another, but doesn't give them a means to tell the exit to do so. Here's one. We define an extension to the BEGIN cell as follows. After the ADDRESS | ':' | PORT | [00] portion, the cell currently contains all [00] bytes. We add a 32-bit flags field, stored as an unsigned 32 bit value, after the [00]. All these flags default to 0, obviously. We define the following flags: bit 1 -- IPv6 okay. We support learning about IPv6 addresses and connecting to IPv6 addresses. 2 -- IPv4 not okay. We don't want to learn about IPv4 addresses or connect to them. 3 -- IPv6 preferred. If there are both IPv4 and IPv6 addresses, we want to connect to the IPv6 one. (By default, we connect to the IPv4 address.) 4..32 -- Reserved. As with so much else, clients should look at the platform version of the exit they're using to see if it supports these flags before sending them. 2.4. Minor changes to proposal 117 GETINFO commands that return an address, and which should return two, should not in fact begin returning two addresses separated by CRLF. They should retain their current behavior, and there should be a new "all my addresses" GETINFO target. 3. Security notes: Letting clients signal that they want or will accept IPv6 addresses creates two partitioning issues that didn't exist before. One is the version partitioning issue: anybody who supports IPv6 addresses is obviously running the new software. Another is option partitioning: anybody who is using a SOCKS4a application will look different from somebody who is using a SOCKS5 application. We can't do much about version partitioning, I think. If we felt especially clever, we could have a flag day. Is that necessary? For option partitioning, are there many applications whose behavior is indistinguishable except that they are sometimes configured to use SOCKS4a and sometimes to use SOCKS5? If so, the answer may well be to persuade as many users as possible to switch those to SOCKS5, so that they get IPv6 support and have a large anonymity set. IPv6 addresses are plentiful, which makes caching them dangerous if you're hoping to avoid tracking over time. (With IPv4 addresses, it's harder to give every user a different IPv4 address for a target hostname with a long TTL, and then accept connections to those IPv4 addresses from different exits over time. With IPv6, it's easy.) This makes proposal 205 especially necessary here.

1 0

Proposal 206: Preconfigured directory sources for bootstrapping
by Nick Mathewson 11 Oct '12

11 Oct '12

Filename: 206-directory-sources.txt Title: Preconfigured directory sources for bootstrapping Author: Nick Mathewson Created: 10-Oct-2012 Status: Open Target: 0.2.4.x Motivation and History: We've long wanted a way for clients to do their initial bootstrapping not from the directory authorities, but from some other set of nodes expected to probably be up when future clients are starting. We tried to solve this a while ago by adding a feature where we could ship a 'fallback' networkstatus file -- one that would get parsed when we had no current networkstatus file, and which we would use to learn about possible directory sources. But we couldn't actually use it, since it turns out that a randomly chosen list of directory caches from 4-5 months ago is a terrible place to go when bootstrapping. Then for a while we considered an "Extra-Stable" flag so that clients could use only nodes with a long history of existence from these fallback networkstatus files. We never built it, though. Instead, we can do this so much more simply. If we want to ship Tor with a list of initial locations to go for directory information, why not just do so? Proposal: In the same way that Tor currently ships with a list of directory authorities, Tor should also ship with a list of directory sources -- places to go for an initial consensus if you don't have a somewhat recent one. These need to include an address for the cache's ORPort, and its identity key. Additionally, they should include a selection weight. They can be configured with a torrc option, just like directory authorities are now. Whenever Tor is starting without a consensus, if it would currently ask a directory authority for a consensus, it should instead ask one of these preconfigured directory sources. I have code for this (see git branch fallback_dirsource_v2) in my public repository. When we deploy this, we can (and should) rip out the Fallback Networkstatus File logic. How to find nodes to make into directory sources: We could take any of three approaches for selecting these initial directory sources. First, we could try to vet them a little, with a light variant of the process we use for authorities. We'd want to look for nodes where we knew the operators, verify that they were okay with keeping the same IP for a very long time, and so forth. Second, we could try to pick nodes for listing with each Tor release based entirely on how long those nodes have been up. Anything that's been a high-reliability directory for a long time on the same IP (like, say, a year) could be a good choice. Third, we could blend the approach and start by looking for up-for-a-long-time nodes, and then also ask the operators whether their nodes are likely to stay running for a long time. I think the third model is best. Some notes on security: Directory source nodes have an opportunity to learn about new users connecting to the network for the first time. Once we have directory guards, that's going to be a fairly uncommon ability. We should be careful in any directory guard design to make sure that we don't fall back to the directory sources any more than we need to. See proposal 207.

1 0

10 days left for proposals for hard stuff [Re: Reminder: Big/tricky/interesting features for 0.2.4 need proposals by 10 October]
by Nick Mathewson 08 Oct '12

08 Oct '12

On Mon, Sep 17, 2012 at 11:47 AM, Nick Mathewson <nickm(a)freehaven.net> wrote: > Hi, all! > > From https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/024 : > > "October 10, 2012: Big feature proposal checkpoint. Any large > complicated feature which requires a design proposal must have its > first design proposal draft by this date." > > There are only 23 days till October 10. If there's something you want > in 0.2.4.x, and it is the kind of thing that needs a design proposal, > and there is no proposal yet, it could be time to start writing! > > What needs a design proposal? Generally: anything that involves a > change to the Tor protocol; anything whose security implications are > nontrivial and need discussion; and anything that will change any Tor > specification. Err on the side of "it needs a proposal." > > What is a "large complicated feature"? Please assume I'm going to be > extremely grumpy here, and err on the side of "it is big". If the > writeup of the proposal that explains how it works, or why to do it > this way is going to take more than a few paragraphs, it is probably > 'big' or 'complex'. > > (Note to would-be system-gamers: Please don't send a sketchy > incomplete draft as a placeholder to get your foot in the door. That's > not cool. If you don't have a draft ready, the feature can wait till > 0.2.5.) > > (Note also: a feature proposal by this deadline is a necessary > condition for getting your big/tricky/complicated feature into 0.2.4, > but not a sufficient condition. It also needs to have a working > implementation on schedule.) > > (Note finally: This is not a promise to not merge stuff that violates > this deadline, but I sure will be trying not to merge such stuff.) > > yrs, > -- > Nick

3 3

resistance to rubberhose and UDP questions
by Eugen Leitl 06 Oct '12

06 Oct '12

I've had an IRC session with the designer of cjdns (on cjdns) who made a few interesting points, and suggestions. Comments? Verbatim chat snip below. 18:03 <@cjd> if you took the components from cjdns, you could build a TOR like protocol which used UDP if possible and made connections much faster 18:04 <+eleitl> I wonder why they didn't choose UDP 18:05 <@cjd> you need to fall back on tcp in case you're firewalled to hell 18:05 <+eleitl> Apparently, they're thinking about it https://blog.torproject.org/blog/moving-tor-datagram-transport 18:06 <@cjd> problem with tor is (correct me if I) 18:06 <@cjd> 'm wrong) 18:06 <@cjd> the directory is signed by the tor foundation 18:07 <@cjd> so they can sign a fake directory and run a bunch of directory servers and when Alice connects to their directory server, they give her a bunch of fake nodes 18:07 <@cjd> run their own botnet with fake tor nodes so your circuit is always owned 18:07 <+eleitl> I don't really know for sure, but there's intrinsic trust to Tor developers, yes. 18:08 <+eleitl> You can run your own Tor network, though. 18:08 <+eleitl> Some botnets do that. 18:08 <@cjd> I trust them to make the software right, esp. since I could check if they did. 18:09 <@cjd> But a little arm twisting can change someone's motives pretty fast. 18:09 <+eleitl> Maintaining signing secrets is a problem. 18:09 <+eleitl> They should have used a P2P design. 18:10 <@cjd> If someone (with government hat?) tells you they can make your life hell... I wouldn't fault them for doing what the man says. 18:10 <@cjd> *wouldn't fault you 18:10 <+eleitl> I'll try bugging some Tor developers about that scenario, and see how they squirm. 18:11 <+eleitl> Also, the UDP connection thing. 18:11 <@cjd> You can "stack" your circuit setup packets if you're using UDP 18:11 <@cjd> stack -> all headers in the same packet 18:12 <@cjd> cjdns does the same thing 18:13 <+eleitl> Can I use snippage from this chat verbatim, or will I need to rephrase? 18:14 <@cjd> sure go ahead 18:14 <+eleitl> Thanks! 18:14 <@cjd> can only speak for myself ofc 18:14 <+eleitl> Sure.

6 7

how to configure the window size of tor browser
by jiang song 05 Oct '12

05 Oct '12

Hi, I want to make the tor browser with certain size, eg. 650*650, so as to do some test on some webpages I used command line like: App/Firefox/firefox --profile Data/profile -height 650 -width 650 but it doesn't work I tried to modify localstore.rdf in Data/profile, but it is dynamically changed by running the browser does anybody know how the size is set in the TBB and which configuration file can I modify ? thanks

2 3

about plugins enable/disable and window size
by esolve esolve 05 Oct '12

05 Oct '12

Hi, I want to control the window size of Firefox browser and whether a plugin is enabled or disabled when a new firefox browser is opened. Currently I'm using a Tor browser which is a modified firefox browser, when I open it, the window size is set by default and flash plugin is by default disabled. even if I try to set them mannually, the next time I open a new firefox window, the setting return to default. I want to know which configuration files or what part of the source codes are controling this default setting, and I want to modify it. can anybody give me some solution or hints? thanks!

1 0

The Tor Project is looking for a Pluggable Transport developer
by Roger Dingledine 05 Oct '12

05 Oct '12

We have funding initially for part-time work, and hopefully it will grow into full-time work. Please spread the word! This job is for the development and maintenance of the flash proxy circumvention system, with a focus on deployment and getting tools in the hands of users: https://crypto.stanford.edu/flashproxy/ https://gitweb.torproject.org/flashproxy.git/ If it goes well, we might have you branch out into improving usability and deployability of other Tor pluggable transports. Applicants must be familiar with Python, JavaScript, and web technologies, particularly WebSocket. You will do usability testing and be in charge of producing binary packages of client software for GNU/Linux, Windows, and OS X. The system's supporting infrastructure is already in place, but may require changes depending on the future development of the client programs. There also is the potential for the development and implementation of new covert rendezvous methods that may have broader use outside the flash proxy system. You will be assisted and mentored by David Fifield, the primary developer of the flash proxy software and co-author of its research paper, and all-around good guy. See the job posting for information on how to apply and what you need to send in with your application: https://www.torproject.org/about/jobs-pluggabletransport Thanks! --Roger

1 0

resistance to rubberhose and UDP questions
by Eugen Leitl 04 Oct '12

04 Oct '12

I've had an IRC session with the designer of cjdns (on cjdns) who made a few interesting points, and suggestions. Comments? Verbatim chat snip below. 18:03 <@cjd> if you took the components from cjdns, you could build a TOR like protocol which used UDP if possible and made connections much faster 18:04 <+eleitl> I wonder why they didn't choose UDP 18:05 <@cjd> you need to fall back on tcp in case you're firewalled to hell 18:05 <+eleitl> Apparently, they're thinking about it https://blog.torproject.org/blog/moving-tor-datagram-transport 18:06 <@cjd> problem with tor is (correct me if I) 18:06 <@cjd> 'm wrong) 18:06 <@cjd> the directory is signed by the tor foundation 18:07 <@cjd> so they can sign a fake directory and run a bunch of directory servers and when Alice connects to their directory server, they give her a bunch of fake nodes 18:07 <@cjd> run their own botnet with fake tor nodes so your circuit is always owned 18:07 <+eleitl> I don't really know for sure, but there's intrinsic trust to Tor developers, yes. 18:08 <+eleitl> You can run your own Tor network, though. 18:08 <+eleitl> Some botnets do that. 18:08 <@cjd> I trust them to make the software right, esp. since I could check if they did. 18:09 <@cjd> But a little arm twisting can change someone's motives pretty fast. 18:09 <+eleitl> Maintaining signing secrets is a problem. 18:09 <+eleitl> They should have used a P2P design. 18:10 <@cjd> If someone (with government hat?) tells you they can make your life hell... I wouldn't fault them for doing what the man says. 18:10 <@cjd> *wouldn't fault you 18:10 <+eleitl> I'll try bugging some Tor developers about that scenario, and see how they squirm. 18:11 <+eleitl> Also, the UDP connection thing. 18:11 <@cjd> You can "stack" your circuit setup packets if you're using UDP 18:11 <@cjd> stack -> all headers in the same packet 18:12 <@cjd> cjdns does the same thing 18:13 <+eleitl> Can I use snippage from this chat verbatim, or will I need to rephrase? 18:14 <@cjd> sure go ahead 18:14 <+eleitl> Thanks! 18:14 <@cjd> can only speak for myself ofc 18:14 <+eleitl> Sure.

1 0

Damian's Status Report - September 2012
by Damian Johnson 04 Oct '12

04 Oct '12

Hi all. As is often the case work and such meant not too much time for tor. For September all I have to report is... * Network Status Document Parsing This has been my main focus for September and it's still not finished... but it's close! Version 3 document parsing just has a couple days of work left, then abstracting it to cover v2 documents and microdescriptors should be relatively easy-ish. I'm really looking forward to merging this feature branch. It has grown quite monstrous... https://gitweb.torproject.org/user/atagar/stem.git/shortlog/refs/heads/docu… * Stem Documentation Hosting For a while now I've had a TODO item for making a nightly cron that built and hosted Stem's new sphinx documentation. I was about to do this when I recalled that meejah once recommended ReadTheDocs, a service that does... well, exactly that. After a few minor bumps [1][2][3] it's now live... https://stem.readthedocs.org/ We definitely need to put effort into making them more reader friendly. At present it's just a dump of all the pydocs which, while informative, is actually a bit overwhelming for new users. Module summary pages would greatly help. * MAPADDRESS Support Ravi submitted a patch for adding MAPADDRESS support to stem's controller. It's a nice addition, especially the integ test... https://trac.torproject.org/6951 * Arm Issues Looked into a couple arm issues... - Tor's start time didn't show up if the system has proc contents but we fail to parse it [5]. - Can't connect when using a control socket with password auth [6]. * Updated Dev Wiki In response to a potential volunteer I wrote a summary of several development tasks [4]. Updated stem's wiki with what was in that email... https://trac.torproject.org/projects/tor/wiki/doc/stem Cheers! -Damian [1] https://github.com/rtfd/readthedocs.org/issues/255 [2] https://github.com/rtfd/readthedocs.org/issues/256 [3] https://gitweb.torproject.org/stem.git/commitdiff/705b61674e8cec9e5608a32c6… [4] https://lists.torproject.org/pipermail/tor-dev/2012-September/004021.html [5] https://trac.torproject.org/6862 [6] https://trac.torproject.org/6881

1 0