<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div dir="ltr"><div>Hi Neel,</div><div><br></div><div>Thanks for this proposal.</div><div><br></div></div><div dir="ltr">On 26 Jun 2019, at 11:15, <a href="mailto:neel@neelc.org">neel@neelc.org</a> wrote:<br></div><blockquote type="cite"><div dir="ltr"><span></span><br><span>I have a new proposal: A Tor Implementation of IPv6 Happy Eyeballs</span><br><span></span><br><span>This is to implement Tor IPv6 Happy Eyeballs and acts as an alternative to Prop299 as requested here: <a href="https://trac.torproject.org/projects/tor/ticket/29801">https://trac.torproject.org/projects/tor/ticket/29801</a></span><br><span></span><br><span>The GitHub pull request is here: <a href="https://github.com/torproject/torspec/pull/87">https://github.com/torproject/torspec/pull/87</a></span><br></div></blockquote><br><div>Here's the proposal content, with my comments:</div><div><br></div><div></div><div></div><blockquote type="cite"><div><span style="background-color: rgba(255, 255, 255, 0);">Filename: 306-ipv6-happy-eyeballs.txt</span></div><div>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">Title: A Tor Implementation of IPv6 Happy Eyeballs</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">Author: Neel Chauhan</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">Created: 25-Jun-2019</span></p>
<p class="p1"></p></div></blockquote><span style="background-color: rgba(255, 255, 255, 0);">Supercedes: 299</span><br><blockquote type="cite"><div><p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">Status: Open</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">Ticket: <a href="https://trac.torproject.org/projects/tor/ticket/29801">https://trac.torproject.org/projects/tor/ticket/29801</a></span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">1. Introduction</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> As IPv4 address space becomes scarce, ISPs and organizations will deploy</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> IPv6 in their networks. Right now, Tor clients connect to guards using</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> IPv4 connectivity by default.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> When networks first transition to IPv6, both IPv4 and IPv6 will be enabled</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> on most networks in a so-called "dual-stack" configuration. This is to not</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> break existing IPv4-only applications while enabling IPv6 connectivity.</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> However, IPv6 connectivity may be unreliable and clients should be able</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> to connect to the guard using the most reliable technology, whether IPv4</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> or IPv6.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> In ticket #27490, we introduced the option ClientAutoIPv6ORPort which</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> lets a client randomly choose between IPv4 or IPv6. However, this</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> random decision does not take into account unreliable connectivity</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> or falling back to the competing IP version should one be unreliable</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> or unavailable.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> One way to select between IPv4 and IPv6 on a dual-stack network is a</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> so-called "Happy Eyeballs" algorithm as per RFC 8305. In one, a client</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> attempts an IP family, whether IPv4 or IPv6. Should it work, the client</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> sticks with the working IP family. Otherwise, the client attempts the</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> opposing version. This means if a dual-stack client has both IPv4 and</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> IPv6, and IPv6 is unreliable, the client uses IPv4, and vice versa.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> In Proposal 299, we have attempted a IP fallback mechanism using failure</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> counters and preferring IPv4 and IPv6 based on the state of the counters.</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> However, Prop299 was not standard Happy Eyeballs and an alternative,</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> standards-compliant proposal was requested in [P299-TRAC] to avoid issues</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> from complexity caused by randomness.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> This proposal describes a Tor implementation of Happy Eyeballs and is</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> intended as a successor to Proposal 299.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">2. Address Selection</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> To be able to handle Happy Eyeballs in Tor, we will need to modify the</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> data structures used for connections to guards, namely the extend info</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> structure.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> The extend info structure should contain both an IPv4 and an IPv6 address.</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> This will allow us to try IPv4 and the IPv6 addresses should both be</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> available on a relay and the client is dual-stack.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> When parsing relay descriptors and filling in the extend info data</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> structure, we need to fill in both the IPv4 and IPv6 address if they both</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> are available. If only one family is available for a relay (IPv4 or IPv6),</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> we should fill in the address for available family and leave the opposing</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> family null.</span></p></div></blockquote><div>When we implement this feature in tor, it would be a good idea to call the</div><div>two addresses "preferred" and "alternate" address. With this design,</div><div>the low-level connection code doesn't have to know about reachable</div><div>addresses, or IPv4/IPv6 preferences. It just has to try them in order.</div><blockquote type="cite"><div>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">3. Connecting To A Relay</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> When a client connects to a guard using an extend info data structure, we</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> should first check if there is an existing authenticated connection. If</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> there is, we should use it.</span></p></div></blockquote><div>Tor's code already does this check: we won't need to change it.</div><blockquote type="cite"><div>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> If there is no existing authenticated connection for an extend info, we</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> should attempt to connect using the first available, allowed, and preferred</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> address. At the time of writing, this is IPv4.</span></p></div></blockquote><div>That's not quite true: most clients use IPv4 by default, but they can be</div><div>configured to prefer IPv6, or only allow certain addresses. And bridge clients</div><div>automatically use IPv6 if they are configured with an IPv6 bridge.</div><blockquote type="cite"><div>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> We should also schedule a timer for connecting using the other address</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> should one be available and allowed, and the first attempted version</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> fails. This should be higher than most client's successful TLS</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> authentication time. I propose that the timer is 15 seconds. The reason</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> for this is to accommodate high-latency connections such as dial-up and</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> satellite.</span></p></div></blockquote><div>In the worst case scenario, users see Tor Browser hang for 15 seconds</div><div>before it makes a successful connection. That's not acceptable.</div><div><br></div><div>Depending on their location, most tor clients authenticate to the first</div><div>hop within 0.5-1.5 seconds. So I suggest we use a 1.5 second delay:</div><div><a href="https://metrics.torproject.org/onionperf-buildtimes.html">https://metrics.torproject.org/onionperf-buildtimes.html</a></div><div><br></div><div>In RFC 8305, the default delay is 250 milliseconds, and the maximum</div><div>delay is 2 seconds. So 1.5 seconds is reasonable for TLS and tor link</div><div>authentication.</div><div><a href="https://tools.ietf.org/html/rfc8305#section-8">https://tools.ietf.org/html/rfc8305#section-8</a></div><div><br></div><div>(This delay will mainly affect initial bootstrap, because all of Tor's</div><div>other connections are pre-emptive, or re-used.)</div><div><br></div><div>A small number of clients may do wasted authentication.</div><div>That's ok. Tor already does multiple bootstrap and guard connections.</div><div><br></div><div>We have talked about this design in the team over the last few months.</div><div>Our key insights are that:</div><div>* TCP connections are cheap, but TLS is expensive</div><div>* most failed TCP connections fail immediately in the kernel, some</div><div> fail quickly with a response from the router, and others are blackholed</div><div> and time out</div><div>* it's unlikely that a client will fail to authenticate to a relay over one</div><div> IP version, but succeed over the other IP version, because the directory</div><div> authorities authenticate to each relay when they check reachability</div><div>* some censorship systems only break authentication over IPv4,</div><div> but they are rare</div><div><br></div><div>So here are some alternative designs:</div><div><br></div><div>1. Tor connects to the preferred address and tries to authenticate.</div><div> On failure, or after a 1.5 second delay, it connects to the alternate address</div><div> and tries to authenticate.</div><div> On the first successful authentication, it closes the other connection.</div><div><br></div><div>This design places the least connection load on the network, but might add</div><div>a bit of extra TLS load.</div><div><br></div><div>2. Tor connects via TCP to the preferred address.</div><div> On failure, or after a 250 ms delay, it connects via TCP to the alternate</div><div> address.</div><div> On the first TCP success, tor attempts to authenticate immediately.</div><div> On authentication failure, or after a 1.5 s delay, tor attempts to</div><div> authenticate over the second TCP connection.</div><div> <span style="background-color: rgba(255, 255, 255, 0);">On the first successful authentication, it closes the other connection.</span></div><div><br></div><div>This design is the most reliable for clients, but it also puts a bit more</div><div>connection load on dual-stack guards and authorities.</div><div><br></div><div><div><span style="background-color: rgba(255, 255, 255, 0);">3. Tor connects via TCP to the preferred address.</span></div><div><span style="background-color: rgba(255, 255, 255, 0);"> On failure, or after a 250ms delay, it connects via TCP to the alternate</span></div><div><span style="background-color: rgba(255, 255, 255, 0);"> address.</span></div><div><span style="background-color: rgba(255, 255, 255, 0);"> On the first TCP success, tor attempts to authenticate, and closes the</span></div></div><div><span style="background-color: rgba(255, 255, 255, 0);"> other connection.</span></div><div><span style="background-color: rgba(255, 255, 255, 0);"><br></span></div><div><span style="background-color: rgba(255, 255, 255, 0);">This design looks similar to a web browser's implementation of Happy</span></div><div><span style="background-color: rgba(255, 255, 255, 0);">Eyeballs, because it closely follows the RFC. That might help hide tor</span></div><div><span style="background-color: rgba(255, 255, 255, 0);">from censors. It adds some extra connection load, but no extra TLS load.</span></div><div><br></div><div>I suggest that we put all 3 alternative designs in the proposal, but start</div><div>by implementing and testing alternative 1.</div><div><br></div><div>When we implement this code, let's put the happy eyeballs part in a</div><div>separate module, as much as possible. That helps us review the code,</div><div>and make sure it has good test coverage. It also stops existing files and</div><div>functions getting too big.</div><blockquote type="cite"><div>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">4. Handling Connection Successes And Failures</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> Should a connection to a guard succeed and is authenticated via TLS, we</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> can then use the connection. In this case, we should cancel all other</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> connection timers and in-progress connections. Cancelling the timers is</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> so we don't attempt new unnecessary connections when our existing</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> connection is successful, preventing denial-of-service risks.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> However, if we fail all available and allowed connections, we should tell</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> the rest of Tor that the connection has failed. This is so we can attempt</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> another guard relay.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">5. Acknowledgments</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> Thank you so much to teor for the discussion of the happy eyeballs proposal.</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> I wouldn't have been able to do this has it not been for your help.</span></p>
<p class="p2"><span style="background-color: rgba(255, 255, 255, 0);"><span class="s1"></span><br></span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);">6. Appendix</span></p>
<p class="p1"><span class="s1" style="background-color: rgba(255, 255, 255, 0);"> [P299-TRAC]: <a href="https://trac.torproject.org/projects/tor/ticket/29801">https://trac.torproject.org/projects/tor/ticket/29801</a></span></p></div></blockquote><br><div>T</div></body></html>