- tor-dev - lists.torproject.org

txtorcon 20.0.0
by meejah 31 Mar '20

31 Mar '20

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I'm pleased to announce txtorcon 20.0.0. This fixes a few bugs and officially deprecates Python 2 support. * Use real GeoIP database or nothing (https://github.com/meejah/txtorcon/issues/250) * Change abstract base classes import in preperation for Python 3.8 (thanks @glowatsk) * Python 3.4 is no longer supported * Python 2 is deprecated; all new code should be Python 3. Support for Python 2 will be removed in a future release. You can download the release from PyPI or GitHub (or of course "pip install txtorcon"): https://pypi.python.org/pypi/txtorcon/20.0.0 https://github.com/meejah/txtorcon/releases/tag/v20.0.0 Releases are also available from the hidden service: http://timaq4ygg2iegci7.onion/txtorcon-20.0.0.tar.gz http://timaq4ygg2iegci7.onion/txtorcon-20.0.0.tar.gz.asc Or via a "version 3" service: http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/txtor… http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/txtor… You can verify the sha256sum of both by running the following 4 lines in a shell wherever you have the files downloaded: cat <<EOF | sha256sum --check 122cd081786396f57718adda1c1a5eb01b8e9a4832fcd140918b45c10359377a txtorcon-20.0.0.tar.gz 5c1145566a8733105da851b794e7b15cb56cea48693635b55c4bddfb79fe629d txtorcon-20.0.0-py2.py3-none-any.whl EOF thanks, meejah -----BEGIN PGP SIGNATURE----- iQFFBAEBCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAl6EDnURHG1lZWphaEBt ZWVqYWguY2EACgkQwmAoAxKAaadv4AgAkIny/Pt55kEppkxlqu4/B4ddfU7oQNLD 8lwoYyUZbYtS8XwiBaqY6SR/UU9MhhswdoPU50WBq3h4Im+3jrmm7MbVD4uR6Uvt BnZIMzCJlQNcGpfbaGmtyrtbBNazQLTDOAxxQ7WM/MD8rip1zlP8MeL7yBgE3VEC JInw9DWN+fbQhbsTQgdiLLLhBXOFqZuXgz7auPnGzu+5LsIeJd6ZnWzuQ6TlzL3R rLgrMAasxTMzKoJB51jmDpCOeaj4b68XK+MY130/xSdENb2ZyGbuSF4LpZUjs4RU Sqiwtevu/bT/th0XBqcg79RHnwGLK/g1pJRB27RaOVbuh6f+lENc4Q== =4q9v -----END PGP SIGNATURE-----

1 0

Improving onion service availability during DoS using anonymous credentials
by George Kadianakis 30 Mar '20

30 Mar '20

Hello list, there has been lots of discussions about improving onion service availability under DoS conditions. Many approaches have been proposed [OOO] but only a few have been tried and even fewer show any real improvements to the availability of the service. An approach that we've been considering is the use of anonymous credentials as a way to prioritize good clients from bad clients. The idea is that the service gives tokens to clients it believes to be good, and prioritizes client with tokens over clients without tokens whenever possible. This is a post to start a discussion of how such approaches could work and whether they are worth pursuing futher. == Preliminaries == === When should the access control take place? === Doing DoS defenses with anon credentials is all about enforcing access control at the right point of the protocol so that the amplification factor of evil clients gets cut as early as possible. Very roughly the phases of the onion service protocol are: descriptor fetch phase, intro phase, rendezvous phase. Let's see how those look like for the purposes of access control: - Doing the access control during the descriptor fetch stage is something worth considering because it's the first phase of the protocol and hence the earliest and best place to soak up any damage from evil clients. There is already a form of optional access control implemented here called "client authorization" and it's worth thinking of what's lacking to make it useful against DoS attackers. I'm gonna address this in section [CLIENTAUTH]. - Doing the access control during the introduction phase is another fruitful approach. Blocking bad clients during introduction means that they dont get to force the service to create a costly rendezvous circuit, and since services have a long-term circuit open towards the intro points it makes it easier for services to pass access control related data to the intro point. This is actually the approach we are gonna be talking most about in this post. - Finally, doing the access control during the rendezvous phase is way too late since by that time the onion service has already spent lots of resources catering the evil client, so let's ignore that. === Entities of an anonymous credential system === Anonymous credential systems traditionally have three entities that concern us: - The Issuer: the entity who issues the credentials/tokens - The Prover: the entity who collects tokens and uses them to get access - The Verifier: the entity who verifies that tokens are legit and grants/restricts access In the world of onion services, the Issuer is naturally the onion service, and the Prover is the onion service client. The Verifier could either be the onion service itself or its introduction points. We will see below how this could work and the relevant tradeoffs. +--------+ +------------+ +--------------------+ | Client |<-+-+-+-->|Intro point |<--+---+-->|Onion service | |(Prover)| |(Verifier?) | |(Issuer)(Verifier?) | +--------+ +------------+ +--------------------+ === How do tokens get around? === A main question here is "How do good clients end up with tokens?". For the purposes of this post, we will assume that clients get these tokens in an out of band fashion. For example, a journalist can give tokens to her sources over Signal so they can use them with Securedrop. Or a forum operator can give tokens to old-time members of the forum to be used during a DoS. A natural chicken-and-egg problem occurs here since how is an onion service supposed to give tokens to its users if it's unreachable because of a DoS? We realize this is a big problem and we are not sure exactly how to solve it. This problem naturally limits the use of anonymous credential solutions, and sorta makes them a second-layer of defense since it assumes a first-layer of defense that allows operators to pass tokens to the good people. A first-layer approach here could perhaps look like PrivacyPass where users get tokens by solving CAPTCHAs. == Anonymous credentials == By surveying the anonymous credential literature we have found various types of anonymous credential schemes that are relevant for us: - Discrete-logarithm-based credentials based on blind signatures: This is a class of anon credential schemes that allow us to separate the verifier from the issuer. In particular this means that we can have the service issue the tokens, but the introduction point being the verifier. They are usually based on blind signatures like in the case of Microsoft's U-Prove system [UUU]. - Discrete-logarithm-based credentials based on OPRF: Another approach here is to use OPRF constructions based on the discrete logarithm problem to create an anonymous credential scheme like in the case of PrivacyPass [PPP]. The downside, IIUC, is that in PrivacyPass you can't have a different issuer and verifier so it's the onion service that needs to do the token verification restricting the damage soaking potential. - KVAC (Keyed-Verification Anonymous Credentials): KVAC-based credentails have comparable performance to Dlog-based credentials, and are also easier to create security proofs for due to them being based on symmetric primitives like algebraic MAC constructions [FFF]. The downside is that they assume that the verifier and the issuer is the same entity (the onion service). KVACs are used by Signal for protecting their group chat metadata [GGG]. Which construction and scheme we choose depends on our constraints and the parameters we are trying to minmax. So let's try to explore these constraints and parameters some more. == Space constraints == Let's assume that a client found some anonymous credentials for a service. How will clients present their credentials to the verifier, being either the service or the intro point? The natural approach here is for the client to include their token as part of the INTRODUCE1 cell. === How much space is available in INTRODUCE1 cells? [SPACE_CONSTRAINTS] === >From a brief experiment, I see that the INTRODUCE1 cell payload total size is 498 bytes, and we are already using 310 bytes from it, so that leaves about 188 bytes for use. The cell is extensible using cell extensions that can go either to the intro point [III] or to the service directly [EEE]. === How big are anonymous credentials redemptions? === >From looking at the literature it seems unlikely that those anonymous credentials can be presented given the size limitations from above. In particular here is the size needed for presenting/redeeming an anonymous credential in the various schemes: - Privacy Pass [PPP]: Credential redemption takes 396 bytes - Signal [GGG]: Their paper does not say exactly, but by peeking into their code it seems like presenting the credential takes about 620 bytes [CCC]. So if I'm not mistaken, it seems like we will need another way to present anonymous credentials to the service or intro point. === How can we trasmit these credentials then? === The unfortunate fact here is that presenting those credentials does not fit into the remaining space of an INTRODUCE1 cell, but it also wouldn't even fit if we created an entirely new relay cell for this purpose (say INTRO_REDEEM) since the relay cell payload is about 498 bytes and all the credentials above, except from PrivacyPass (!), are bigger than that. This means that we will either need to use a space-compact anonymous credential scheme, or to implement wide relay cells as suggested in ticket #33650 (which seems like lots of work!). == Computational constraints [COMPUTATIONAL_CONSTRAINTS] == Another big topic for consideration here is the computational resources required for the various operations of the scheme. In particular, how hard it is to issue such tokens, present such tokens and to verify such tokens. Different schemes have different computational profiles. I'm not gonna attempt to summarize the literature here because it will take me more time than I have right now. However, it's clear that our main priority here is a lightweight verification procedure, so that services or intro points spend as little energy as possible validating (potentially evil) tokens. It's imperative that verifying tokens is orders of magnitude more lightweight than the regular job that a service would do if credentials did not exist (i.e. doing path selection, creating a rendezvous circuit, doing the crypto, etc.) Our second priority here is a lighweight token issuance procedure, so that services don't spend too much time issuing tokens, especially if this is done in an automatic real-time manner that can be abused by an adversary. And finally, we don't particularly care about the token presentation procedure being lightweight, since this is a procedure that the attacker will also have to do and hence we want to incur as much costs to them as possible. == Discussion == ==== What's lacking from HSv3 client authorization [CLIENTAUTH] ==== It's really important to understand what's lacking from descriptor-level client authorization [QQQ] with regards to DoS defences before we spend time and energy implementing any anonymous credential approach. To use the descriptor-level client authorization feature for DoS protection the onion service operator creates a mirror of the website that is only reachable using descriptor-level client auth and then they pass client auth tokens to the clients they trust to be good natured. The above system can also be automated and turned into a web service. However this system has a bunch of drawbacks: - Given the implicit and anonymous nature of the descriptor-level client authorization system, if an evil actor gets their hand in a client auth token and they start DoSing the service, then the operator has no means to distinguish that client from any other client. Hence there is no way to identify individual bad clients and revoke them. This is the biggest issue with descriptor-based client authorization for the purposes of DoS defense. - Managing the access list (adding and removing clients) in the descriptor-level client authorization system is an expensive process because it requires uploading a new set of descriptors. Doing this too often (in an automated way) can cause lots of network traffic and also perhaps race conditions with clients who fetch descriptors. - Because the authentication happens by adding new lines on the descriptor, there is a fundamental limit on how many authed clients it can support. In particular, with some back-of-the-envelope calculations [ZZZ] an onion service can support a max of 250 authed clients this way. However, there is nothing stopping the operator from passing the same token to multiple clients and managing them as groups. The above issues would be addressed with intro-level client authorization since the authentication is explicit and done in real-time. It's worth thinking of the above drawbacks more and how we can bypass them to do something useful with the current system before we jump into new complexities and tradeoffs. --- Thanks for reading and looking forward to your feedback :) [OOO]: https://trac.torproject.org/projects/tor/ticket/31223 [ZZZ]: https://lists.torproject.org/pipermail/tor-dev/2016-November/011658.html [QQQ]: https://eprint.iacr.org/2013/516.pdf [PPP]: https://www.petsymposium.org/2018/files/papers/issue3/popets-2018-0026.pdf [UUU]: https://www.microsoft.com/en-us/research/project/u-prove/ [FFF]: https://eprint.iacr.org/2013/516.pdf [GGG]: https://signal.org/blog/signal-private-group-system/ [III]: https://github.com/torproject/torspec/blob/f81b1e6cc53b91abb3ae206807bc371f… [EEE]: https://github.com/torproject/torspec/blob/f81b1e6cc53b91abb3ae206807bc371f… [CCC]: https://github.com/signalapp/zkgroup/blob/4294e428216113d81f5c0cb50f578797d…

2 2

Chutney tests fail for tor/maint-0.3.5 (bug #33677)
by c 29 Mar '20

29 Mar '20

Hi, For the Outreachy.org internship project I decided to take on bug #33677 [1]. I followed steps to run `make test-network-all` both on tor master and maint-0.3.5 branches, using Chutney master branch. I tested before and after tweaking value of HS_WAIT_FOR_UNCHECKED_DIR_INFO just to ensure my testing environment was sane; all tests passed (except for two that were skipped on master: mixed+hs-v23 and mixed+hs-v23-ipv6, and one on 0.3.5: mixed+hs-v2, I assume this is expected though) on the first go, on both branches of Tor. Tests pass on Tor master after setting HS_WAIT_FOR_UNCHECKED_DIR_INFO to 0, but on maint-0.3.5 I've run tests a few times (both with and without CHUTNEY_DEBUG set) each time with a different outcome. Some of the tests which fail: basic-min [2], single-onion-v23 [3], bridges+ipv6-min [4], and ipv6-exit-min [5]. I have excerpted logs at the bottom of this message, narrowed down to test failures; let me know if I need to provide full logs or additional information. I'll keep digging on this issue; I just wanted to show my current progress and share what I hope to be useful information. Caitlin [1]: <https://trac.torproject.org/projects/tor/ticket/33677> "Stop waiting a set time for onion service descriptors" [2]: basic-min.log [...] Connecting: Exit to 127.0.0.1:4747 via client localhost:9005 DEBUG: socket 5 connecting to ('localhost', 9005)... DEBUG: Registering send-data-11 DEBUG: Registering check-12 Transmitting Data: DEBUG: Socks 4a request to 127.0.0.1:4747 DEBUG: Test status: {'send-data-11': 'not done', 'check-12': 'not done'}: 2/0/0 DEBUG: Test status: {'send-data-11': 'not done', 'check-12': 'not done'}: 2/0/0 DEBUG: Done with run(); all_done == False and failure_count == 0 Transmission: Failure [...] chutney verify round 1/1 failed Chutney failed 1 bootstraps; we may have a problem here. FAIL basic-min (exit status: 1) [3]: single-onion-v23.log [...] Connecting: HS to bkjqakom5elyh45a.onion:5858 (127.0.0.1:4747) via client localhost:9005 DEBUG: socket 7 connecting to ('localhost', 9005)... DEBUG: Registering send-data-21 DEBUG: Registering check-22 HS to moymkdharroenjd6zarfurdwmrr55j2zune6k7wafyfakwy2cis327id.onion:5858 (127.0.0.1:4747) via client localhost:9005 DEBUG: socket 8 connecting to ('localhost', 9005)... DEBUG: Registering send-data-23 DEBUG: Registering check-24 Transmitting Data: DEBUG: Socks 4a request to bkjqakom5elyh45a.onion:5858 DEBUG: Socks 4a request to moymkdharroenjd6zarfurdwmrr55j2zune6k7wafyfakwy2cis327id.onion:5858 DEBUG: new client from 127.0.0.1:54314 (fd=9) DEBUG: successfully connected (fd=8) DEBUG: Succeeded send-data-23 DEBUG: successful verification DEBUG: Succeeded check-24 DEBUG: Test status: {'send-data-21': 'not done', 'check-22': 'not done', 'send-data-23': 'success', 'check-24': 'success'}: 2/2/0 DEBUG: Test status: {'send-data-21': 'not done', 'check-22': 'not done', 'send-data-23': 'success', 'check-24': 'success'}: 2/2/0 DEBUG: Done with run(); all_done == False and failure_count == 0 Transmission: Failure [...] chutney verify round 1/1 failed Chutney failed 1 bootstraps; we may have a problem here. FAIL single-onion-v23 (exit status: 1) [4]: bridges+ipv6-min.log [...] Couldn't launch test005bc command '/home/c/git/tor/src/app/tor -f /home/c/git/chutney/net/nodes/005bc/torrc': exit 1, output 'Mar 28 10:47:36.717 [notice] Tor 0.3.5.10-dev (git-38e07b88fac51f2a) running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1d, Zlib 1.2.11, Liblzma 5.2.4, and Libzstd 1.3.7. Mar 28 10:47:36.717 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning Mar 28 10:47:36.717 [notice] Read configuration file "/home/c/git/chutney/net/nodes/005bc/torrc". Mar 28 10:47:36.721 [warn] You have used DirAuthority or AlternateDirAuthority to specify alternate directory authorities in your configuration. This is potentially dangerous: it can make you look different from all other Tor users, and hurt your anonymity. Even if you've specified the same authorities as Tor uses by default, the defaults could change in the future. Be sure you know what you're doing. Mar 28 10:47:36.721 [warn] TestingTorNetwork is set. This will make your node almost unusable in the public Tor network, and is therefore only advised if you are building a testing Tor network! Mar 28 10:47:36.722 [notice] Opening Socks listener on 127.0.0.1:9005 Mar 28 10:47:36.722 [warn] Could not bind to 127.0.0.1:9005: Address already in use. Is Tor already running? Mar 28 10:47:36.722 [notice] Opening Control listener on 127.0.0.1:8005 Mar 28 10:47:36.722 [warn] Could not bind to 127.0.0.1:8005: Address already in use. Is Tor already running? Mar 28 10:47:36.722 [notice] Opening Control listener on /home/c/git/chutney/net/nodes/005bc/control Mar 28 10:47:36.722 [notice] Opened Control listener on /home/c/git/chutney/net/nodes/005bc/control Mar 28 10:47:36.722 [notice] Closing partially-constructed Control listener on /home/c/git/chutney/net/nodes/005bc/control:0 Mar 28 10:47:36.722 [warn] Failed to parse/validate config: Failed to bind one of the listener ports. Mar 28 10:47:36.722 [err] Reading config failed--see warnings above. ' [...] Launching chutney using Python 3.7.6 DEBUG: Tor version 0.3.5.10-dev (git-38e07b88fac51f2a). test000a is running with PID 2870: Tor 0.3.5.10-dev (git-38e07b88fac51f2a) test001a is running with PID 2873: Tor 0.3.5.10-dev (git-38e07b88fac51f2a) test002ba is running with PID 2911: Tor 0.3.5.10-dev (git-38e07b88fac51f2a) test003r is running with PID 2879: Tor 0.3.5.10-dev (git-38e07b88fac51f2a) test004br is running with PID 2882: Tor 0.3.5.10-dev (git-38e07b88fac51f2a) test005bc is stopped: Tor 0.3.5.10-dev (git-38e07b88fac51f2a) 5/6 nodes are running [...] bootstrap-network.sh failed Chutney failed 1 bootstraps; we may have a problem here. FAIL bridges+ipv6-min (exit status: 1) [5]: ipv6-exit-min.log [...] Connecting: Exit to 127.0.0.1:4747 via client localhost:9005 DEBUG: socket 5 connecting to ('localhost', 9005)... DEBUG: Registering send-data-11 DEBUG: Registering check-12 Transmitting Data: DEBUG: Socks 4a request to 127.0.0.1:4747 DEBUG: Test status: {'send-data-11': 'not done', 'check-12': 'not done'}: 2/0/0 DEBUG: Test status: {'send-data-11': 'not done', 'check-12': 'not done'}: 2/0/0 DEBUG: Done with run(); all_done == False and failure_count == 0 Transmission: Failure [...] chutney verify round 1/1 failed Chutney failed 1 bootstraps; we may have a problem here. FAIL ipv6-exit-min (exit status: 1)

2 1

Walking onions status update: week 3 notes
by Nick Mathewson 29 Mar '20

29 Mar '20

Walking Onions: week 3 update On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send out these updates once a week. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html Week 2: Specifying details of SNIP and ENDIVE formats, in CDDL. https://lists.torproject.org/pipermail/tor-dev/2020-March/014181.html You might like to check out those updates, and their references, if this update doesn't make sense to you. === As you might recall, the SNIPs need to be nice and small, but they need to be completely self-contained. The ENDIVE, on the other hand, needs to be diff-friendly, and compressible. Relays need to derive all the SNIPs from the ENDIVE. The ENDIVE and SNIP formats that I worked on during last week [SNIPFMT] was designed to meet these criteria. This week, I wrote up the algorithms to extract signed SNIPs from an ENDIVE. This is in section 4 [RECONST] of the specs in my work-in-progress repository [GITREPO] This was mainly straightforward consequence from the work I did last week, but there were a few things that turned up. One trickier aspect of this work is that we want to move most of the decision-making to the authorities, and have the relays simply do an expansion algorithm. Updating all the relays would be slow, so relays need to be able to handle new index types and data types without fully understanding them. That means that we need to specify an algorithm that can work flexibly with many future kinds of data. Another tricky aspect is that the data which the relays need to put into SNIPs all needs to be signed, so all of that data needs to come out the same when relays calculate it as the when the authorities calculate it. Thus, all the algorithms for building Merkle trees and indices and SNIPRouterData bodies need to be deterministic. I think that I got all of this relatively right in [RECONST], but there are probably some mistakes hiding in there. I'm hoping that there is a cleaner alternative to the weighted-index reconstruction algorithm -- the one that's there right now puts a lot of requirements on the input data, in order to avoid floating-point operations. === After working on ENDIVE expansion, I moved on to the problem of authorities voting. There's a lot of uncertainty here and decisions I'll need to make. The biggest decision is how the new voting process for ENDIVEs should relate to the current voting process for consensus documents. The choices seem to be: * Have two processes with different kinds of votes. * Expand the information that goes into current votes, and generate ENDIVEs as a new kind of consensus from the current voting process. * Design the ENDIVE voting process so that it can also be used to generate current-style consensuses. Using separate processes might be simplest, but it does carry risk for duplicated code, and for the two voting processes reaching divergent results. It would also be potentially bad if an authority could vote differently in each process. The second approach -- where ENDIVEs are another kind of consensus generated from current-style voting documents -- has some charm to it, but it perpetuates our current consensus-voting system, and makes us specify a second encoding of CBOR objects as text, if we want to vote on them meaningfully. This is a bit nasty, since the standard-specified text encoding of cbor is not a bijection, and doesn't mesh too well with our old directory format. Therefore the third approach seems a little better to me now. In it, we'd specify our votes as a CBOR-based format loosely based on the current vote format, and describe a way to extract from them the same data that is currently used for our voting algorithm. This would let us encode CBOR within the document naturally, while not having to throw away too much of our existing voting specification and voting code. I've got a few more ideas here, though I don't have any specs yet. I've been collecting the ideas in a notes file [VNOTES] so I don't forget them. === While working on voting notes, I found some areas where the ENDIVE and SNIP formats might be better. First, I hadn't thought about authority certificates. Those should go in, so that we can have the ENDIVE-signing keys be signed themselves with the authorities' long-lived identities. Second, there seem to be some opportunities for transmitting indices in a simpler way. Instead of assigning a weight to each router separately, for example, we could derive one weighted index from another weighted index by re-weighting it, as we do with the "bandwidth-weights" values in the current consensus. Teor also made some good suggestions for improving the diff format; I incorporated one and made a note to check the other. === I also spent a little while this week reading about BLS signatures, since they are the leading candidate for how we might do threshold signatures here. === Next week I'm going to try to get voting all specified. This will be a pretty deep dive, since I will need to maintain some backward compatibility with existing voting algorithms. There may be opportunities for simplification, however. === [GITREPO] https://github.com/nmathewson/walking-onions-wip [RECONST] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/04-recon… d [SNIPFMT] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/02-endiv… [VNOTES] https://github.com/nmathewson/walking-onions-wip/blob/master/notes/voting.t…

3 3

Walking onions status update: week 4 notes
by Nick Mathewson 29 Mar '20

29 Mar '20

Walking Onions: week 4 update On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send out these updates once a week. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html Week 2: Specifying details of SNIP and ENDIVE formats, in CDDL. https://lists.torproject.org/pipermail/tor-dev/2020-March/014181.html Week 3: Expanding ENDIVEs into SNIPs. https://lists.torproject.org/pipermail/tor-dev/2020-March/014194.html This week was pretty rough for me; the impact of being isolated at home is starting to kick in, and my kid is still trying to find a schedule that works for him. I got a lot less done this week than I had planned. That said, I _did_ get some good design done, in two areas: voting, and circuit extension. == Voting I'm trying to generalize our voting algorithms to minimize the need for new consensus formats in the future. I want most new fields to be self-describing in terms of how to tabulate votes for them. Trying to work this out has taken a bunch of energy, though and I'm not quite sure I'm doing it right. The work here is unfinished. I still think it's reasonable to encode existing votes and new votes in a new cbor-based structure. By using compression, a little redundancy in votes shouldn't hurt us too badly -- though that's something I need to benchmark eventually before we build this. I've come up with a way for voters to use diffs and compression when _uploading_ votes: first, the voter asks for a summary of what mechanisms are supported by the recipient and which older votes it has. Next, the voter can use that information to upload a vote diff with an appropriate compression format. Right now I believe that I have most of the fields for the votes figured out, but I need to clean them up a _lot_. SNIP bodies can work similarly to how microdescriptors work now, if we like -- or they could involve individual votes on input fields. I think I'm going to write up what I can here, and then revisit SNIP bodies once I've worked some on section 6 (the "fancy" client behavior). While working on voting, I found a need for various changes in the already written sections. For one example, the previous description of the ENDIVE format included the SNIP signatures as part of the signed ENDIVE. But we can't actually generate that as part our voting protocol: the authorities can't sign SNIPs until they know the complete ENDIVE contents, and so putting the SNIP signatures in the signed part of the ENDIVE would add an extra round of voting and signatures. Instead, I've moved the SNIP signatures (and related stuff) into the signature header of the ENDIVE. I've also worked somewhat on the so-called "root document" holding the network configuration parameters. I'm hoping that this document can have an effective lifetime of weeks or months for clients, and that clients can download only the parts that they need. The format is in section 02, but needs more description. The work-in-progress voting writeup is in [VOTING], but it is not so close to done. I think it will need a lot of writing before it's ready for even preliminary review; it doesn't make sense as it is, and it contradicts itself in places. == Extending circuits There are a few wrinkles here that we hadn't actually specified in earlier walking onions work! When one relay is told to extending a circuit to a target relay by index, the extending relay actually needs to include part of a SNIP for that second relay: otherwise, the extending relay won't know which onion key the client expects to receive for it. We don't want to send an entire SNIP -- only the parts that the client will expect to match. In addition to wide EXTEND cells, we'll also need wide RENDEZVOUS and INTRODUCE1 and INTRODUCE2 and TRUNCATED cells. All together, that means that having a general "cell fragmentation" mechanism is probably warranted; I've started a separate proposal for that [WIDE_EVERYTHING]. We're going to need a general extension format for CREATE and EXTEND cells, to signal other information, like SNIPs, requests for SNIPs, and so on. We also need a way to ask for SNIPs without actually performing a handshake. I've tried to fit these all together in a fairly logical way in [EXTENDING], but there's more to write here, including examples. == Fun facts The walking onion proposal is now the second-longest that we have ever had, behind only proposal 224 (v3 hidden services). == Next steps In this coming week I plan to try to wrap up section 3 on voting and section 5 on extending circuits. I'm going to go back to the start of section 2 and start revising all I've written so far for consistency. Time permitting, I want to sketch out all the knowns and unknowns for section 6 (on tricky client behavior). [EXTENDING] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/05-exten… [VOTING] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/02-votin… [WIDE_EVERYTHING] https://github.com/nmathewson/walking-onions-wip/blob/master/other-proposal…

2 1

Walking Onions status update: week 2 notes
by Nick Mathewson 29 Mar '20

29 Mar '20

Walking onions -- week 2 update Hi! On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. I'm going to try to send these out thee updates once a week, in case anybody is interested. My previous updates are linked below: Week 1: formats, preliminaries, git repositories, binary diffs, metaformat decisions, and Merkle Tree trickery. https://lists.torproject.org/pipermail/tor-dev/2020-March/014178.html You might like to have a look at that update, and its references, if this update doesn't make sense to you. === This week, I worked specifying the nitty-gritty of the SNIP and ENDIVE document formats. I used the CBOR meta-format [CBOR] to build them, and the CDDL specification language [CDDL] to specify what they should contain. As before, I've been working in a git repository at [GITHUB]; you can see the document I've been focusing on this week at [SNIPFMT]. (That's the thing to read if you want to send me patches for my grammar.) There were a few neat things to do here: * I had to define SNIPs so that clients and relays can be mostly agnostic about whether we're using a merkle tree or a bunch of signatures. * I had to define a binary diff format so that relays can keep on downloading diffs between ENDIVE documents. (Clients don't download ENDIVEs). I did a quick prototype of how to output this format, using python's difflib. * To make ENDIVE diffs as efficient as possible, it's important not to transmit data that changes in every ENDIVE. To this end, I've specified ENDIVEs so that the most volatile parts (Merkle trees and index ranges) are recomputed on the relay side. I still need to specify how these re-computations work, but I'm pretty sure I got the formats right. Doing this calculation should save relays a bunch of bandwidth each hour, but cost some implementation complexity. I'm going to have to come back to this choice going forward to see whether it's worth it. * Some object types are naturally extensible, some aren't. I've tried to err on the size of letting us expand important things in the future, and using maps (key->value mappings) for object that are particularly important. In CBOR, small integers are encoded with a little less space than small strings. To that end, I'm specifying the use of small integers for dictionary keys that need to be encoded briefly, and strings for non-tor and experimental extensions. * This is a fine opportunity to re-think how we handle document liveness. Right now, consensus directories have an official liveness interval on them, but parties that rely on consensuses tolerate larger variance than is specified in the consensus. Instead of that approach, the usable lifetime of each object is now specified in the object, and is ultimately controlled by the authorities. This gives the directory authorities more ability to work around network tolerance issues. Having large lifetime tolerances in the context of walking onions is a little risky: it opens us up to an attack where a hostile relay holds multiple ENDIVEs, and decides which one to use when responding to a request. I think we can address this attack, however, by making sure that SNIPs have a published time in them, and that this time moves monotonically forward. * As I work, I'm identifying other issues in tor that stand in the way of a good efficient walking onion implementation that will require other follow-up work. This week I ran into a need for non-TAP-based v2 hidden services, and a need for a more efficient family encoding. I'm keeping track of these in my outline file. Fun fact: In number of bytes, the walking onions proposal is now the 9th-longest proposal in the Tor proposal repository. And it's still growing! Next week, I'm planning to specify ENDIVE reconstruction, circuit extension, and maybe start on a specification for voting. [CBOR] RFC 7049: "Concise Binary Object Representation (CBOR)" https://tools.ietf.org/html/rfc7049b [CDDL] RFC 8610: "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures" https://tools.ietf.org/html/rfc8610 [GITREPO] https://github.com/nmathewson/walking-onions-wip [SNIPFMT] https://github.com/nmathewson/walking-onions-wip/blob/master/specs/02-endiv…

3 6

Tails vs the capacity of the Meek bridges
by anonym 24 Mar '20

24 Mar '20

Hi, tl;dr: if Tails makes it too easy to use Meek bridges, could it overload the current set of Meek bridges? First some background: during startup Tails can be told to start Tor Launcher so users can e.g. configure any bridges they want. So far we have not provided any pre-configured bridges, i.e. the only option has been to manually enter the bridge information you have obtained yourself. In Tails' threat model it is assumed that adversaries monitor the default bridges provided by the Tor Browser, and that our users want to avoid detection of that, so we are not interested in adding the default bridges to Tails, but we are interested in adding support for Meek [1] (at least because it's the only PT that works in China), since our understanding is that it adds enough plausible deniability to avoid the above problem. So, in summary, Tails would like to provide two options in its (patched) Tor Launcher, Meek or manually providing the bridge info. [2] However, we wonder if the combination of not providing the default bridges while making Meek available could overwhelm the Meek bridges; we expect some significant amount of Tails bridge users to select the Meek option over manual entry simply out of convenience. Let's do some back-of-the-envelope estimations to see what we can expect: (Assumption: Tor Browser users and Tails users are very similar, e.g. similar ratios want to use each PT, similar requirements for "convenience over security", and so on.) Looking at your metrics, there are 1000k daily Tor Browser users [3], overall bridge/PT usage is 50k [4], Meek usage is around 10k [5], so 5% of Tor Browser users use bridges/PTs, and 1% use Meek. On the Tails side, we measure around 30k daily users (from update pings). >From what I said above, I don't think we can expect only 1% of the Tails users to pick Meek; I would expect it to be closer to the 1% of Meek users plus the x% of default bridge users, but I couldn't find stats for that in your metrics. Still, we know that it cannot be more than the percentage of all bridge/PT users, so we know that 1% + x% <= 5%. So, let's just assume the worst case, that 5% of Tails users will use Meek, and we have that we expect 5% of 30k = 1.5k additional Meek users. Also, if we consider places where Meek is the only options, Tails probably has close to zero users there, but once Meek is supported it could grow. I guess China is the only such place (?), so with its 1250 Meek users in China [6] we can expect 1250 * 30k/1000k = 34 new Tails users using Meek. Basically a negligible amount. Even if we consider all Meek users, 10k * 30k/1000k = 300 doesn't change much. So, all-in-all, we can expect Tails to bring up to 1.5k + 300 = 1.8k new Meek users, but since those are upper-bound estimations it would probably be much lower. Looking on Meek usage over time, it seems to fluctuate way more than that, e.g. during the summer of 2019 it was up to over 25k, i.e. more than three times what it is now. So I guess we don't have to worry about shocking the Meek bridges? OTOH, a possible side-effect is that this change in Tails increases usage of Meek outside of China. Perhaps whoever pays the bills for the Meek instances don't want this? Please advice! Also, please let us know if there is something else we haven't thought of! Cheers! [1] https://redmine.tails.boum.org/code/issues/8243 [2] If you are really interested you can check out our PoC/WIP here: https://nightly.tails.boum.org/build_Tails_ISO_feature-8243-meek/builds/las… [3] https://metrics.torproject.org/webstats-tb.html [4] https://metrics.torproject.org/userstats-bridge-transport.html [5] https://metrics.torproject.org/userstats-bridge-transport.html?transport=!%… [6] https://metrics.torproject.org/userstats-bridge-combined.html?start=2019-12…

4 5

Re: [tor-dev] GSOC'20
by Sanjiban Sengupta 23 Mar '20

23 Mar '20

Dear Sir, This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I have practical and working knowledge of C, C++, Python and Java, for web, I am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel and know the technicalities to apply these to solve modern real life problems. On going through the project proposals, I found the projectImplementing Salmon as a bridge distribution mechanism, and Tor Weather interesting to work upon and contribute and thus will be thankful for your kind guidance. Thus I request the mentors to kindly guide me for the beginning processes. Regards, Sanjiban Sengupta On Sat, Mar 21, 2020 at 11:47 AM Sanjiban Sengupta <b518041(a)iiit-bh.ac.in> wrote: > Dear Sir, > > This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT > Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I > have practical and working knowledge of C, C++, Python and Java, for web, I > am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS > and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel > and know the technicalities to apply these to solve modern real life > problems. > > On going through the project proposals, I found the projectImplementing > Salmon as a bridge distribution mechanism, and Tor Weather interesting to > work upon and contribute and thus will be thankful for your kind guidance. > > Thus I request the mentors to kindly guide me for the beginning processes. > > Regards, > Sanjiban Sengupta > > > > On Sat, Mar 21, 2020 at 3:50 AM Sanjiban Sengupta <b518041(a)iiit-bh.ac.in> > wrote: > >> Dear Sir, >> >> This is Sanjiban Sengupta, sophomore in Computer Engineering from IIIT >> Bhubaneswar, India, would like to contribute to Tor Project for GSoC'20, I >> have practical and working knowledge of C, C++, Python and Java, for web, I >> am familiar with HTML, CSS, JS, Bootstrap and frameworks such as ReactJS >> and NodeJS, also i am acquainted with concepts of ML and AI, Linux Kernel >> and know the technicalities to apply these to solve modern real life >> problems. >> >> On going through the project proposals, I found the projectImplementing >> Salmon as a bridge distribution mechanism, and Tor Weather interesting to >> work upon and contribute and thus will be thankful for your kind guidance. >> >> Thus I request the mentors to kindly guide me for the beginning processes. >> >> Regards, >> Sanjiban Sengupta >> >

2 1

Lets give every circuit its own exit IP?
by nusenu 18 Mar '20

18 Mar '20

The unbearable situation with Google's reCAPTCHA motivated this email (but it is not limited to this specific case). This idea came up when seeing a similar functionality in unbound (which has it for a different reason). Assumption: There are systems that block some tor exit IP addresses (most likely the bigger once), but they are not blocked due to the fact that they are tor exits. It just occurred that the IP got flagged because of "automated / malicious" requests and IP reputation systems. What if every circuit had its "own" IP address at the exit relay to avoid causing collateral damage to all users of the exit if one was bad? (until the exit runs out of IPs and starts to recycle previously used IPs again) The goal is to avoid accumulating a bad "reputation" for the single used exit IP address that affects all tor users of that exit. Instead of doing it on the circuit level you could do it based on time. Change the exit IP every 5 minutes (but do _not_ change the exit IPs for _existing_ circuits even if they live longer than 5 minutes). Yes, no one has that many IPv4 addresses but with the increasing availability of IPv6 at exits and destinations, this could be feasible to a certain extend, depending on how many IPv6 addresses the exit operator has. There are exit operators that have entire /48 IPv6 blocks. problems: - will not solve anything since reputation will shift to netblocks as well (How big of a netblock are you willing to block?) - you can tell two tor users easily apart from each other even if they use the same exit (or more generally: you can tell circuits apart). There might be all kinds of bad implications that I'm not thinking off right now. - check.tpo would no longer be feasible - how can do we still provide the list of exit IPs for easy blocking? Exits could signal their used netblock via their descriptor. What if they don't? (that in turn opens new kinds of attacks where an exit claims to be /0 and the target effectively blocks everything) - more state to track and store at the exit -... some random thoughts, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

4 6

Auto close old tor GitHub pull requests?
by teor 16 Mar '20

16 Mar '20

Hi all, I'd like to use a GitHub bot to automatically close old tor and torspec pull requests. The Network Team currently uses GitHub to review pull requests (but we use Trac for task tracking). Some GitHub users won't contribute to a project with lots of open pull requests, because they think it's been abandoned. (Or they think we're bad at merging PRs.) But lots of our PRs get left open because we've squashed and merged the PR, and GitHub doesn't recognise the squashed commits, so GitHub doesn't auto-close the PR. I'm proposing closing PRs with no activity for 60 days, but I think anywhere between 30 and 180 days would work. (Most of our backports happen within 60 days.) For more details, see: https://trac.torproject.org/projects/tor/ticket/33629 T -- teor ----------------------------------------------------------------------

1 0

Request to provide code for Padding in Tor
by DIVYA GUPTA 16 Mar '20

16 Mar '20

Hello, We are under-graduate students from Malaviya National Institute of Technology currently pursuing Computer Science & Engineering. We are working on developing a defense for Tor against Website Fingerprinting Attacks. While reading about tor we came across padding which is implemented in Tor. It would be great if you could make available the code for the padding. Regards, Divya.

2 1

Fwd: Upcoming Tor security releases to fix a denial-of-service issue
by Nick Mathewson 16 Mar '20

16 Mar '20

---------- Forwarded message --------- From: Nick Mathewson <nickm(a)torproject.org> Date: Mon, Mar 16, 2020 at 1:25 PM Subject: Upcoming Tor security releases to fix a denial-of-service issue To: <tor-talk(a)lists.torproject.org> Hello! Some time this week, we currently plan to put out a set of security updates for all supported versions of Tor. These releases will fix a pair of denial-of-service bugs: one that we are classifying at "low" severity, and one that we are classifying at "high" severity. Our recommendation will be for everybody, including relays and clients, to upgrade once packages are available for their platforms. Although these vulnerabilities are "only" denial-of-service issues, any denial-of-service attack against Tor could be leveraged by an attacker to aid in a traffic analysis attack. To the best of our knowledge, these vulnerabilities are not being exploited in the wild. Currently supported release series are 0.3.5, 0.4.1, 0.4.2, and 0.4.3 (alpha). If you have not yet upgraded to one of those, the time to do so is soon. For our policy and process for handing security issues, please see: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Securit… best wishes, -- Nick

1 0

SOOC (Same Origin Onion Certificates) discussion tomorrow
by Richard Pospesel 10 Mar '20

10 Mar '20

Hi Everyone, The current UX situation around HTTPS-enabled onionsites is not the best: Tor Browser shows security warning splash screens and icons in scenarios that do not warrant them. On top of that, the only way for your onion site to not show these warnings is by getting an EV cert which is only practical if you have the $$$. alecmuffet's SOOC proposal ( https://github.com/alecmuffett/onion-dv-certificate-proposal/blob/master/te… ) does seem to fix these UX problems for us. Though the proposal is still incomplete, we can make forward progress with a prototype implementation as the certificate specification is fully fleshed out. We'll be talking about this tomorrow during the Sponsor 27 meeting tomorrow on Tuesday 10th May @ 1500UTC in #tor-meeting. Please feel free to come by if you wish to discuss. best, -Richard

2 1

Walking Onions status update: week 1 notes
by Nick Mathewson 08 Mar '20

08 Mar '20

Hi! On our current grant from the zcash foundation, I'm working on a full specification for the Walking Onions design. If you haven't read it, Walking Onions is a set of ideas meant to transition Tor away from its current dependency on a directory system, to improve scaling in the long term, and performance of low-bandwidth clients in the short term. I'm going assume in the rest of my email that you already have a decent idea of how Tor works, and of the Walking Onions proposal. If you need a Walking Onions refresher, see proposal 300 [PROP300] and the whitepaper that Chelsea Komlo, Ian Goldberg, and I have been working on [WHITEPAPER]. The current scope for this work is to try to solve all the open design questions for Walking Onions, and to write a set of specifications we could use to build the system. I'm aiming to have the initial versions of these specs wrapped up in April. Rather than dumping everything onto the mailing list at once, I'm going to try working in the open, sending weekly updates about my work. Design and specification are a lot of fun for me; I hope you'll have fun too reading along. This past week, I started by writing: - An outline of the set of specifications we'd need to have before we could implement walking onions. (OUTLINE0.txt) - A list of areas of uncertainty that we need to solve before or during writing the specs. (UNCERTAINTY.txt) - A list of related problems that we might wind up solving along with the walking onion specs (OPPORTUNITIES.txt) - A rough grouping of initial tasks, which should eventually turn into a schedule. (PLAN.txt) You can find all these documents in a working git repository [GITREPO] that I've been making for this purpose, if you like. I don't intend for this repository to be a permanent reference or to be necessarily useful for anybody but me: everything important about it should eventually wind up in specifications and on this mailing list. == The biggest area of uncertainty was to pick a meta-format for ENDIVEs and SNIPs. I looked at our existing directory meta-format, along with protobufs, messagepack, and a whole bunch of other systems. I also looked into what kind of efficiency we'd get by just using a hand-tooled non-extensible binary format, so that I could see what kind of overhead we were looking at. The issue here is that I think we don't want to be using a text-based metaformat for SNIPs: we need SNIPs to be as small as possible, and our current text-based directory metaformat only compresses well when we're using huge amounts of it at once. But if we're doing a binary format, we probably shouldn't hand-roll it. Trunnel lets us handle our existing formats safely, but those formats aren't so efficient in practice, and they aren't too standardized. If we use a standardized binary format, then we get encoding, decoding, debugging dumps, optimization, and lots of other stuff "for free". After looking at a pretty huge variety of formats, I think that our best bet is something called "CBOR", standardized as RFC 7049 [CBOR]. It is substantially based on MessagePack, but with a number of improvements. Here are some of CBOR's advantages: + It's optimized first for allowing encoding/decoding to be done with small, simple code -- and second, for having a small binary format. IMO it succeeds at both: the metaformat is simple, and there are decoder implementations in less than 1kb of binary size. + You can think of its data model as a "binary json"; it's very flexible. (yes I know about the other binary jsons) + It's standardized, and there are lots of implementations of it, in lots of different languages. + It has a data definition language [CDDL], which we don't actually need to implement, but which we can use to help write our specs and validate our implementation. + It has a textual encoding for debugging, and a one-way mapping to json. Here are some of its disadvantages: - It doesn't have a built-in object mapping like protobufs does. - It allows multiple encodings for a single piece of data. (The CBOR RFC explains how to make a canonical encoding.) - Our diff story gets slightly more complex when we have a binary format. (I'd be happy to talk about other possibilities, but please make sure you know about the bikeshed first. [BIKESHED]) == Doing diffs from one ENDIVE to the next will be helpful if we want to keep relay bandwidth on the small side. We'll need binary diffs for this. Fortunately, I think that shouldn't be too hard to do with our existing diff code: instead of doing line-by-line diffs, we'll just do CBOR-item-by-item diffs, and encode them in a byte-oriented way. I've got a prototype half-written to make sure this is right. The simplicity of CBOR makes this pretty easy. == There are a few ways to authenticate SNIPs. It's looking like right now the most efficient will be merkle trees, with the root of the tree signed by a threshold signature algorithm like BLS. One of the neat things about merkle trees is that if we are transmitting the leaves and the signature on the root, we don't need to actually transmit the intermediate nodes as part of the ENDIVE. So that's cool, and it will make ENDIVE diffs smaller. One of the not-so-neat things is that the merkle tree paths are a bit long. There are ways to make them shorter: you only need to transmit one digest for each layer, plus a bit to say which hash it is. (I learned about these tricks while working on [WHITEPAPER], thanks to Ian and Chelsea.) One risky thing I've been thinking about is whether we can use a shorter-than-256-bits hash for the merkle tree nodes, if we don't need to worry about collision resistance for the digest functions. The SNIPs are generated by authorities, but they do contain a certain amount of adversary-created material. Randomized hashes with an unpredictable key might help, particularly for intermediate nodes on the tree? [XMSS] has some ideas here. (Suggested by Jack Lloyd) == In the coming week, I'm hoping to finish up making my draft schedule, start doing research as needed to handle more open questions, and write an initial spec for the ENDIVE and SNIP formats. [BIKESHED] "Why Should I Care What Color the Bikeshed Is?" http://bikeshed.com/ [CBOR] RFC 7049: "Concise Binary Object Representation (CBOR)" https://tools.ietf.org/html/rfc7049 [CDDL] RFC 8610: "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures" https://tools.ietf.org/html/rfc8610 [GITREPO] https://github.com/nmathewson/walking-onions-wip [PROP300] "Walking Onions: Scaling and Saving Bandwidth" https://gitweb.torproject.org/torspec.git/plain/proposals/300-walking-onion… [WHITEPAPER] "Walking Onions: Scaling Anonymity Networks while Protecting Users" https://crysp.uwaterloo.ca/software/walkingonions/ [XMSS] RFC 8391: "XMSS: eXtended Merkle Signature Scheme". https://tools.ietf.org/html/rfc8391

1 0

Regarding GSoC 2020
by Aman Rastogi 05 Mar '20

05 Mar '20

Respected Developers, I am Aman, GSoC 2020 aspirant, and i would like to work for Torr. I am interested in "Privacy Friendly Web" and "OONI Explorer Advanced Search". Can I get some guidance, please? Thanking you, and sorry for such an informal mail. -- Aman

2 1

Request for onionbalance v3 pre-alpha testing
by George Kadianakis 03 Mar '20

03 Mar '20

Hello list, we've been developing Onionbalance v3 for the past months, and I'm pretty hyped to say that the project has reached a stability point that could benefit from some initial testing by curious and adventurous developers and users. The project is not yet ready for proper use in actual production environments (more on this later), but my hope is that this testing will reveal bugs that I can't catch in my test environment and receive user feedback that will speed up the overall process and allow a faster and more stable initial release. This email features a pretty complicated pseudo-guide that if it gets followed to completion it will hopefully result in a functional onionbalance setup. As I said the guide is complicated and in ugly text format, so I suggest you put on some calming music (perhaps some Bill Evans or some Com Truise or some Tor) and go slowly. Trying to speed run this will result in disappointment and perhaps even partial or total loss of faith in humanity. == So what's the current status of Onionbalance v3? == The project lives as a fork of Donncha's original Onionbalance project in this repository: https://github.com/asn-d6/onionbalance The current status is that during my testing I have onionbalanced a real v3 service with two backend instances successfully for a few days with no obvious reachability issues. It also works on a private tor network (using Chutney) for hours with no issues. All the above happened on Debian stable and testing. Also, here are some features that are currently missing but will be included in the initial launch: - This new onionbalance will support both v2 and v3 onion services, altho currently v2 support is botched and only v3 will work for the purposes of this testing. If you can quickly revive v2 support patches are welcome! - Any sort of documentation about v3 mode (this post is all the documentation there is for now!) - Missing streamlined setup process (installation scripts, packages or dependency tracking) - Code documentation and unittests missing - There is no daemon mode. You will have to run this in screen for now - Fun bugs included! Furthermore, v2 and v3 support will not have feature parity at the time of the initial launch, and we will have to incrementally build these features: - Cool v2 features like DISTINCT_DESCRIPTORS are not yet implemented so v3 cannot load balance as deeply as v3 yet (patches welcome!) - There is no way to transfer your existing v3 onion service to onionbalance (patches welcome!) - There is no torrc generation for instances as in v2 (patches welcome!) - It will be possible for clients and HSDirs to figure out whether a descriptor comes from an onionservice vs a regular v3 onion. Making them indistinguishable is possible but requires more engineering and will be left as a TODO for now (see #31857) Finally, there is no guarantee that configs generated with this pre-alpha version of Onionbalance will be forward compatible when we actually launch the project. This means that if you setup onionbalance v3 right now, you might need to re-set it up in a month or so when the actual release happens. tl;dr This is a call for experimental testing and it's meant to help the developers and accelerate development and not to be used in anything remotely serious or close to production. The actual release for operators and users is coming in a month or so. == Before we get started... == OK so let's assume that you still want to help out! Given that this is a pre-alpha test I'm gonna assume that you fill the following prerequisites otherwise you are gonna have an adventure: [ ] Comfortable with git [ ] Familiar with v2 onionbalance (at least in terms of terminology and how it works) [ ] Familiar with setting up and configuring (v3) onion services [ ] Familiar with Linux (or in the mood to find weird bugs in other platforms) [ ] Familiar with compiling C projects and chasing their dependencies [ ] Familiar with running Python projects and chasing their dependencies [ ] Patience to push through various import error messages that might appear The above are not mandatory but if you don't check out all the boxes you might encounter various errors and roadblocks that will frustrate. Actually... even if you check out all these boxes you might encounter various errors that might frustrate you. == Some theory == In any case, if you want to learn more about the architecture of Onionbalance, see: https://blog.torproject.org/cooking-onions-finding-onionbalance As part of this guide, you might hear me say the term "frontend" which is what we call master onionbalance onion service instance (whose address clients type on their browser), or the term "backend" which is the load balancing instances that do the actual introduction/rendezvous work for the frontend. So for example, in the visual below, Alice fetches the frontend's descriptor but does the introduction/rendezvous with the backends: ================================================================================ + + + Alice fetches frontend +^^^^^^^^^^^^^^^^^^^^^^^^^^^^+ + + +~~~~~~~~~~+ descriptor +~~~~~~+ HSDir hash ring ) + + | Alice +~~~~~~~~~~~~~~~~~~~+ +vvvvvvvvvvvv+vvvvvvvvvvvvvvv+ + + | | | + + |the client| +-----------+ | + + +~~~+~~~~~~+ |backend | | Upload frontend + + | |instance 1 +-------+ | descriptor + + | +-----------+ | | + + | | +-------+-----------+ + + | +-----------+ +----+ | + + +~~~~~~~~~~~~~+backend | | frontend | + + Alice does |instance 2 +------------+ onionbalance | + + introduction +-----------+ +----+ service | + + with backend | +-------------------+ + + +-----------+ | + + |backend +-------+ + + |instance 3 | + + +-----------+ + + + ================================================================================ OK enough with theory. Let's jump to practice: == Installation instructions == For the purposes of this guide I will assume you have a frontend service and a bunch of backend instances that you want to load balance. ==== Setup frontend service ==== Our first goal is to get onionbalance and Tor running on your frontend service. - Installing Tor Let's start: SSH to your frontend service, and get yourself a Tor binary to use as the control port for onionbalance. You will want a recent version of Tor (above 0.4.2) but other than that there is no strict requirement. Feel free to get it off your package manager or git. Setup a minimal torrc with a control port enabled. As an example: """ SocksPort 0 ControlPort 127.0.0.1:6666 DataDirectory /home/user/frontend_data/ """ Feel free to tweak your torrc as you feel (also enable logging), but for the purposes of this guide I assume that your control port is at 127.0.0.1:6666. - Running onionbalance for the first time Now we need to run onionbalance and generate keys and a config file for it. Here the fun starts: Let's start by git cloning 'https://github.com/asn-d6/onionbalance'. To run onionbalance v3 you are gonna need a bunch of fun Python dependencies including (but perhaps not limited to): - Python3 above 3.7 - Python3 packages for: pyyaml, pycrypto, setproctitle, future - stem version 1.8 or up (so that it includes v3 support) Please collect all the above whichever way you prefer: if you don't have them, things are gonna crash at some point. There is no dependency tracking for v3 onionbalance right now (i.e. requirements.txt) so I don't enforce the above in an elegant way. Now that you are good to go with dependencies, feel free to install onionbalance using the setup.py script, or you can run it from the source directory if you prefer that. Now is the time to create keys and a config file for your frontend service. Let's do that by running the configuration wizard: $ python3 onionbalance-config.py It should ask you if you want v3 or v2 (say v3!), and a bunch of other questions and by the end you should have a config file somewhere (by default at ./config/master/config.yaml) and the corresponding frontend key file (again at ./config/master/*key). The onion address of your frontend service can be found in the bottom of your config file. So if it says: key: wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.key your onion address is "wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.onion" For now, note down the onion address of the frontend service and let's move on to the next step! Make sure you got a v3 onion address out of this process, otherwise something went wrong. If you got to this stage without any major turbulance, you will end up with a functional onionbalance in no time. If you had lots of problems so far, brew up some green tea, because there is more fun coming! ==== Setup backend instances ==== OK now with the frontend onion address noted down, let's move to setting up your backend instances: SSH to one of your backend instances and let's setup Tor in there! The catch is that onionbalance v3 backend instances need a very recent version of Tor because of some issues in the v3 protocol (#32709). As a matter of fact, the patch required has not yet been merged to upstream Tor (it will be merged in mid February because we are currently at a code freeze) so you will need to use my personal repository for now (as indicated by the last comment in #32709). So let's git clone 'https://github.com/asn-d6/tor.git' and checkout branch 'ticket32709_044_02'. Build that branch (using the classic './autogen.sh && ./configure && make' combo) and you should have a Tor binary at ./src/app/tor. If this doesn't happen, you are gonna need to install various C dependencies like libssl-dev, libevent-dev, etc. Now you will need a torrc file for your backend instance. As in onionbalance v2, the torrc file needs to setup an onion service (and in this case a v3 one). So far so good but here comes the twist: 1) In the end of your torrc file, inside the HiddenService block please add the following line: 'HiddenServiceOnionBalanceInstance 1'. 2) Now go in your hidden service directory where the hostname and hs_ed25519_public_key files are living (you will need to startup Tor once with the onion service enabled for the directory to get created) you need to create a new file with the name 'ob_config' that has the following line inside: 'MasterOnionAddress wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.onion' but substitute the onion address with your frontend's onion address. The points (1) and (2) above are extremely important and if you didn't do them correctly, nothing is gonna work. If you want to ensure that you did things correctly, start up Tor, and check that your *info* log file includes the following line: [info] ob_option_parse(): OnionBalance: MasterOnionAddress wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.onion registered If you don't see that, then something went wrong. Please try again from the beginning of this section till you make it! This is the hardest part of the guide too, so if you can do that you can do anything (fwiw, we are at 75% of the whole procedure right now). After you get that, also make sure that your instances are directly reachable (e.g. using Tor browser). If they are not reachable, then onionbalance won't be able to see them either and things are not gonna work. OK, you are done! Now do the same for the other backend instances and note down the onion addresses of your backend instances because we are gonna need them for the next and final step. ==== Get it running! ==== OK now let's login back to the frontend server! Go to your onionbalance config and add your instance addresses in the right fields. In the end it should look like this (for a setup with 4 backend instances): """ services: - instances: - address: 4a2zwxetkje5lwvahfv75arzrctn6bznkwmosvfyobmyv2fc3idbpwyd.onion name: node1 - address: i5aqar7fgqcsqwalwjltrb3w3xyj23ppgsnuzhhkzlhbt5337aw2joad.onion name: node2 - address: uutpxl2cfkk5qa7z4xtgdxssnhutmoo2y2rw6igh5ez4hpxaz4dap7ad.onion name: node3 - address: 5xeigil3hjxltcs3ti4skerhpc252ci7ejynrtm73v3bcbxlfuulsjqd.onion name: node4 key: wrxdvcaqpuzakbfww5sxs6r2uybczwijzfn2ezy2osaj7iox7kl7nhad.key """ Now let's fire up onionbalance by running the following command: $ python3 onionbalance.py -v info -c config/master/config.yaml -p 6666 If everything went right, onionbalance should start running and after a bit your frontend service should be reachable!!! You are an onionbalance expert! If something did not go right, that's OK too, don't get sad because this was quite complicated. Please check all your logs and make sure you did everything right according to this guide. Keep on hammering at it and you are gonna get it. If nothing seems to work, please get in touch with some details and I can try to help you. == Now what? == Now that you managed to make it work, please monitor your frontend service and make sure that it's reachable all the time. Check your logs for any errors or bugs and let me know if you see any. If you want you can make onionbalance logging calmer by using the '-v warning' switch. (Also if you see this log message in your frontend Tor logs: "Bug: Unrecognized signal 132..." that's normal and it will be fixed with #33104). If you are in the mood for coding feel free to start hacking on onionbalance, but I would suggest you first get in touch over Github because I'm also working at it at the same time and it would be a shame if we do duplicate work. If you find bugs or do any quick bugfixes, please submit them over Github! If you do code review, please let me know through Github of any issues you find. Also, if you are gonna be at FOSDEM this upcoming weekend feel free to get in touch if you have any questions or you want to live test it. If you have any other feedback, I'd be glad to hear it! == Next steps == I'm gonna be working on this for the next month: fixing bugs, writing documentation, writing tests, creating packages, submitting and fixing bug reports in tor and stem etc. Expect more updates on this mailing list, and also a blog post to announce the initial launch of the project. Finally, if there are any errors on this post, I will reply to this thread with an errata! Hope this works for you and you have fun onionbalancing your services! Thanks a lot for the testing and looking forward to your feedback! o/ PS: Major thanks to Donncha for the super clean onionbalance codebase (I did not expect to reuse so much code), the Tor network team for the support, and Damian for all the work on stem v3 support! Also OTF for the $$$ ;)

1 2

interested in working on a ticket
by indigo omega 24 Feb '20

24 Feb '20

Hello, I am a new contributor interested in working on ticket 13184: https://trac.torproject.org/projects/tor/ticket/13184 . The ticket says that it was assigned 8 months ago, but then deleted? I am having trouble understanding if this ticket is under current development or not. Should I start working on it and submit it for review, or should I email someone, perhaps a maintainer, to see if someone is actively working on it. Any advice would be much appreciated. Thank you, Amir Ghorbanian

3 2

tor sending requests with a token
by Pouya Miralaee 21 Feb '20

21 Feb '20

Hello to the community and good evening! I have some websites that i want to hide from public eyes, without going to implement them as a HS. So here is my question: can i implement a certificate authority on my servers, and then instruct tor to use that certificate for requesting the websites? or something like a token generated on the servers and tor use that token as an authenticated client? The big picture of my scenario is that i have a custom tor browser (some browser extensions pre-installed by default) and i want my clients using that particular browser only access the mentioned websites! Cheers! -- *Pouya Miralayi*

1 0

TOR Browser Circuit Selection of Exit Nodes
by Vianney Gomezgil Yaspik 19 Feb '20

19 Feb '20

Good evening, A group of students at Johns Hopkins University and I have been analyzing the circuit selection algorithm for TOR´s browser EXIT nodes. It is an exploratory project trying to discover how do the EXIT nodes are selected every time the circuit changes. So far, we have discovered that the time that the browser is accessed, calendar date, physical location, use of a bridge (or not), and the entry node do not change the pattern of the EXIT nodes. Moreover, than approximately 80% of all exit nodes come from 10 specific countries, even though these 10 countries only account for approximately 50% of all available exit nodes. All of which has led us to conclude that the selection of the EXIT nodes in the TOR browser is not random. We would like to further explore however, what are the factors that determine which relay becomes the exit node every time a circuit is changed. Is there anyone that we could speak to or that could give us further insight as to how the selection of the exit node in TOR´s circuit works? v/r

3 2

Making Tor's CI Faster
by teor 13 Feb '20

13 Feb '20

Hi, Want to help make Tor's CI go faster? As part of the Sponsor 55 IPv6 work, I need to run Chutney IPv6 tests on every Tor commit, as part of Tor's CI. We're currently running these tests with fast_finish, because: * Travis CI only has IPv6 on macOS * macOS Chutney tests take 20-45 minutes in Travis CI But fast_finish requires allow_failure. We can't have allow_failure on code that's we're modifying every day, so I'm going to make the Travis IPv6 Chutney job mandatory. To speed up CI after this change, I'm going to make a new IPv6-only test-network target, and run it in the macOS CI: https://trac.torproject.org/projects/tor/ticket/33280 I've also identified one redundant Travis CI job that we can delete: https://trac.torproject.org/projects/tor/ticket/33195#comment:3 Overall, these changes should reduce total CI runtime by 20-35 minutes. Due to parallelism, the wall clock time should reduce by 10-35 minutes. We can squeeze the most benefit from the parallelism by sorting jobs in speed order (longest first): https://trac.torproject.org/projects/tor/ticket/33194 If you have any other ideas for making Tor's CI go faster, please let me know. Slow CI is a particular issue for backports, because each 0.3.5 backport updates 9 branches: * {maint,release}-{0.3.5,0.4.1,0.4.2,0.4.3} and * master. T -- teor ----------------------------------------------------------------------

2 1

Proposal 313: Relay IPv6 Statistics
by teor 10 Feb '20

10 Feb '20

Hi, Here is an initial draft of Proposal 313: Relay IPv6 Statistics. This proposal includes: * logging the number of IPv6 relays in the consensus, and * relays publishing IPv6 connection and consumed bandwidth statistics. This is the third of 3 proposals: * Proposal 311: Relay IPv6 Reachability * Proposal 312: Automatic Relay IPv6 Addresses * Proposal 313: Relay IPv6 Statistics I revised proposals 311 and 312 last week, and merged them to torspec as drafts. There are still some TODO items in the proposal, about: * safely collecting these new statistics on bridges, and * getting accurate IPv6 connection statistics. If you know about tor's statistics, please give us some feedback! The full text is included below, and it is also available as a GitHub pull request: https://github.com/torproject/torspec/pull/108 The related tickets are #33159 (proposal) and #33051 and #33052 (implementation): https://trac.torproject.org/projects/tor/ticket/33159 https://trac.torproject.org/projects/tor/ticket/33051 https://trac.torproject.org/projects/tor/ticket/33052 Please feel free to reply on this list, or via GitHub pull request comments. Filename: 313-relay-ipv6-stats.txt Title: Relay IPv6 Statistics Author: teor Created: 10-February-2020 Status: Draft Ticket: #33159 0. Abstract We propose that Tor relays (and bridges) should log the number of relays in the consensus that support IPv6 extends, and IPv6 client connections. We also propose that Tor relays (and bridges) should collect statistics on IPv6 connections and consumed bandwidth. Like tor's existing connection and consumed bandwidth statistics, these new IPv6 statistics will be published in each relay's extra-info descriptor. 1. Introduction Tor relays (and bridges) can accept IPv6 client connections via their ORPort. But current versions of tor need to have an explicitly configured IPv6 address (see [Proposal 312: Relay Auto IPv6 Address]), and they don't perform IPv6 reachability self-checks (see [Proposal 311: Relay IPv6 Reachability]). As we implement these new IPv6 features in tor, we want to monitor their impact on the IPv6 connections and bandwidth in the tor network. Tor developers also need to know how many relays support these new IPv6 features, so they can test tor's IPv6 reachability checks. (In particular, see section 4.3.1 in [Proposal 311: Relay IPv6 Reachability]: Refusing to Publish the Descriptor.) 2. Scope This proposal modifies Tor's behaviour as follows: Relays, bridges, and directory authorities log the number of relays that support IPv6 clients, and IPv6 relay reachability checks. They also log the corresponding consensus weight fractions. As an optional change, tor clients may also log this information. Relays, bridges, and directory authorities collect statistics on: * IPv6 connections, and * IPv6 consumed bandwidth. The design of these statistics will be based on tor's existing connection and consumed bandwidth statistics. Tor's existing consumed bandwidth statistics truncate their totals to the nearest kilobyte. The existing connection statistics do not perform any binning. We do not proposed to add any extra noise or binning to these statistics. Instead, we expect to leave these changes until we have a consistent privacy-preserving statistics framwework for tor. As an example of this kind of framework, see [Proposal 288: Privacy-Preserving Stats with Privcount (Shamir version)]. We avoid: * splitting connection statistics into clients and relays, and * collecting circuit statistics. These statistics are more sensitive, so we want to implement privacy-preserving statistics, before we consider adding them. Throughout this proposal, "relays" includes directory authorities, except where they are specifically excluded. "relays" does not include bridges, except where they are specifically included. (The first mention of "relays" in each section should specifically exclude or include these other roles.) Tor clients do not collect any statistics for public reporting. Therefore, clients are out of scope in this proposal. (Except for some optional changes to client logs, where they log the number of IPv6 relays in the consensus). When this proposal describes Tor's current behaviour, it covers all supported Tor versions (0.3.5.7 to 0.4.2.5), as of January 2020, except where another version is specifically mentioned. 3. Logging IPv6 Relays in the Consensus We propose that relays (and bridges) log: * the number of relays, and * the consensus weight fraction of relays, in the consensus that: * have an IPv6 ORPort, * support IPv6 reachability checks, and * support IPv6 clients. In order to test these changes, and provide easy access to these statistics, we propose implementing a script that: * downloads a consensus, and * calculates and reports these statistics. As well as the statistics listed above, this script should also report the following relay statistic: * support IPv6 reachability checks and IPv6 clients. The following consensus weight fractions should divide by the total consensus weight: * have an IPv6 ORPort (all relays have an IPv4 ORPort), and * support IPv6 reachability checks (all relays support IPv4 reachability). The following consensus weight fractions should divide by the "usable Guard" consensus weight: * support IPv6 clients, and * support IPv6 reachability checks and IPv6 clients. "Usable Guards" have the Guard flag, but do not have the Exit flag. If the Guard also has the BadExit flag, the Exit flag should be ignored. We propose that these logs happen whenever tor: * receives a consensus from a directory server, or * loads a live, valid, cached consensus from disk. As an optional change, tor clients may also log this information. Some of this information is not directly relevant to clients, but these logs may help developers (and users). 4. Collecting IPv6 Consumed Bandwidth Statistics We propose that relays (and bridges) collect IPv6 consumed bandwidth statistics. To minimise development and testing effort, we propose re-using the existing "bw_array" code in rephist.c. In particular, tor currently counts these bandwidth statistics: * read, * write, * dir_read, and * dir_write. We propose adding the following bandwidth statistics: * ipv6_read, and * ipv6_write. (The IPv4 statistics can be calculated by subtracting the IPv6 statistics from the existing total consumed bandwidth statistics.) We propose adding a new BandwidthStatistics torrc option and consensus parameter, which activates reporting of all these statistics. Currently, the existing statistics are controlled by ExtraInfoStatistics, but we propose using the new BandwidthStatistics option for them as well. The default value of this option should be "auto", which checks the consensus parameter. If there is no consensus parameter, the default should be 1. (The existing bandwidth statistics are reported by default.) TODO: Should we collect IPv6 bandwidth statistics on bridges? On bridges, should bandwidth statistics be on or off by default? If we do want to collect bridge statistics, and we think it's safe, modify proposals 311 and 312 to allow bridge statistics. 5. Collecting IPv6 Connection Statistics We propose that relays (and bridges) collect IPv6 connection statistics. To minimise development and testing effort, we propose re-using the existing "bidi" code in rephist.c. (This code may require some refactoring, because the "bidi" totals are globals, rather than a struct.) In particular, tor currently counts these connection statistics: * below threshold, * mostly read, * mostly written, and * both read and written. We propose adding IPv6 variants of all these statistics. (The IPv4 statistics can be calculated by subtracting the IPv6 statistics from the existing total connection statistics.) We propose using the existing ConnDirectionStatistics torrc option, and adding a consensus parameter with the same name. This option will control the new and existing connection statistics. The default value of this option should be "auto", which checks the consensus parameter. If there is no consensus parameter, the default should be 0. (The existing connection direction statistics are reported by default.) TODO: Do enough relays report ConnDirectionStatistics, for accurate IPv6 connection statistics? * at least 25% of relays have IPv6 * at the end of the project, we expect at least 33% of relays to have deployed tor 0.4.4-stable If not, we should turn on ConnDirectionStatistics by default. (Or set the consensus parameter for a few days, to collect these statistics.) 6. Directory Protocol Specification Changes We propose adding IPv6 variants of the consumed bandwidth and connection direction statistics to the tor directory protocol. We propose the following additions to the [Tor Directory Protocol] specification, in section 2.1.2. Each addition should be inserted below the existing consumed bandwidth and connection direction specifications. "ipv6-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL [At most once] "ipv6-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL [At most once] Declare how much bandwidth the OR has used recently, on IPv6 connections. See "read-history" and "write-history" for more details. (The migration notes do not apply to IPv6.) "ipv6-conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL [At most once] Number of IPv6 connections, that are used uni-directionally or bi-directionally. See "conn-bi-direct" for more details. We also propose the following replacement, in the same section: "dirreq-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL [At most once] "dirreq-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL [At most once] Declare how much bandwidth the OR has spent on answering directory requests. See "read-history" and "write-history" for more details. (The migration notes do not apply to dirreq.) This replacement is optional, but it may avoid the 3 *read-history definitions getting out of sync. 7. Optional Changes We propose some optional changes to help relay operators, tor developers, and tor's network health. We also expect that these changes will drive IPv6 relay adoption. Some of these changes may be more appropriate as future work, or along with other proposed features. 7.1. Log IPv6 Statistics in Tor's Heartbeat Logs We propose this optional change, so relay operators can see their own IPv6 statistics: We propose that tor logs its IPv6 consumed bandwidth and connection statistics in its regular "heartbeat" logs. These heartbeat statistics should be collected over the lifetime of the tor process, rather than using the state file, like the statistics in sections 4 and 5. Tor's existing heartbeat logs already show its consumed bandwidth and connections (in the link protocol counts). We may also want to show IPv6 consumed bandwidth and connections as a propotion of the total consumed bandwidth and connections. These statistics only show a relay's local bandwidth usage, so they can't be used for reporting. 7.2. Show IPv6 Statistics on Consensus Health The [Consensus Health] website displays a wide rage of tor statistics, based on the most recent consensus. We propose this optional change, to: * help relay operators diagnose issues with IPv6 on their relays, and * to drive IPv6 adoption on tor relays. Consensus Health adds an IPv6 section, with the following relay statistics: * have an IPv6 ORPort, * support IPv6 reachability checks, * support IPv6 clients, and * support IPv6 reachability checks and IPv6 clients. The definitions of these statistics are in section 3. These changes can be tested using the script proposed in section 3. 7.3. Add an IPv6 Reachability Pseudo-Flag on Relay Search The [Relay Search] website displays tor relay information, based on the current consensus and relay descriptors. We propose this optional change, to: * help relay operators diagnose issues with IPv6 on their relays, and * drive IPv6 adoption on tor relays. Relay Search adds a pseudo-flag for relay IPv6 reachability support. This pseudo-flag should be given to relays that have: * a reachable IPv6 ORPort (in the consensus), and * support tor subprotocol version "Relay=3" (or later). See [Proposal 311: Relay IPv6 Reachability] for details. TODO: Is this a useful change? Are there better ways of driving IPv6 adoption? 7.4. Add IPv6 Connections and Consumed Bandwidth Graphs to Tor Metrics The [Tor Metrics: Traffic] website displays connection and bandwidth information for the tor network, based on relay extra-info descriptors. We propose these optional changes, to: * help tor developers improve IPv6 support on the tor network, * help diagnose issues with IPv6 on the tor network, and * drive IPv6 adoption on tor relays. Tor Metrics adds the following information to the graphs on the Traffic page: Consumed Bandwidth by IP version * added to the existing [Tor Metrics: Advertised bandwidth by IP version] page * as a stacked graph, like [Tor Metrics: Advertised and consumed bandwidth by relay flags] Fraction of connections used uni-/bidirectionally by IP version * added to the existing [Tor Metrics: Fraction of connections used uni-/bidirectionally] page * as a stacked graph, like [Tor Metrics: Advertised and consumed bandwidth by relay flags] 8. Test Plan We provide a quick summary of our testing plans. 8.1. Testing IPv6 Relay Consensus Count Logging We propose to test these changes using chutney networks. However, chutney creates a limited number of relays, so we also need to test these changes on the public tor network. Therefore, we propose to test these changes on the public network with a small number of relays and bridges. We can use the script and the tor logs to cross-check these calculations. The Tor Metrics team may also independently check these calculations. Once these changes are merged, they will be monitored by tor developers, as more volunteer relay operators deploy the relevant tor versions. (And as the number of IPv6 relays in the consensus increases.) 8.2. Testing IPv6 Extra-Info Statistics We propose to test the connection and consumed bandwidth statistics using chutney networks. However, chutney runs for a short amount of time, and creates a limited amount of traffic, so we also need to test these changes on the public tor network. In particular, we have struggled to test statistics using chutney, because tor's hard-coded statistics period is 24 hours. (And most chutney networks run for under 1 minute.) Therefore, we propose to test these changes on the public network with a small number of relays and bridges. During 2020, the Tor Metrics team will analyse these statistics on the public tor network, and provide IPv6 progress reports. We expect that we may discover some bugs during the first analysis. Once these changes are merged, they will be monitored by tor developers, as more volunteer relay operators deploy the relevant tor versions. (And as the number of IPv6 relays in the consensus increases.) References: [Consensus Health]: https://consensus-health.torproject.org/ [Proposal 288: Privacy-Preserving Stats with Privcount (Shamir version)]: https://gitweb.torproject.org/torspec.git/tree/proposals/288-privcount-with… [Proposal 311: Relay IPv6 Reachability]: https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-rea… [Proposal 312: Relay Auto IPv6 Address]: https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv… [Relay Search]: https://metrics.torproject.org/rs.html [Tor Directory Protocol]: (version 3) https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt [Tor Manual Page]: https://2019.www.torproject.org/docs/tor-manual.html.en [Tor Metrics: Advertised and consumed bandwidth by relay flags]: https://metrics.torproject.org/bandwidth-flags.html [Tor Metrics: Advertised bandwidth by IP version]: https://metrics.torproject.org/advbw-ipv6.html [Tor Metrics: Fraction of connections used uni-/bidirectionally]: https://metrics.torproject.org/connbidirect.html [Tor Metrics: Traffic]: https://metrics.torproject.org/bandwidth-flags.html [Tor Specification]: https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt

3 3

CVE-2020-8516 Hidden Service deanonymization
by juanjo 06 Feb '20

06 Feb '20

Since no one is posting it here and talking about it, I will post it. https://nvd.nist.gov/vuln/detail/CVE-2020-8516 The guy: http://www.hackerfactor.com/blog/index.php?/archives/868-Deanonymizing-Tor-… Is this real? Are we actually not verifying if the IP of the Rend is a node in the Tor network?

6 6

Proposal 312: Automatic Relay IPv6 Addresses
by teor 04 Feb '20

04 Feb '20

Hi, Here is an initial draft of Proposal 312: Automatic Relay IPv6 Addresses. This proposal includes: * relay auto IPv6 addresses, and * relay auto IPv6 ORPorts. This is the second of 3 proposals: * Proposal 311: Relay IPv6 Reachability * Proposal 312: Automatic Relay IPv6 Addresses * Proposal 313: Relay IPv6 Statistics (I haven't written the final one yet.) I also want to make some minor changes to Proposal 306, so that bridge IPv6 behaviour stays in sync with client IPv6 behaviour. (See section 7 of this proposal for details.) There is still one TODO item in the proposal, about Tor's current behaviour. If you know the answer, please let me know. The full text is included below, and it is also available as a GitHub pull request: https://github.com/torproject/torspec/pull/105 The related tickets are #33073 (proposal) and #5940 (implementation): https://trac.torproject.org/projects/tor/ticket/33073 https://trac.torproject.org/projects/tor/ticket/5940 Please feel free to reply on this list, or via GitHub pull request comments. Filename: 312-relay-auto-ipv6-addr.txt Title: Tor Relays Automatically Find Their IPv6 Address Author: teor Created: 28-January-2020 Status: Draft Ticket: #33073 0. Abstract We propose that Tor relays (and bridges) should automatically find their IPv6 address, and use it to publish an IPv6 ORPort. For some relays to find their IPv6 address, they may need to fetch some directory documents from directory authorities over IPv6. (For anonymity reasons, bridges are unable to fetch directory documents over IPv6, until clients start to do so.) 1. Introduction Tor relays (and bridges) currently find their IPv4 address, and use it as their ORPort and DirPort address when publishing their descriptor. But relays and bridges do not automatically find their IPv6 address. However, relay operators can manually configure an ORPort with an IPv6 address, and that ORPort is published in their descriptor in an "or-address" line (see [Tor Directory Protocol]). Many relay operators don't know their relay's IPv4 or IPv6 addresses. So they rely on Tor's IPv4 auto-detection, and don't configure an IPv6 address. When operators do configure an IPv6 address, it's easy for them to make mistakes. IPv6 ORPort issues are a significant source of relay operator support requests. Implementing IPv6 address auto-detection, and IPv6 ORPort reachability checks (see [Proposal 311: Relay IPv6 Reachability]) will increase the number of working IPv6-capable relays in the tor network. 2. Scope This proposal modifies Tor's behaviour as follows: Relays, bridges, and directory authorities: * automatically find their IPv6 address, and * for consistency between IPv4 and IPv6 detection: * start using IPv4 ORPort and DirPort for IPv4 address detection, and * re-order IPv4 address detection methods. Relays and directory authorities (but not bridges): * fetch some directory documents over IPv6. For anonymity reasons, bridges are unable to fetch directory documents over IPv6, until clients start to do so. (See [Proposal 306: Client Auto IPv6 Connections].) This proposal makes a small, optional change to existing client behaviour: * clients also check IPv6 addresses when rotating TLS keys for new networks. In addition to the changes to IPv4 address resolution, most of which won't affect clients. (Because they do not set Address, ORPort, or DirPort.) Throughout this proposal, "relays" includes directory authorities, except where they are specifically excluded. "relays" does not include bridges, except where they are specifically included. (The first mention of "relays" in each section should specifically exclude or include these other roles.) When this proposal describes Tor's current behaviour, it covers all supported Tor versions (0.3.5.7 to 0.4.2.5), as of January 2020, except where another version is specifically mentioned. 3. Finding Relay IPv6 Addresses We propose that tor relays (and bridges) automatically find their IPv6 address, and use it to publish an IPv6 ORPort. For some relays to find their IPv6 address, they may need to fetch some directory documents from directory authorities over IPv6. (For anonymity reasons, bridges are unable to fetch directory documents over IPv6, until clients start to do so.) 3.1. Current Relay IPv4 Address Implementation Currently, all relays (and bridges) must have an IPv4 address. IPv6 addresses are optional for relays. Tor currently tries to find relay IPv4 addresses in this order: 1. the Address torrc option 2. the address of the hostname (resolved using DNS, if needed) 3. a local interface address (by making a self-connected socket, if needed) 4. an address reported by a directory server (using X-Your-Address-Is) When using the Address option, or the hostname, tor supports: * an IPv4 address literal, or * resolving an IPv4 address from a hostname. If tor is running on the public network, and an address isn't globally routable, tor ignores it. (If it was explicitly set in Address, tor logs an error.) If there are multiple valid addresses, tor chooses: * the first address returned by the resolver, * the first address returned by the local interface API, or * the latest address returned by a directory server. 3.2. Finding Relay IPv6 Addresses We propose that relays (and bridges) try to find their IPv6 address. For consistency, we also propose to change the address resolution order for IPv4 addresses. We use the following general principles to choose the order of IP address methods: * Explicit is better than Implicit, * Local Information is better than a Remote Dependency, * Trusted is better than Untrusted, and * Reliable is better than Unreliable. Within these constraints, we try to find the simplest working design. Therefore, we propose that tor tries to find relay IPv4 and IPv6 addresses in this order: 1. the Address torrc option 2. the advertised ORPort address 3. the advertised DirPort address (IPv4 only; relays, not bridges) 4. a local interface address (by making a self-connected socket, if needed) 5. the address of the host's own hostname (resolved using DNS, if needed) 6. an address reported by a directory server (using X-Your-Address-Is) (Each of these address resolution steps is described in more detail, in its own subsection.) While making these changes, we want to preserve tor's existing behaviour: * resolve Address using the local resolver, if needed, * ignore private addresses on public tor networks, and * when there are multiple valid addresses, choose the first or latest address, as appropriate. 3.2.1. Make the Address torrc Option Support IPv6 First, we propose that relays (and bridges) use the Address torrc option to find their IPv4 and IPv6 addresses. There are two cases we need to cover: 1. Explicit IP addresses: * allow the option to be specified up to two times, * use the IPv4 address for IPv4, * use the IPv6 address for IPv6. Configuring two addresses in the same address family is a config error. 2. Hostnames / DNS names: * allow the option to be specified up to two times, * look up the configured name, * use the first IPv4 and IPv6 address returned by the resolver, and Resolving multiple addresses in the same address family is not a runtime error, but only the first address from each family will be used. These lookups should ignore private addresses on public tor networks. If multiple IPv4 or IPv6 addresses are returned, the first public address from each family should be used. We should also support the following combinations: A. IPv4 Address / hostname (for IPv6 only), B. IPv6 Address / hostname (for IPv4 only), C. IPv4 Address only / try to guess IPv6, then check its reachability (see section 4.3.1 in [Proposal 311: Relay IPv6 Reachability]), and D. IPv6 Address only / guess IPv4, then its reachability must succeed. There are also similar configurations where a hostname is configured, but it only provides IPv4 or IPv6 addresses. Combination C is the most common legacy configuration. We want to support the following outcomes for legacy configurations: * automatic upgrades to guessed and reachable IPv6 addresses, * continuing to operate on IPv4 when the IPv6 address can't be guessed, and * continuing to operate on IPv4 when the IPv6 address has been guessed, but it is unreachable. At this time, we do not propose guessing multiple IPv4 or IPv6 addresses and testing their reachability (see section 3.4.2). It is an error to configure an Address option with a private IPv4 or IPv6 address, or with a hostname that does not resolve to any publicly routable IPv4 or IPv6 addresses. If the Address option is not configured for IPv4 or IPv6, or the hostname lookups do not provide both IPv4 and IPv6 addresses, address resolution should go to the next step. 3.2.2. Use the Advertised ORPort IPv4 and IPv6 Addresses Next, we propose that relays (and bridges) use the first advertised ORPort IPv4 and IPv6 addresses, as configured in their torrc. The ORPort address may be a hostname. If it is, tor should try to use it to resolve an IPv4 and IPv6 address, and open ORPorts on the first available IPv4 and IPv6 address. Tor should respect the IPv4Only and IPv6Only port flags, if specified. (Tor currently resolves IPv4 addresses in ORPort lines. It may not look for an IPv6 address.) Relays (and bridges) currently use the first advertised ORPort IPv6 address as their IPv6 address. We propose to use the first advertised IPv4 ORPort address in a similar way, for consistency. Therefore, this change may affect existing relay IPv4 addressses. We expect that a small number of relays may change IPv4 address, from a guessed IPv4 address, to their first advertised IPv4 ORPort address. In rare cases, relays may have been using non-advertised ORPorts for their addresses. This change may also change their addresses. We propose ignoring private configured ORPort addresses on public tor networks. (Binding to private ORPort addresses is supported, even on public tor networks, for relays that use NAT to reach the Internet.) If an ORPort address is private, address resolution should go to the next step. 3.2.3. Use the Advertised DirPort IPv4 Address Next, we propose that relays use the first advertised DirPort IPv4 address, as configured in their torrc. The following DirPort configurations can not be used for address resolution, because they are not supported: * bridge DirPorts, and * advertised IPv6 DirPorts. The DirPort address may be a hostname. If it is, tor should try to use it to resolve an IPv4 address, and open a DirPort on the first available IPv4 address. Tor should only look for IPv6 addresses if the IPv6Only port flag is specified. (Since advertised IPv6 DirPorts are not supported, a working configuration may also require the NoAdvertise flag.) Relays currently use the first advertised ORPort IPv6 address as their IPv6 address. We propose to use the first advertised IPv4 DirPort address in a similar way, for consistency. Therefore, this change may affect existing relay IPv4 addressses. We expect that a very small number of relays may change IPv4 address, from a guessed IPv4 address, to their first advertised IPv4 DirPort address. (But we expect that most relays that change will be using their ORPort address.) We propose ignoring private configured DirPort addresses on public relays. (Binding to private DirPort addresses is supported, for networks that use NAT.) If a DirPort address is private, address resolution should go to the next step. 3.2.4. Use Local Interface IPv6 Address Next, we propose that relays (and bridges) use publicly routable addresses from the OS interface addresses or routing table, as their IPv4 and IPv6 addresses. Tor has local interface address resolution functions, which support most major OSes. Tor uses these functions to guess its IPv4 address. We propose using them to also guess tor's IPv6 address. We also propose modifying the address resolution order, so interface addresses are used before the local hostname. This decision is based on our principles: interface addresses are local, trusted, and reliable; hostname lookups may be remote, untrusted, and unreliable. Some developer documentation also recommends using interface addresses, rather than resolving the host's own hostname. For example, on recent versions of macOS, the man pages tell developers to use interface addresses (getifaddrs) rather than look up the host's own hostname (gethostname and getaddrinfo). Unfortunately, these man pages don't seem to be available online, except for short quotes (see [getaddrinfo man page] for the relevant quote). If the local interface addresses are unavailable, tor opens a self-connected UDP socket to a publicly routable address, but doesn't actually send any packets. Instead, it uses the socket APIs to discover the interface address for the socket. Tor already ignores private IPv4 interface addresses on public relays. (Binding to private DirPort addresses is supported, for networks that use NAT.) We propose to also ignore private IPv6 interface addresses. If all IPv4 or IPv6 interface addresses are private, address resolution should go to the next step. 3.2.5. Use Own Hostname IPv6 Addresses Next, we propose that relays (and bridges) get their local hostname, look up its addresses, and use them as its IPv4 and IPv6 addresses. We propose to use the same underlying lookup functions to look up the IPv4 and IPv6 addresses for: * the Address torrc option (see section 3.2.1), and * the local hostname. However, OS APIs typically only return a single hostname. Even though the hostname lookup may use remote DNS, we propose to use it on directory authorities, to maintain compatibility with current configurations. Even if it is remote, we expect the configured DNS to be somewhat trusted by the operator. The hostname lookup should ignore private addresses on public relays. If multiple IPv4 or IPv6 addresses are returned, the first public address from each family should be used. If all IPv4 or IPv6 hostname addresses are private, address resolution should go to the next step. 3.2.6. Use Directory Header IPv6 Addresses Finally, we propose that relays get their IPv4 and IPv6 addresses from the X-Your-Address-Is HTTP header in tor directory documents. To support this change, we propose that relays start fetching directory documents over IPv4 and IPv6. We propose that bridges continue to only fetch directory documents over IPv4, because they try to imitate clients. (Most clients only fetch directory documents over IPv4, a few clients are configured to only fetch over IPv6.) When client behaviour changes to use both IPv4 and IPv6 for directory fetches, bridge behaviour can also change to match. (See section 3.4.1 and [Proposal 306: Client Auto IPv6 Connections].) We propose that directory authorities should ignore addresses in directory headers. Allowing other authorities (or relays?) to change a directory authority's published IP address may lead to security issues. Instead, if interface and hostname lookups fail, tor should stop address resolution, and return a permanent error. (And issue a log to the operator, see below.) We propose to use a simple load balancing scheme for IPv4 and IPv6 directory requests: * choose between IPv4 and IPv6 directory requests at random. We do not expect this change to have any load-balancing impact on the public tor network, because the number of relays is much smaller than the number of clients. However, the 6 directory authorities with IPv6 enabled may see slightly more directory load, particularly over IPv6. To support this change, tor should also change how it handles IPv6 directory failures on relays: * avoid recording IPv6 directory failures as remote relay failures, because they may actually be due to a lack of IPv6 connectivity on the local relay, and * issue IPv6 directory failure logs at notice level, and rate-limit them to one per hour. If a relay is: * explicitly configured with an IPv6 address, or * a publicly routable, reachable IPv6 address is discovered in an earlier step, tor should start issuing IPv6 directory failure logs at warning level. (Alternately, tor could stop doing IPv6 directory requests entirely. But we prefer designs where all relays behave in a similar way, regardless of their internal state.) For some more complex directory load-balancing schemes, see section 3.5.2. Tor already ignores private IPv4 addresses in directory headers. We propose to also ignore private IPv6 addresses in directory headers. If all IPv4 and IPv6 addresses in directory headers are private, address resolution should pause, and return a temporary error. Whenever address resolution fails, tor should warn the operator to set the Address torrc option for IPv4 and IPv6. (If IPv4 is available, and only IPv6 is missing, the log should be at notice level.) Address resolution should continue the next time tor receives a directory header containing a public IPv4 or IPv6 address. 3.2.7. Disabling IPv6 Address Resolution Relays that have a reachable IPv6 address, but that address is unsuitable for the relay, need to be able to disable IPv6 address resolution. Based on [Proposal 311: Relay IPv6 Reachability], and this proposal, those relays would: * discover their IPv6 address, * open an IPv6 ORPort, * find it reachable, * publish a descriptor containing that IPv6 ORPort, * have the directory authorities find it reachable, * have it published in the consensus, and * have it used by clients. Currently, relays are required to have an IPv4 address. So if the guessed IPv4 address is unsuitable, operators can set the Address option to a suitable IPv4 address. But IPv6 addresses are optional, so relay operators may need to disable IPv6 entirely. We propose a new torrc-only option, AddressDisableIPv6. This option is set to 0 by default. If the option is set to 1, tor disables IPv6 address resolution, IPv6 ORPorts, IPv6 reachability checks, and publishing an IPv6 ORPort in its descriptor. 3.2.8. Automatically Enabling an IPv6 ORPort We propose that relays that discover their IPv6 address, should open an ORPort on that address, and test its reachability (see [Proposal 311: Relay IPv6 Reachability], particularly section 4.3.1). The ORPort should be opened on the port configured in the relay's ORPort torrc option. Relay operators can use the IPv4Only and IPv6Only options to configure different ports for IPv4 and IPv6. If both reachability checks succeed, relays should publish their IPv4 and IPv6 ORPorts in their descriptor. If only the IPv4 ORPort check succeeds, and the IPv6 address was guessed (rather than being explicitly configured), then relays should publish their IPv4 ORPort in their descriptor. 3.3. Consequential Tor Client Changes We do not propose any required client address resolution changes at this time. However, clients will use the updated address resolution functions to detect when they are on a new connection, and therefore need to rotate their TLS keys. This minor client change allows us to avoid keeping an outdated version of the address resolution functions, which is only for client use. Clients should skip address resolution steps that don't apply to them, such as: * the ORPort option, * the DirPort option, and * the Address option, if it becomes a relay module option. 3.4. Alternative Address Resolution Designs We briefly mention some potential address resolution designs, and the reasons that they were not used in this proposal. (Some designs may be proposed for future Tor versions, but are not necessary at this time.) 3.4.1. Future Bridge IPv6 Address Resolution Behaviour When clients automatically fetch directory documents via relay IPv4 and IPv6 ORPorts by default, bridges should also adopt this dual-stack behaviour. (For example, see [Proposal 306: Client Auto IPv6 Connections].) When bridges fetch directory documents via IPv6, they will be able to find their IPv6 address using directory headers (see 3.2.6). 3.4.2. Guessing Muliple IPv4 or IPv6 Addresses We avoid designs which guess (or configure) multiple IPv4 or IPv6 addresses, test them all for reachability, and choose one that works. Using multiple addresses is rare, and the code to handle it is complex. It also requires careful design to avoid: * conflicts between multiple relays on the same address (tor allows up to 2 relays per IPv4 address) * flapping / race conditions / address switching * rare 3.4.3. Rejected Address Resolution Designs We reject designs that try all the different address resolution methods, score addresses, and then choose the address with the highest score. These designs are a generalisation of designs that try different methods in a set order (like this proposal). They are more complex than required. Complex designs can confuse operators, particularly when they fail. Operators should not need complex address resolution in tor: most relay addresses are fixed, or change occasionally. And most relays can reliably discover their address using directory headers, if all other methods fail. If complex address resolution is required, it can be configured using a dynamic DNS name in the Address torrc option, or via the control port. We also avoid designs that use any addresses other than the first (or latest) valid IPv4 and IPv6 address. These designs are more complex, and they don't have clear benefits: * sort addresses numerically (avoid address flipping) * sort addresses by length, then numerically (also minimise consensus size) * store a list of previous addresses in the state file, and use the most recently used address that's currently available. Operators who want to avoid address flipping should set the Address option in the torrc. Operators who want to minimise the size of the consensus should use all-zero IPv6 host identifiers. 3.5. Optional Efficiency and Reliability Changes We propose some optional changes for efficiency and reliability, and describe their impact. Some of these changes may be more appropriate in future releases, or along with other proposed features. 3.5.1. Only Use Authenticated Directory Header IPv4 and IPv6 Addresses We propose this optional change, to improve relay address accuracy and reliability. Relays should only use: * authenticated directory fetches, * to directory authorities, to discover their own IPv4 and IPv6 addresses. (Both changes are optional, and they can be made separately.) Tor supports authenticated, encrypted directory fetches using BEGINDIR over ORPorts (see the [Tor Specification] for details). Relays currently fetch unencrypted directory documents over DirPorts. The directory document itself is signed, but the HTTP headers are not authenticated. (Clients and bridges only fetch directory documents using authenticated directory fetches.) Using authenticated directory headers for relay addresses: * avoids caches (or other machines) mangling X-Your-Address-Is headers in transit, and * avoids attacks where directories are deliberately given an incorrect IP address. To make this change, we need to modify two different parts of tor: * when making directory requests, relays should fetch some directory documents using BEGINDIR over ORPorts, and * when using the X-Your-Address-Is HTTP header to guess their own IPv4 or IPv6 addresses, relays ignore directory documents that were not fetched using BEGINDIR over ORPorts. Optionally, relays should also ignore addresses from other relays: * when using the X-Your-Address-Is HTTP header to guess their own IPv4 or IPv6 addresses, relays ignore directory documents that were not fetched from directory authorities. Ideally, we would like all relays to do all their directory fetches: * using BEGINDIR over ORPorts, and * to directory authorities. However, this change may be unsustainable during high network load (see [Ticket 33018: Dir auths using an unsustainable 400+ mbit/s]). Therefore, we propose a simple load-balancing scheme between address resolution and non-address resolution requests: * when relays do not know their own IP addresses, they should make as many address resolution directory fetches as is sustainable, and * when relays know their own IP addresses, they should make an occasional address resolution directory fetch, to learn if their address has changed. We use the load-balancing criteria in the next section, to select the ratio between: * ORPort connections to directory authorities, and * DirPort connections to directory mirrors. 3.5.1.1. General Load Balancing Criteria We propose the following criteria for choosing load-balancing ratios: The selected ratios should be chosen based on the following factors: * the current number of directory fetches that a relay makes: * when bootstrapping with an empty cache directory, and * in a steady state (per hour, or per new consensus), (these numbers aren't currently collected by tor, so we may need to write some extra code to include them in the heartbeat logs), * relays need to discover their IPv4 and IPv6 addresses to bootstrap, * it only takes one successful directory fetch from one authority for a relay to discover its IP address, * BEGINDIR over ORPort requires and TLS connection, and some additional tor cryptography, so it is more expensive for authorities than a DirPort fetch (and it can not be cached by a HTTP cache), * minimising wasted CPU (and bandwidth) on IPv4-only relays, and * other potential changes to relay directory fetches (see [Ticket 33018: Dir auths using an unsustainable 400+ mbit/s]) The selected ratios should allow almost all relays to update both their IPv4 and IPv6 addresses: * at least twice when they bootstrap (to allow for fetch failures), and * at least once per directory fetch (or per hour). In this proposal, relays choose between IPv4 and IPv6 directory fetches at random (see section 3.2.6 for more detail). See the next section for an alternative load-balancing scheme. 3.5.2. Load Balancing Between IPv4 and IPv6 Directories We propose this optional change, to improve the load-balancing between IPv4 and IPv6 directories, when used by relays to find their IPv4 and IPv6 addresses (see section 3.2.6). This change may only be necessary if the following changes result in poor load-balancing, or other relay issues: * randomly selecting IPv4 or IPv6 directories (see section 3.2.6), or * only using directory headers for addresses when they come from directory authorities, via an authenticated connection (see section 3.5.1). We propose a new torrc option and consensus parameter: MaxNumIPv4DirectoryAttempts. This option limits the number of IPv4 directory requests, before the relay makes an IPv6 directory request. It should only apply to attempts that are expected to provide a usable IPv4 or IPv6 address in their directory header. (Based on sections 3.2.6 and 3.5.1.) The design is similar to MaxNumIPv4BootstrapAttempts in [Proposal 306: Client Auto IPv6 Connections]. Here is a quick sketch of the design: * when MaxNumIPv4DirectoryAttempts is reached, select an IPv6-capable directory, and make an IPv6 connection attempt, * use a directory authority, or an ORPort, if required (see section 3.5.1), * use a default value between 2 and 4: * the ideal value for load-balancing is >= 2 (because 6/9 directory authorities are already on IPv6) * the ideal value for minimising failures is ~4 (because relays won't waste too much CPU or bandwidth) * choose the default value based on the load-balancing criteria in section 3.5.1.1. Alternately, we could wait until [Proposal 306: Client Auto IPv6 Connections] is implemented, and use the directory fetch design from that proposal. 3.5.3. Detailed Address Resolution Logs We propose this optional change, to help diagnose relay address resolution issues. Relays (and bridges) should log the address chosen using each address resolution method, when: * address resolution succeeds, * address resolution fails, * reachability checks fail, or * publishing the descriptor fails. These logs should be rate-limited separately for successes and failures. The logs should tell operators to set the Address torrc option for IPv4 and IPv6 (if available). 3.5.4. Add IPv6 Support to is_local_addr() We propose this optional change, to improve the accuracy of IPv6 address detection from directory documents. Directory servers use is_local_addr() to detect if the requesting tor instance is on the same local network. If it is, the directory server does not include the X-Your-Address-Is HTTP header in directory documents. Currently, is_local_addr() checks for: * an internal IPv4 or IPv6 address, or * the same IPv4 /24 as the directory server. We propose also checking for: * the same IPv6 /48 as the directory server. We choose /48 because it is typically the smallest network in the global IPv6 routing tables, and it was previously the recommended per-customer network block. (See [RFC 6177: IPv6 End Site Address Assignment].) Tor currently uses: * IPv4 /8 and IPv6 /16 for port summaries, * IPv4 /16 and IPv6 /32 for path selection (avoiding relays in the same network block). 3.5.5. Add IPv6 Support to AuthDirMaxServersPerAddr We propose this optional change, to improve the health of the network, by rejecting too many relays on the same IPv6 address. Modify get_possible_sybil_list() so it takes an address family argument, and returns a list of IPv4 or IPv6 sybils. Use the modified get_possible_sybil_list() to exclude relays from the authority's vote, if there are more than AuthDirMaxServersPerAddr on the same IPv4 or IPv6 address. Since these relay exclusions happen at voting time, they do not require a new consensus method. 3.5.6. Use a Local Interface Address on the Default Route We propose this optional change, to improve the accuracy of local interface IPv4 and IPv6 address detection (see section 3.2.4). Rewrite the get_interface_address*() functions to choose an interface address on the default route, or to sort default route addresses first in the list of addresses. (If the platform API allows us to find the default route.) For more information, see [Ticket 12377: Prefer default route when checking local interface addresses]. This change may not be necessary, because the directory header IP address method will find the IP address of the default route, in most cases (see section 3.2.6). 3.5.7. Add IPv6 Support Using gethostbyname2() We propose these optional changes, to add IPv6 support to hostname resolution on older OSes. These changes affect: * the Address torrc option, when it is a hostname (see section 3.2.1), and * automatic hostname resolution (see section 3.2.5). Use gethostbyname2() to add IPv6 support to hostname resolution on older OSes, which don't support getaddrinfo(). But this change may be unnecessary, because: * Linux has used getaddrinfo() by default since glibc 2.20 (2014) * macOS has recommended getaddrinfo() since before 2006 * since macOS adopts BSD changes, most BSDs would have switched to getaddrinfo() in a similar timeframe * Windows has supported getaddrinfo() since Windows Vista; tor's minimum supported Windows version is Vista. See [Tor Supported Platforms] for more details. When looking up hostnames using gethostbyname() or gethostbyname2(), if the first address is a private address, we may want to look at the entire list of addresses. Some struct hostent versions (example: current macOS) also have a h_addr_list rather than h_addr. (They define h_addr as h_addr_list[0], for backwards compatibility.) However, having private and public addresses resolving from the same hostname is a rare configuration, so we may not need to make this change. (On OSes that support getaddrinfo(), tor searches the list of addresses for a publicly routable address.) As an alternative, if we believe that all supported OSes have getaddrinfo(), we could simply remove the gethostbyname() code, rather than trying to modify it to work with IPv6. Most relays can reliably discover their address using directory headers, if all other methods fail. Or operators can set the Address torrc option to an IPv4 or IPv6 literal. 3.5.8. Change Relay OutboundBindAddress Defaults We propose this optional change, to improve the reliability of IP address-based filters in tor. For example, the tor network treats relay IP addresses differently when: * resisting denial of service, and * selecting canonical, long-term connections. (See [Ticket 33018: Dir auths using an unsustainable 400+ mbit/s] for the initial motivation for this change: resisting significant bandwidth load on directory authorities.) Now that tor knows its own addresses, we propose that relays (and bridges) set their IPv4 and IPv6 OutboundBindAddress to these discovered addresses, by default. If binding fails, tor should fall back to an unbound socket. Operators would still be able to set a custom IPv4 and IPv6 OutboundBindAddress, if needed. Currently, tor doesn't bind to a specific address, unless OutboundBindAddress is configured. So on relays with multiple IP addresses, the outbound address comes from the chosen route (usually the default route). 4. Directory Protocol Specification Changes We propose explicitly supporting IPv6 X-Your-Address-Is HTTP headers in the tor directory protocol. We propose the following changes to the [Tor Directory Protocol] specification, in section 6.1: Servers MAY include an X-Your-Address-Is: header, whose value is the apparent IPv4 or IPv6 address of the client connecting to them. IPv6 addresses SHOULD/MAY (TODO) be formatted enclosed in square brackets. TODO: require brackets? What does Tor currently do? For directory connections tunneled over a BEGIN_DIR stream, servers SHOULD report the IP from which the circuit carrying the BEGIN_DIR stream reached them. Servers SHOULD disable caching of multiple network statuses or multiple server descriptors. Servers MAY enable caching of single descriptors, single network statuses, the list of all server descriptors, a v1 directory, or a v1 running routers document, with appropriate expiry times (around 30 minutes). Servers SHOULD disable caching of X-Your-Address-Is headers. 5. Test Plan We provide a quick summary of our testing plans. 5.1. Test Find Relay IPv6 Addresses We propose to test these changes using chutney networks. However, chutney creates a limited number of configurations, so we also need to test these changes with relay operators on the public network. Therefore, we propose to test these changes on the public network with a small number of relays and bridges. Once these changes are merged, volunteer relay and bridge operators will be able to test them by: * compiling from source, * running nightly builds, or * running alpha releases. 5.2. Test Existing Features We will modify and test these existing features: * Find Relay IPv4 Addresses We do not plan on modifying these existing features: * relay address retries * existing warning logs But we will test that they continue to function correctly, and fix any bugs triggered by the modifications in this proposal. 6. Ongoing Monitoring To monitor the impact of these changes, relays should collect basic IPv4 and IPv6 connection and bandwidth statistics (see [Proposal 313: Relay IPv6 Statistics]). We may also collect separate statistics on connections from: * clients (and bridges, because they act like clients), and * other relays (and authorities, because they act like relays). Some of these statistics may be included in tor's heartbeat logs, making them accessible to relay operators. We do not propose to collect additional statistics on: * bridges, * address resolution, * circuit counts, or * failure rates. Collecting statistics like these could impact user privacy, or relay security. 7. Changes to Other Proposals [Proposal 306: Client Auto IPv6 Connections] needs to be modified to keep bridge IPv6 behaviour in sync with client IPv6 behaviour. (See section 3.2.6.) References: [getaddrinfo man page]: See the quoted section in https://stackoverflow.com/a/42351676 [Proposal 306: Client Auto IPv6 Connections]: One possible design for automatic client IPv4 and IPv6 connections is at https://gitweb.torproject.org/torspec.git/tree/proposals/306-ipv6-happy-eye… (TODO: modify to include bridge changes with client changes) [Proposal 311: Relay IPv6 Reachability]: https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-rea… [Proposal 313: Relay IPv6 Statistics]: https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-sta… (TODO) [RFC 6177: IPv6 End Site Address Assignment]: https://tools.ietf.org/html/rfc6177#page-7 [Ticket 12377: Prefer default route when checking local interface addresses]: https://trac.torproject.org/projects/tor/ticket/12377 [Ticket 33018: Dir auths using an unsustainable 400+ mbit/s]: https://trac.torproject.org/projects/tor/ticket/33018 [Tor Directory Protocol]: (version 3) https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt [Tor Specification]: https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt [Tor Supported Platforms]: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/Support… (End)

4 10

Options for Congestion Control in Tor-like Networks
by Mike Perry 30 Jan '20

30 Jan '20

This near-book-length mail is a pre-proposal to help us review previous congestion control ideas for Tor, and brainstorm new ones. My goal is to turn this material into one or more proposals and provide ways of evaluating these options. With proper voodoo, maybe we can resurrect a couple oldies from the graveyard of mixnet lore, and remix them into some Vaporwave bangers. If we're real lucky with some fresh ideas, maybe we can even make a hip hop Internet streaming chart buster from this mix. But long mail is looong, ok? It's sooo long it has its own soundtrack. Grab your favorite links from the references at the end, maybe print everything out, find the nearest sofa or bed, and curl up with it all. Flip on some Pretty Lights. Let's get that flow Finally Moving. Turn off the tv! The flow is More Important Than Michael Jordan. Listen: the flow is Hot Like Sauce. I hope you're an Organ Donor - Extended Overhaul might be necessary. We're going on a mental Midnight Voyage by Ghostland Observatory. But don't worry, we'll make it by 4 AM, without being trapped there, I swear. Hopefully that's not Pushing Time with ya Ample Mammal(s). Relax: there will still be time for some Codeine(g) Dreaming. If you finish reading all of it before you go ROCKABYE BABY, you're definitely a High Roller. All of this is almost exactly an hour of Bumpin' In The Voodoo with Manic Focus. If you'd rather take your time and chillax with something Muy Tranquilo but less Gramatik, just put on some Blank Banshee. But don't fall asleep, or George Clanton will Kill You In Bed. Ready? Ok, back to serious business. Motivation: Load balancing (TorFlow/sbws), circuit build timeout cutoffs (CBT), and QoS (KIST+EWMA) have provided huge perf wins for Tor. Tuning and improving these systems will continue to provide latency improvements in the average case, but congestion control is the missing piece we need for the network to operate properly at high utilization levels. Congestion control is also necessary to increase the throughput of the network at even at low utilization levels. Historically, Tor performance shows high correlation to network utilization levels, and I believe this is largely due to the effects of ephemeral congestion: https://lists.torproject.org/pipermail/tor-scaling/2019-June/000051.html In fact, Section 6.3 of https://www.freehaven.net/anonbib/cache/murdoch-pet2008.pdf uses Pollaczek-Khinchin queuing theory to show that expected Tor queue latency is proportional to network utilization divided by spare capacity, if queues are not otherwise bounded somehow (by congestion control). Summary: The rest of this post is organized as follows: First, I go over TCP and ECN, because I build upon those for a couple new ideas. Then, I summarize Tor's current SENDME flow control and review its many failings. Then, I review past attempts at congestion control for Tor in the research literature. Then, I propose four new candidate ideas. Finally, I conclude the post with some ideas for evaluation and further analysis. Unless you are a congestion control expert who is also deeply familiar with Tor's many failed attempts in this area, I strongly recommend wading through this post in order. The summaries are meant to get you up to speed without having to do quite as much reading as I did, and they are filled with in-line URL references in case you want to dig deeper. If you still want to skip ahead to the highlights, search this mail for [PROPOSAL_CANDIDATE]. There are four such sections. Those sections build on other ideas, though. For those, search for [TCP_HISTORY], [BOOTLEG_RTT_TOR], [FORWARD_ECN_TOR], and [BACKWARD_ECN_TOR]. Search for [TRACK_LISTING] to find a summary table of all the new ideas from this post, with their associated key properties. Crucially absent from this pre-proposal post is the closely related discussion of QoS/scheduling, which we need to make decisions about which circuit(s) to select for delivering congestion control signals, when congestion occurs. For now, this post just handwaves this piece as "use either EWMA or something RED-like". There is much literature on these mechanisms, enough to warrant separate treatment, but thankfully the choice of which to use is orthogonal to the choice of congestion control signal delivery mechanism. -I. History of Internet Congestion Control [TCP_HISTORY] In TCP, each endpoint maintains send and receive buffers of packets, called windows. The receive window holds packets until a contiguous chunk is received, at which point data is delivered to the application and an acknowledgement packet is sent to the other end. The send window size is doubled every ack until the first packet is dropped, after which it is increased by 1 for each acked packet, and halved for each drop. This is called Slow Start with AIMD (Additive-Increase Multiplicative-Decrease). Packets that are not acked are retransmitted from the send window buffer after a timeout of one RTT (Round Trip Time). Drops are caused by intermediate routers' fixed-size queues being full, but can also be caused by poor network conditions (which leads to sub-optimal throughput). In this way, TCP responds to congestion anywhere on the path, and takes around one RTT to detect said congestion. Even this detailed summary is a simplified model. In reality, TCP is way more complicated than that. The real thing has like 23 states, and a whole bunch of kruft from the 80s and 90s that we'll want to cut out of our mix. We just need the key hooks. Decades later, Explicit Congestion Notification (ECN) was proposed to allow routers to explicitly set a flag on TCP packets to signal that their queue is getting full, instead of dropping packets. For abstruse compatibility and reliability reasons, when a router adds this flag to a packet, it is sent all the way to one endpoint. That endpoint *then* re-adds the flag to acks going in the other direction, all the way to the other endpoint, who then backs off as if it detected a packet drop. This is called Forward ECN: https://tools.ietf.org/html/rfc3168#section-6.1 Unfortunately, because of the requirement to bounce the congestion notification flag all the way off the other endpoint, it *still* takes Forward ECN at least one RTT to respond to congestion, and so almost no intermediate routers have bothered to deploy it -- the gains are only marginal. Also interesting for our purposes is the vastly simplified Backwards ECN that uses separate ICMP signaling to eliminate the endpoint echo RTT for much faster responsiveness to congestion: https://tools.ietf.org/html/draft-salim-jhsbnns-ecn-00 Backwards ECN was never widely deployed either, because ICMP is easy to spoof+spam, requires extra packet overhead, many routers filter it, and no strong authentication binds it to the TCP connection. But enough history! Onward! @. Status quo of the Tor flow: SENDME sadness and pain Tor has a window-based flow control system called SENDMEs. This system does not provide congestion control, as window sizes are hard-coded constants. I believe it is important to review this system in detail, so we can avoid making any of the same mistakes again. Bear with me. Recall that Tor circuits are 3 hops long when contacting the Internet (Guard, Middle, Exit), and 7 hops long when a client contacts an onion service. TCP is used for connection in between these hops. One or more reliable streams are multiplexed inside each circuit. Each endpoint of a circuit (client and Exit, or client and onion service) maintains a remaining-to-send cell count for each circuit and stream, which is refilled when it receives a SENDME from the other end. Each time it sends a cell, it decrements this count. When this count reaches 0 without receiving corresponding SENDMEs, it will stop sending data. The initial value of these counts is 1000 cells for circuits and 500 cells for streams, and each SENDME refills 100 cell counts for circuits, and 50 cell counts for streams. Crucially: no backpressure happens on interior relay connections for this windowing system. Instead, we just queue. If you have a TCP background, you may find it easier to mentally replace "SENDME" with "ACK", and you won't be far off. Its just that our acks always ack a fixed amount of cells, and the window size is also fixed. In the steady state stage, window updates of 50 or 100 cells per SENDME results in an upper bound of performance inversely proportional to half the circuit RTT. This is 2*CELL_SIZE*50/RTT for streams, and 2*CELL_SIZE*100/RTT for circuits. For streams with a ~100msec RTT, this is ~500Kbytes/sec. This means that once the Tor network has enough capacity for RTT to be close to link transit time (ie: no queue delay), adding more Tor relays will *not* make Tor faster. This also means that onion service throughput will almost always be much lower than Exit throughput. The RTT is much much higher for 7 hop onion circuits than for 3 hop exits, so even if there is plenty of spare network capacity, Onion service sites will *never* download as fast as their clearweb equivalents with our current flow control. All of this is done just to provide flow control, so that data flow can stop in the event of endpoint stalls and/or interior relay failure. It does not actually provide fairness or limit congestion when multiple clients are used. Additionally, because nothing proves the endpoint has read the data, a client that does not read data but keeps sending SENDMEs to the Exit can deliberately force queues to build up in the Guard node (due to the client-blocked TCP layer), to induce OOM conditions and crashes. We need to upgrade all clients and all Exit relays to fix this problem, which will take quite some time: https://gitweb.torproject.org/torspec.git/tree/proposals/289-authenticated-… Even with honest or authenticated behavior, the SENDME protocol means that each circuit can queue up to 1000 cells at an overloaded bottleneck Tor router, which load balancing can help alleviate, but can't correct in transient and degenerate cases. This is why high capacity Exits have obscene memory requirements when Exits are scarce (ie: below ~1/3 total network throughput), despite there being no memory leaks. This also tells us that Tor relays *cannot* scale to support larger numbers of users without a corresponding linear increase in memory requirements. As a side effect of this protocol, it is possible for the client and the Exit to compute the RTT of a circuit by measuring time between every 50th or 100th sent cell and the associated SENDME arrival. Some previous work makes use of this to try to detect congestion, but it is not a property we want to keep, if we can avoid it. No deployed code uses it, and its presence is worrisome: http://people.cs.ksu.edu/~eyv/papers/latency_leak-ccs07.pdf https://www.robgjansen.com/publications/howlow-pets2013.pdf To further complicate things, these SENDME windows only apply to end-to-end data cells on the circuit. Tor also supports other non-data end-to-end cells, as well as non-end-to-end data cells (so-called "leaky pipe topology", which is used for circuit padding), which are not counted against SENDME windows. Both of these details have also caused us problems in the SENDME design. Still, for now, we will ignore both of these details. Full proposal treatment will need to consider them, though. I. Vaporwaving to ghosts in the anonymity graveyard Ok, so let's party like it's 2010 and go chase some ghosts of past anonymity literature and lore. In my review of this area, I have identified the following causes of death for previous congestion control attempts in Tor: * Side channels and other anonymity risks * Slow or limited responsiveness to queue pressure * Poor fairness properties * Poor throughput * Lack of stream flow control * Endpoint fairness cheating ability * Deployment costs Ghost 1: Drop Signaling Cause of Death: Side channel issues; deployment cost Early proposals for congestion control for Tor attempted to simply tunnel OS TCP streams. This uses the OS's native implementation of TCP Slow Start and AIMD as congestion control methods. These attempts all quickly died, due to OS TCP stack fingerprinting problems (ie: nmap): https://murdoch.is/papers/tor11datagramcomparison.pdf Two years ago, I tried to make the case for using QUIC as a drop-in replacement for Tor circuits, and use QUIC's streams to carry Tor streams. Such a network could leverage QUIC's TCP-like drop-signaled congestion control algorithm internally, and terminate QUIC at the Exit node, transforming the QUIC streams into TCP connections initiated from the Exit. This avoids the TCP fingerprinting issues. In fact, even without a full datagram network, Tor could still deploy relay-only drop signaling if we added windowing and retransmission at the circuit layer: https://lists.torproject.org/pipermail/tor-dev/2018-March/013026.html Unfortunately, the present/absent bit vector of missing packets in this window is a communication channel between malicious Tor relays or Internet routers and the Exit node. This effect was an obscure piece of ancient mix network lore until Nick Mathewson publicly documented it on tor-dev a little over a year ago: https://lists.torproject.org/pipermail/tor-dev/2018-November/013562.html I was the voice of optimism in that grim post, but there is not a lot of hope to be had. It still seems to me that adding cover traffic will reduce the bandwidth of this side channel, but it is a deeply unsolved traffic analysis problem to quantify the level of cover traffic needed, on top of the engineering expense of protocol and cryptographic revamping needed to support cell drops at intermediate relays. It may also be possible to add a MAC scheme that prevents excessive drops by intermediate relays and/or internet routers, but then we may lose a lot of congestion signaling capability and responsiveness. Aside: This drop side channel is much worse for VPN systems that blindly tunnel OS-level TCP. Intermediate Internet routers between the VPN client and the VPN server can use this side channel to send information to any Internet router after the VPN server, via exposed TCP sequence numbers. In a Tor-like system with Exit stream termination, the Exit relay must be one of the side channel participants. For more fun VPN TCP side channels, see also https://seclists.org/oss-sec/2019/q4/122 and https://tools.ietf.org/html/rfc6040. Ghost 2: Datagram Tor with uTP/LEDBAT Signaling Cause of Death: Responsiveness; fairness/cheating; side channels uTP/LEDBAT is the bittorrent transport that measures the RTT of a connection, and backs off if high latency is detected. The theory is that latency above the maximum acceptable value (100ms by spec) indicates that router queues have formed, due to competing traffic causing a bottleneck. Typically these queues form at poor-quality consumer edge routers that queue way too much for their link capacity and user counts. https://tools.ietf.org/html/rfc6817 https://research.torproject.org/techreports/libutp-2013-10-30.pdf Because LEDBAT's goal is to use the link fully, and yield capacity in presence of competing traffic, LEDBAT congestion control requires that target latency upper bounds be known, and also expects some queuing to occur to cause this latency. If network drops still occur, it also uses TCP-like behavior, with similar side channel issues for our usecase. If inherent path latency ever exceeds the LEDBAT target max, throughput will plummet to near-zero. This is bad for Tor, as our path latency is highly variable, especially between 7 hop onion circuits vs 3 hop exit circuits. Additionally, malicious LEDBAT clients can cheat by tolerating a *larger* max delay than other clients. They will thus accept larger queue sizes, which allow larger windows, which allow marginally better throughput, at the expense of more congestion latency for everyone. Ghost 3: Congestion Aware Tor Cause of Death: Hazy anonymity analysis; responsiveness; throughput Congestion Aware Tor uses more detailed per-node latency estimates to measure transient congestion-induced delay and respond to it by altering path selection: https://www.cypherpunks.ca/~iang/pubs/Congestion_Aware_FC12.pdf The downside of Congestion Aware Tor is that rather than managing the congestion window, it suggests simply migrating circuits and changing path weights. This has hazy effects on anonymity, as the set of viable paths in the network is reduced by some hard-to-predict amount. Our Circuit Build Timeout code (CBT) is able to do this kind of path pruning in a significantly more precise manner by setting the circuit timeout such that nearly exactly 80% of all possible network paths are used, but it does not continually perform this timing measurement once the circuit is built. We could build a hybrid system with CBT-style math, with continual measurements, with circuit migration, and with conflux multipath, but this will still require retransmission buffers for reliable circuit migration, will be slower to respond than true window control, and can't actually cap congestion. Worse, circuit migration won't reduce congestion at Exit nodes (you have to keep the same Exit or streams will die). Even with these changes, it also will not remove the throughput cap, or improve onion service throughput. Ghost 4: N23 Tor (aka DefenestraTor) Cause of Death: Stream control; side channels; poor throughput N23 Tor is described in Section 4.2 of the DefenestraTor paper: https://cseweb.ucsd.edu/~savage/papers/PETS11.pdf Basically, each router limits queue length for each circuit to N2 + N3, which are consensus and circuit parameters, respectively. N3 is tuned per circuit by latency measures similar to Congestion Aware Tor, but is still capped at 500 per circuit. This system died primarily because we could not figure out how to bolt stream flow control back on to it. But that is a solvable problem (and any circuit-level congestion control system we deploy must solve it): https://lists.torproject.org/pipermail/tor-dev/2012-November/004138.html https://lists.torproject.org/pipermail/tor-dev/2012-November/004143.html I was not involved in the evaluation of N23 because I was designing and building Tor Browser at the time, but upon diving into it now, I have many additional concerns. First: Because queues are per-circuit but backpressure is per-connection, the backpressure suffers from multiplexing information leak issues. If a circuit manages to fill its N23 limit, the only way for the system to have backpressure to enforce N23 queue sizes is to stop reading entirely on that circuit's upstream TCP connection, stalling *all* other circuits on *only* that particular connection. This property is extremely worrisome from an anonymity standpoint, as it gives a client adversary a mechanism to experimentally stall pairwise router connections to probe for the path taken by a long-lived active target circuit (such as an onion service intro point). Second: It's not clear to me how credits are sent in both directions on a circuit, so that the congestion control can be applied in both directions. Does anyone remember? (Does anyone care?) Third: At the end of the day, the system still has total queue lengths that scale in proportion to the number of circuits on the relay, rather than being globally fixed and controlled through fairness properties. In my view, this is the final nail in the coffin: it's not an improvement for scalability, actually decreases throughput (due to the new lower window cap), and can't control congestion enough to reduce the long-tail latency. Indeed, the perf CDFs from the paper show all of these effects, if you look closely. Ghost 5: RTT Tor [BOOTLEG_RTT_TOR] Cause of Death: Sender cheating; Small max window size Buried in the Defenestrator paper is an unnamed system that uses RTT to estimate congestion and update windows accordingly, much like LEDBAT. It is described in Section 4.1 of https://cseweb.ucsd.edu/~savage/papers/PETS11.pdf I'm going to call this system RTT Tor. RTT Tor works by tracking the min and max RTT observed on a circuit, using the SENDME cell counting side effect. It then picks a threshold timeout T, at a tuneable point in between the min and max observed RTT. If the most recently measured RTT exceeds T, the circuit is declared congested. Unlike LEDBAT, because RTT Tor dynamically chooses T rather than using a hardcoded target RTT, it is possible to use on Tor. The window starts at 100 cells. So long as the RTT stays below T, the window grows by an additional 100 cells every SENDME. If the RTT exceeds T, the window is cut in half. Window size max is capped at 1000 cells. The window cannot go below 100 cells. This system depends upon the Exit node's ability to measure RTT to the client in order to have a downstream window. This capability has been shown to impact client anonymity: http://people.cs.ksu.edu/~eyv/papers/latency_leak-ccs07.pdf https://www.robgjansen.com/publications/howlow-pets2013.pdf Furthermore, endpoints can cheat by choosing a higher T value or otherwise acting as if their window size is increasing when it is not. However, while the paper does not describe a solution for such cheating, it is possible to detect if one endpoint is honest. Because both endpoints can measure the RTT, both endpoints can track what their peer's window should be growing to based on this algorithm, and compare that to an estimate of their peer's window size. The peer's window size can be estimated by counting the number of cells that arrive in RTT/2 amount of time. If the RTT is asymmetric or highly variable, this detection mechanism will have false positives, but we can at least reduce the ability and incentive to cheat so long as one of the endpoints is honest. But if both endpoints cheat, intermediate routers have no way of limiting the data that is in flight or can queue up, except for closing the circuit through the circuit OOM killer. Hence the system still must impose a safe cell window max. It must also impose this max size because some circuits may have an RTT_min that is not much different from the RTT_max, even though the circuit is very congested (because it is continually very congested). In this scenario, the system would keep adding more congestion until the circuit OOM killer kicks in. But, as far as options go, this is not a bad one. Plus, it only requires the client and the Exit to support it. II. Fresh Remixes and New Ideas So what if we took some hits from the 90s and remixed their hooks, Tor style? Let's start with the simplest, most TCP-like scheme we can come up with, and then discuss potential variants and improvements to them, and see which of these ideas mix together well. Remix 1: Forward ECN [FORWARD_ECN_TOR] What if we tried to make something as close to TCP ECN as possible for Tor? Let's use an initial window size of 100 cells, and make our SENDMEs into 100 cell ACKs. Let's create a RELAY_COMMAND_QUEUE_PRESSURE relay cell command that can be sent by a relay towards a client whenever the sum total of queued cells exceeds some limit. Which circuit gets the cell can be chosen by EWMA or some other QoS algorithm (ie: RED-like weighted random choice, as per TCP ECN). The cell would indicate the direction that the circuit was loudest on (ie: client-bound or exit-bound). When the client gets the cell, if the direction is exit-bound, the client halves its outbound window. If the direction is client-bound, the client sends the cell back to the Exit node, who then halves its client-bound SENDME send window. (Recall that Tor's "relay commands" are encrypted to/from the client, with Tor's per-hop circuit keys. This means that extra relay cells can be injected by any hop in the circuit towards the client, and intermediate relays cannot read these cells' relay commands. As we will soon see in [BACKWARD_ECN_TOR] and related ideas, using cleartext "cell commands" instead can help make things more efficient by allowing signaling of congestion without requiring extra cell injection, but this also introduces more side channel risk). We can use Slow Start with AIMD here: Before the first RELAY_COMMAND_QUEUE_PRESSURE, outbound windows would double every window. After the first RELAY_COMMAND_QUEUE_PRESSURE, they would increase by say 100, every window. Or something similar. To reduce the potential for side channel injection, the client can ensure that RELAY_COMMAND_QUEUE_PRESSURE do not arrive more often than once per window. To prevent patterns from trivially being encoded on these cells, the client can delay relaying them until they are on a K % M == 0 boundary, for M=5 or 10 or so (high M will reduce responsiveness to congestion). The client can also refuse to relay these cells if they would cause the Exit's window to drop below some minimum size (say 128). Intuition tells me the relays should use total outbound queue size for deciding *when* to send these cells, and use specific circuit queue length and/or activity to decide *which* circuit to send them on. In this way, we globally limit queue size (per hop latency) regardless of the number of circuits, and still enforce fairness among these circuits. However, picking parameters and algorithms for all of this is a research problem, or at least requires much more QoS RFC and research literature review. There may be additional optimizations too, like sending these cells on circuits of any TCP connection that starts blocking or exceeding per-connection queue limits. While this will work if clients are well-behaved, this system has a few drawbacks. First, it will be slow to respond to client-induced download congestion at middle nodes, since we have to wait a full circuit RTT for the client to relay cells to the Exit. Since the Exit is responsible for the download window for web-like clients, this is also the direction that needs the most congestion control, which is unfortunate. Second, clients can easily cheat, even by malfunction: all they have to do is refuse or forget to relay cells to an Exit, and they get faster downloads at the expense of more network congestion for everyone. We could do what RTT Tor did and cap the window size at 1000, but then we're back in the situation where throughput is artificially capped at some bizarre function of circuit RTT, and worst-case latency will not improve as much as we want. We could rely more heavily on the circuit OOM killer, and kill circuits that queue too much on the assumption that they are cheating/malfunctioning, but tuning this to avoid false positives may be tricky. Third, while this system could technically be deployed even if some nodes do not support it yet, this is risky, as nodes who do not support it will experience excessive congestion at worst, and no improvement at best. Fourth, onion services are tricky to handle. Because of the way rendezvous circuits are joined, this system will send RELAY_COMMAND_QUEUE_PRESSURE towards the client for the client-side of the rend circuit, and towards the service for the service end of the circuit. Then, either of these sides would echo that signal as a RELAY_COMMAND_QUEUE_PRESSURE *all* the way to the other side. Either or both could cheat in this case, and again, the only recourse we have is for each relay to enforce strict circuit queue length limits via the circuit OOM killer. Remix 2: Forward ECN mixed with RTT Tor [FORWARD_ECN_RTT_TOR] [PROPOSAL_CANDIDATE] Ok, [FORWARD_ECN_TOR] sounds decent, but because the client mediates everything, it will be slow to respond to client-destined download congestion, and clients can cheat. Also, if some relays don't support the signal yet, the window may become too large for their congested status. What if we *also* mix in [BOOTLEG_RTT_TOR], so that an endpoint's send window can only grow if *both* conditions hold: the measured circuit RTT stays below RTT Tor's T threshold, *and* there are no congestion signals coming from [FORWARD_ECN_TOR]? The addition of the signaling cell from [FORWARD_ECN_TOR] also allows us to be more robust in the cheating detection described in [BOOTLEG_RTT_TOR], and eliminate false positives. If either endpoint detects a window that is growing too large for their measured circuit RTT (by counting the number of cells arriving in that RTT, and comparing that to what the window should be based on the RTT window growth algorithm), it can send a warning shot RELAY_COMMAND_QUEUE_PRESSURE, explicitly telling the cheater to back off. If the window still grows because the other endpoint ignores this warning shot, it can close the circuit. This system is fully incrementally deployable: it can be used if only the client and Exit support it. It is resilient to cheating so long as Exits are honest, without false positives, and even without intermediate relay support. While we still aren't doing better than 1 RTT response to congestion, we now have a good answer for cheating that doesn't have any false positive risk for good actors. Furthermore, because we also have explicit congestion signaling, we no longer have to worry as much about imposing a max window size, to protect circuits for which RTT_min is close to RTT_max due to persistent congestion. Good actors on these circuits will back off, and for actors acting badly enough to add additional congestion, RTT_max will increase enough for them to eventually be detected. Downside: Since onion services have no Exit node, if we want to use the RTT mechanism to mitigate cheating for them, we must terminate congestion control at the Rendezvous point for this proposal idea, so that onion service clients and services can't collude to get faster service. The RP would then be responsible for examining each half of the spliced circuit's congestion windows, and sending ECN signals down whichever side had a larger window. This may or may not be worth the additional complexity, as opposed to just employing the circuit OOM killer if circuit queues get too long (due to cheating). The only remaining downside I see is that we are relying on baking the ability to make RTT measurements into the protocol. Still, I think this is worth turning into its own proposal. Does anyone see any other issues, or have any suggestions? Remix 3: Backward ECN [BACKWARD_ECN_TOR] Ok so the [FORWARD_ECN_RTT_TOR] mix is pretty tight. We've got an incrementally deployable system that is somewhat resistant to cheaters, and responds to congestion within an RTT. Can we do better? Yes. Remember that Backward ECN IETF draft that we covered in [TCP_HISTORY] earlier? It had that RTT/2 response with a hyphy hook. Let's sample that. First: just like the other proposals, we still need to use SENDMEs, to ack received packets. However, since we are not measuring RTT here, it does not have to be exactly every 100 cells. The SENDME can be sent at a randomized cell count to obscure RTT, with a different randomized ack cell count value included in the SENDME. In addition to RELAY_COMMAND_QUEUE_PRESSURE, which is sent to the client and can't be read by any other intermediate relays, let's make a new *cell* command type, CELL_COMMAND_RELAY_QUEUE_PRESSURE. Because this is a cell command, it can be read by intermediate relays, and can be set by an intermediate relay on any cell heading towards the Exit, simply by flipping cell_t.command from CELL_COMMAND_RELAY to CELL_COMMAND_RELAY_QUEUE_PRESSURE, so long as all relays in the circuit understand this new command. This also avoids sending any additional empty cells when congestion occurs in the common Web download direction, which is nice. We should still respond to congestion with Slow Start AIMD: when the client or exit gets this relay cell or cell command, it cuts its send window in half in that direction. Otherwise windows grow during transmission similar to TCP (ie double the window size each window until the first congestion cell, then linear). The downside of these cells being a new end-to-end cell_t.command is that it opens up side channel attacks we saw with the RELAY_EARLY attack: https://blog.torproject.org/tor-security-advisory-relay-early-traffic-confi… The attack is to flip this cell_t.command field on a sequence of cells to encode a bitstring, to enable the Exit to send data to the Guard. So in this mix, let's still use RELAY_COMMAND_QUEUE_PRESSURE towards the client. We will only use the relay-visible cell_t.command CELL_COMMAND_RELAY_QUEUE_PRESSURE towards the Exit. This means we only need to worry about Guard to Exit side channels, which require much more information to be useful (ie: they must encode client IP address or some other unique client identifier). Note that because all intermediate relays can see the cell_t.command, they can enforce similar properties as clients did in [FORWARD_ECN_TOR], to limit side channels in the Guard to Exit direction. For instance, they can ensure that these commands do not get sent more often than once per window of client-bound incoming cells. They can also enforce that the Exit-bound cell_t.command = CELL_COMMAND_RELAY_QUEUE_PRESSURE in the outgoing direction is only set on K % M == 0 cell count boundaries, for low M=5 or 10. Note that this K % M spacing actually works better for longer circuits, since there is more chance for other relays to flip the congestion bit at each M spacing position, which damages the reliability of the side channel signal. When middle relays enforce that the window is not allowed to drop below win_min=128, a malicious Guard can only inject log2(win_size)-log2(win_min) of these cells in a burst. Once the middle node detects that the window size would be below the legal minimum, it knows either cheating is happening, or a side channel is being used. For a typical win_size of ~8k (ie: 16X faster throughput than current Tor) and a win_min=128, this detection will be triggered in ~6 properly spaced fake CELL_COMMAND_RELAY_QUEUE_PRESSURE commands. However, in the AIMD steady state for this side channel, a malicious guard can attempt to keep the window size as close to win_min as possible, without going below it. Since the window size grows linearly after the first ECN signal, the guard gets to send an additional CELL_COMMAND_RELAY_PRESSURE cell_t.command around once every win_min client-bound incoming cells. So long as no other relay is also sending congestion control signals, the guard can keep flipping bits roughly this often (though the K % M == 0 requirement would restrict the placement of these cells). Downsides: Even just 6 bits of reliable information can be damaging if target client anonymity sets are already small, and so long as client-bound incoming cells keep arriving, there are still more opportunities to flip the cell_t.commands from the Guard to the Exit. The big question here is can these 6+ cell_t.command flips be turned into a large side channel, despite K % M == 0 placement enforcement? And can this side channel be made reliable? If so (and it seems likely that this is so), then this whole idea needs revision (see [EDGE_FORWARD_ECN_TOR] and [START_BACKWARD_ECN_TOR] for possible revisions). We also need all intermediate relays to upgrade to properly forward the CELL_COMMAND_RELAY_QUEUE_PRESSURE cell command just like CELL_COMMAND_RELAY. Also, clients can still cheat by ignoring the encrypted RELAY_COMMAND_QUEUE_PRESSURE. On the plus side, clients who cheat can at best get faster upload speeds. They cannot get faster download speeds unless the Exits also decide to cheat. Finally, the story for cheating onion service endpoints is similarly complicated: Do we have the RP mediate, or fall back to the circuit OOM killer yet again? Remix 4: Backward ECN mixed with RTT Tor [BACKWARD_ECN_RTT_TOR] If we really are worried about upload cheating, we can mix RTT Tor back in, just like we did for [FORWARD_ECN_RTT_TOR], and use its cheating detection. An endpoint that detects abnormally large window sizes (based on arrival cell counts per RTT) could send the RELAY_COMMAND_QUEUE_PRESSURE warning shot to reduce the window. So now we have a system that responds in RTT/2 when supported by all intermediate relays, and falls back to RTT response in the presence of one cheater endpoint or lack of support by intermediate relays. Unfortunately, this still has the side channel issues of [BACKWARD_ECN_TOR], and so I have not tagged it as a proposal candidate. Remix 5: Backward ECN with Transparent Trap [BACKWARD_ECN_TRAP_TOR] Ok so the "SENDME ack exactly 100 cells" 2-step beat is getting a little tired. Can we devise any ways to trap cheaters without having a predicable RTT measurement baked into the protocol, and still respond to congestion in RTT/2? There is a way to defeat the wizard. But there be dragons in this trap house, and they really want to soul bond using side channels while everybody is watching. Hence I have not flagged this as a proposal candidate. Plus, as these mixed metaphors and obscure references suggest, this remix is very busy and complicated. Still, let's consider it because it may be useful to remix later. Let's take [BACKWARD_ECN_TOR], and instead of using RELAY_COMMAND_QUEUE_PRESSURE towards the client, let's use cell_t.command=CELL_COMMAND_RELAY_QUEUE_PRESSURE in both directions. This allows us to perform the same kinds of side channel and cheating checks at the middle node in the Exit to Guard direction, and also avoids sending any extra relay cells during congestion. Just as in [BACKWARD_ECN_TOR], let's randomize the cell count at which we send each SENDME, and include a different randomized ack count in the SENDME, so that each endpoint can add cells back to the window for that SENDME, but cannot compute RTT. Let's also still use Slow Start with AIMD (double the window if there are no congestion signals, half the window on congestion signals, and grow it linear after that). However, this is still not sufficient to stop Exit to Guard side channels! The Exit to Guard direction is different than the Guard to Exit direction: relays can inject arbitrary amounts of full relay cells for the client in the Guard direction, whereas they could not do so in the Guard to Exit direction. In Tor (and other mixnets), this property is called "leaky pipe topology", and is what allows us to send RELAY_COMMAND_DROP padding cells from the middle relay to the client (and vice-versa). This means that the combination of these injected cells plus congestion control can be used to encode a pattern from Exit to Guard, and subvert the checks at the middle node, because relays can inject dummy cells sent towards the client to meet the win_size and M spacing requirements. The good news is that the client-side circuit padding system only allows RELAY_COMMAND_DROP from a hop for which the client has negotiated a padding machine. This means that Exit nodes cannot inject padding for assistance in any side channel usage, regardless of the congestion control scheme we choose. The bad news is that vanilla Tor allows the presence of invalid protocol-violating cells, and does not close the circuit for these cases. Best case, it issues a log message. Worst case: it silently drops them. Thankfully, the vanguards addon is designed such that any cell that is not specifically marked as valid by Tor counts as a dropped cell, after which the circuit will be closed. This is a fail-closed defense. If new code is added for cell processing that forgets to flag cells as properly processed by calling circuit_read_valid_data(), those cells count as false positive dropped cells, rather than false negatives. This impacts reliability, but has the advantage that we won't shoot ourselves in the foot by adding new code that is too permissive without noticing that this code simply does not work properly because circuits get closed when it is exercised. All this means we will need to port this defense into Tor itself, though, or dummy cells can be injected by the Exit for side channel assistance in congestion control schemes or other cases: https://github.com/mikeperry-tor/vanguards/blob/master/README_TECHNICAL.md#… But what about legit padding causing the Guard to think that win_size and M spacing requirement are violated? Well, so long as any relay with an active padding machine stores the congestion signals so that it can delay them such that they are sent on the next K % M == 0 boundary after padding is applied, these limits can still be enforced, without revealing the quantity of padding. Additionally, any guard node congestion caused by the sum of real+padding traffic can be controlled under this scheme, because the middle relay can observe the guard's congestion signals and decrease padding approperiately. Downsides: If preventing *any* side channel bits from flowing from the Exit to the Guard is important, this scheme likely does not accomplish that. It does seem that Exit to Guard side channels are far more dangerous than Guard to Exit, since all the Exit has to say is "hey this client looked at something I think is interesting", where as the Guard has to communicate a unique identifier for the anonymity set, so that concern alone may kill this idea. This scheme is also complicated, and any implementation errors will bring back the RELAY_EARLY side channel attack in the Exit to Guard direction, even if the math might claim otherwise. Remix 6: Edge-Forward ECN [EDGE_FORWARD_ECN_TOR] [PROPOSAL_CANDIDATE] The common problem with both [BACKWARD_ECN_TOR] and [BACKWARD_ECN_TRAP_TOR] is that they enable side channels by one or both of the edges of the circuit (Guard to Exit, or Exit to Guard). Forward ECN mitigates these side channels, because the edge no longer gets to encode its congestion signal at specific points in a traffic pattern. This works better than win_size and M position limiting by itself, and it can be used in combination with client-side window size checks and M position limiting, too. If middle nodes could somehow reliably detect the Guard and Exit endpoints of a circuit, then the middle node(s) could force one or both endpoints to use Forward ECN [FORWARD_ECN_TOR]. Then the circuit edges would not be able to communicate with each other by directly encoding congestion signals in deterministic traffic patterns. This is easier said than done. In practice, middles can tell that they are middles because they do not get any authentication cells from the client, and they are not asked to open any streams. But what we really need is for middles to be able to tell if their *neighbors* are the circuit edges, because these edges are the only ones that we actually need to forbid from using cell_t.command from BACKWARD_ECN_TOR; it is fine if multiple middles use cell_t.command (for example, in onion service circuits). The only way I can think of to reliably detect circuit edges from the middle position is to forbid either Exits or Guards nodes from being used as middles, so that middles could know when they were next to one of these node types, and therefore next to the edge of the circuit. Then middles could forbid these edges' use of the cleartext CELL_COMMAND_RELAY_QUEUE_PRESSURE. This only-middles-as-middles stratification requirement has been proposed before for several other reasons, so maybe it is not a horrible idea? This is more tricky for onion service circuits, though. For those, we could require that sensitive positions (such as HSDIR, RP, and IP) also be Exit-flagged nodes? Or maybe there is another clever way to detect just the edges of a circuit from the middle position? Even if there is, this proposal still has some cheating risk, as clients can refuse to relay these edge congestion signals to get marginally better throughput. Again, RTT measurements can be preserved here as a backstop against cheating, but for the additional anonymity risk. Otherwise we have to fall back to the circuit OOM killer. Remix 7: Start Backward ECN [START_BACKWARD_ECN_TOR] [PROPOSAL_CANDIDATE] The Slow Start with AIMD window management for all of these systems means that we most need the quick RTT/2 response to the congestion early on in the Slow Start phase, while the window size is growing exponentially. This is also when the network is most vulnerable to cheaters. So let's allow only 1 (or maybe 2) CELL_COMMAND_RELAY_QUEUE_PRESSURE cell_t.commands per circuit in the Guard to Exit direction as per [BACKWARD_ECN_TOR], and maybe even 1 CELL_COMMAND_RELAY_QUEUE_PRESSURE cell_t.command in the Exit to Guard direction from [BACKWARD_ECN_TRAP_TOR]. After these cell_t.command limits are hit, middle nodes forbid any further use of this cell type on that circuit, and enforce that all circuit member relays switch to [FORWARD_ECN_TOR] or [FORWARD_ECN_RTT_TOR] from then on (depending on how concerned we are about cheating vs disclosing circuit RTT). This is my all-around favorite of the options. The only downside is that we have to choose between disclosing RTT, or risking some cheating, but since the cheating can now only happen after slow start has finished, it will provide less speed advantages. If clients cheat with the goal of causing memory exhaustion at relays, the circuit OOM killer can kick in. Remix 8: Your idea here! Anyone have any other old lore to resurrect? Any new ideas? Bonus B-Side Track: Stream Flow Control [PROPOSAL_CANDIDATE] All of the systems discussed in this post address the problem of circuit-level congestion control, but do not perform any stream-level flow control. While we were prototyping N23, Andreas Krey pointed out that a single blocked stream could consume the entire circuit window with unread packets, causing the surprising result that other active streams on the same circuit will also stall: https://lists.torproject.org/pipermail/tor-dev/2012-November/004138.html He then went on to propose some alternatives: XON/XOFF, bundling stream window updates in circuit SENDMEs, or just killing any blocked streams that have too many unread cells: https://lists.torproject.org/pipermail/tor-dev/2012-November/004143.html XON/XOFF is the simplest option, but we will need heuristics to avoid excessive XON/XOFF chatter on application streams that frequently block in normal operation. It also will have RTT/2 delay before it actually causes the other end to stop sending. If the blocked stream is sending cells at the full bandwidth of the circuit, this could very well be a full circuit window worth of cells before the XOFF makes it across. We can just ack those cells with a circuit-level SENDME even though they were not delivered to their blocked stream, but we have to be careful to authenticate what we read in this case, so we do not re-introduce the unauthenticated SENDME attack: https://gitweb.torproject.org/torspec.git/tree/proposals/289-authenticated-… Can we learn anything from the layered flow control in QUIC? It is excessively complicated: https://docs.google.com/document/d/1F2YfdDXKpy20WVKJueEf4abn_LVZHhMUMS5gX6P… III. Evaluation First, does anyone have any strong objections to any of the remixes tagged with PROPOSAL_CANDIDATE? It will save time if we can decide right from the start that any are completely unacceptable. Bonus points if we can mod them back into being acceptable, or mod one of the remixes that are not tagged with PROPOSAL_CANDIDATE into being acceptable. We also need some criteria to help us determine how we're going to compare the remaining ones. Here is a condensed [TRACK_LISTING], for quick review/comparison and criteria brainstorming: ______________________________________________________________________ | MIX | CHEATING | RESPONSE| SIDE | UPGRADE | | TRACK | DETECTION | TIME | CHANNELS | PATH? | |~~~~~~~~~~~~~~~~~~~~~~|~~~~~~~~~~~|~~~~~~~~~|~~~~~~~~~~~~|~~~~~~~~~~| |BOOTLEG_RTT_TOR | Exit RTT |100c+RTT | RTT | Exits | |FORWARD_ECN_TOR | Circ OOM | RTT | None? | Full Net | |FORWARD_ECN_RTT_TOR | Exit RTT | RTT | RTT |Exits;Full| |BACKWARD_ECN_TOR |Middles;OOM| RTT/2 | Guard->Exit| Full Net | |BACKWARD_ECN_RTT_TOR |Middles;RTT|RTT/2;RTT| Guard->Exit|Exits;Full| |BACKWARD_ECN_TRAP_TOR | Middles | RTT/2 |Guard<->Exit| Full Net | |EDGE_FORWARD_ECN_TOR |Middles;OOM| 2*RTT/3 | None? | Full Net | |START_BACKWARD_ECN_TOR|Middles;OOM|RTT/2;RTT| Low/None? | Full Net | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ What are the things we need to decide if a scheme is acceptable? Here's a laundry list. Please do add any others you think of: - Side channel accounting - Interaction with non-data traffic, padding traffic, and leaky-pipes - Anonymity analysis as per https://www.robgjansen.com/publications/howlow-pets2013.pdf - No/minimal cheating - No throughput cap - Quick responsiveness (at least RTT; RTT/2 is better) - Fairness analysis (may be a separate proposal on QoS/EWMA) - Onion service deployment - Simulation methodology that will reflects live deployment - Deployment costs - Upgrade path Must-Read References: Cultural etymology of Tor's end-of-decade donations campaign aesthetic: https://en.wikipedia.org/wiki/Vaporwave https://blog.torproject.org/better-internet-possible-ive-seen-it Tor congestion latency is proportional to network utilization divided by spare_capacity: Section 6.3 of https://www.freehaven.net/anonbib/cache/murdoch-pet2008.pdf RTT-based Congestion Control for Tor: Section 4.1 of https://cseweb.ucsd.edu/~savage/papers/PETS11.pdf TCP ECN: https://tools.ietf.org/html/rfc3168#section-6.1 Backward ECN: https://tools.ietf.org/html/draft-salim-jhsbnns-ecn-00 Stream Flow Control Issues and Ideas: https://lists.torproject.org/pipermail/tor-dev/2012-November/004138.html https://lists.torproject.org/pipermail/tor-dev/2012-November/004143.html Side channels in circ command cells: https://blog.torproject.org/tor-security-advisory-relay-early-traffic-confi… -- Mike Perry

1 0

Prop 311: Relay IPv6 Reachability
by teor 29 Jan '20

29 Jan '20

Hi, Here is an initial draft of Proposal 311: Relay IPv6 Reachability. This proposal includes: * relay IPv6 ORPort extends, and * relay IPv6 ORPort reachability checks. This is the first of 3 proposals: * Proposal 311: Relay IPv6 Reachability * Proposal 312: Automatic Relay IPv6 Addresses * Proposal 313: Relay IPv6 Statistics (I haven't written the others yet!) I also want to make some minor changes to Proposal 306, so that bridge IPv6 behaviour stays in sync with client IPv6 behaviour. There are still a few TODO items in the proposal, mostly about Tor's current behaviour. If you know the answers, please let me know. The full text is included below, and it is also available as a GitHub pull request: https://github.com/torproject/torspec/pull/103 The related tickets are #24404 (proposal) and #24403 (implementation): https://trac.torproject.org/projects/tor/ticket/24404 https://trac.torproject.org/projects/tor/ticket/24403 Please feel free to reply on this list, or via GitHub pull request comments. Filename: 311-relay-ipv6-reachability.txt Title: Tor Relay IPv6 Reachability Author: teor Created: 22-January-2020 Status: Draft Ticket: #24404 0. Abstract We propose that Tor relays and bridges should check the reachability of their IPv6 ORPort, before publishing their descriptor. To check IPv6 ORPort reachability, relays and bridges need to be able to extend circuits via other relays, and back to their own IPv6 ORPort. 1. Introduction Tor relays and bridges currently check the reachability of their IPv4 ORPort and DirPort before publishing them in their descriptor. But relays and bridges do not test the reachability of their IPv6 ORPorts. However, Directory Authorities make direct connections to relay IPv4 and IPv6 ORPorts, to test each relay's reachability. Once a relay has been confirmed as reachable by a majority of authorities, it is included in the consensus. (Currently, 6 out of 9 Directory Authorities perform IPv4 and IPv6 reachability checks. The others just check IPv4.) The Bridge authority makes direct connections to bridge IPv4 ORPorts, to test each bridge's reachability. Depending on its configuration, it may also test IPv6 ORPorts. Once a bridge has been confirmed as reachable by the bridge authority, it is included in the bridge networkstatus used by BridgeDB. Many relay and bridge operators don't know when their relay's IPv6 ORPort is unreachable. They may not find out until they check [Relay Search], or their traffic may drop. For new operators, it might just look like Tor simply isn't working, or it isn't using much traffic. IPv6 ORPort issues are a significant source of relay and bridge operator support requests. Implementing IPv6 ORPort reachability checks will provide immediate, direct feedback to operators in the relay or bridge's logs. It also enables future work, such as automatically discovering relay and bridge addresses for IPv6 ORPorts (see [Proposal 312]). 2. Scope This proposal modifies Tor's behaviour as follows: Relays: * circuit extension, * OR connections for circuit extension, * reachability testing. Bridges: * reachability testing only. This proposal does not change client behaviour. When this proposal describes Tor's current behaviour, it covers all supported Tor versions (0.3.5.7 to 0.4.2.5), as of January 2020. 3. Allow Relay IPv6 Extends To check IPv6 ORPort reachability, relays and bridges need to be able to extend circuits via other relays, and back to their own IPv6 ORPort. We propose that relays start to extend some circuits over IPv6 connections. We do not propose any changes to bridge extend behaviour. 3.1. Current IPv6 ORPort Implementation Currently, all relays and bridges must have an IPv4 ORPort. IPv6 ORPorts are optional for relays and bridges. Tor supports making direct IPv6 OR connections: * from directory authorities to relay ORPorts, * from the bridge authority to bridge ORPorts, * from clients to relay and bridge ORPorts. Tor relays and bridges accept IPv6 ORPort connections. But IPv6 ORPorts are not currently included in extend requests to other relays. And even if an extend cell contains an IPv6 ORPort, bridges and relays will not extend via an IPv6 connection to another relay. Instead, relays will extend circuits: * Using an existing authenticated connection to the requested relay (which is typically over IPv4), or * Over a new connection via the IPv4 ORPort in an extend cell. If a relay receives an extend cell that only contains an IPv6 ORPort, the extend typically fails. 3.2. Relays Extend to IPv6 ORPorts We propose that relays make some connections via the IPv6 ORPorts in extend cells. Relays will extend circuits: * using an existing authenticated connection to the requested relay (which may be over IPv4 or IPv6), or * over a new connection via the IPv4 or IPv6 ORPort in an extend cell. Since bridges try to imitate client behaviour, they will not adopt this new behaviour, until clients begin routinely connecting via IPv6. (See [Proposal 306].) 3.2.1. Making IPv6 ORPort Extend Connections Relays can make a new connection over IPv6 when: * there is no existing authenticated connection to the requested relay, and * the extend cell contains an IPv6 ORPort. If these conditions are satisfied, and the extend cell also contains an IPv4 ORPort, we propose that the relay choose between an IPv4 and an IPv6 connection at random. If the extend cell does not contain an IPv4 ORPort, we propose that the relay connects over IPv6. (Relays should support IPv6-only extend cells, even though they are not used to test relay reachability in this proposal.) A successful IPv6 connection also requires that: * the requested relay has an IPv6 ORPort. But extending relays must not check the consensus for other relays' IPv6 information. Consensuses may be out of date, particularly when relays are doing reachability checks for new IPv6 ORPorts. See section 3.3.2 for other situations where IPv6 information may be incorrect or unavailable. 3.2.2. No Tor Client Changes Tor clients currently include IPv4 ORPorts in their extend cells, but they do not include IPv6 ORPorts. We do not propose any client IPv6 extend cell changes at this time. The Tor network needs more IPv6 relays, before clients can safely use IPv6 extends. (Relays do not require anonymity, so they can safely use IPv6 extends to test their own reachability.) We also recommend prioritising client to relay IPv6 connections (see [Proposal 306]) over relay to relay IPv6 connections. Because client IPv6 connections have a direct impact on users. 3.3. Alternative Extend Designs We briefly mention some potential extend designs, and the reasons that they were not used in this proposal. (Some designs may be proposed for future Tor versions, but are not necessary at this time.) 3.3.1. Future Relay IPv6 Extend Behaviour Random selection of extend ORPorts is a simple design, which supports IPv6 ORPort reachability checks. However, it is not the most efficient design when: * both relays meet the requirements for IPv4 and IPv6 extends, * a new connection is required, * the relays have either IPv4 or IPv6 connectivity, but not both. In this very specific case, this proposal results in an average of 1 circuit extend failure per new connection. (Because relays do not try to connect to the other ORPort when the first one fails.) If relays try both the IPv4 and IPv6 ORPorts, then the circuit would succeed. For example, relays could try the alternative port after a 250ms delay, as in [Proposal 306]. This design results in an average circuit delay of up to 125ms per new connection, rather than failure. However, partial relay connectivity should be uncommon. And relays keep connections open long-term, so new relay connections are a small proportion of extend requests. Therefore, we defer implementing any more complex designs. Since we propose to use IPv6 extends to test relay reachability, occasional circuit extend failures have a very minor impact. 3.3.2. Future Bridge IPv6 Extend Behaviour When clients automatically connect to relay IPv4 and IPv6 ORPorts by default, bridges should also adopt this behaviour. (For example, see [Proposal 306].) 3.3.3. Rejected Extend Designs Some designs may never be suitable for the Tor network. We rejected designs where relays check the consensus to see if other relays support IPv6, because: * relays may have different consensuses, * the extend cell may have been created using a version of the [Onion Service Protocol] which supports IPv6, or * the extend cell may be from a relay that has just added IPv6, and is testing the reachability of its own ORPort (see Section 4). We avoided designs where relays try to learn if other relays support IPv6, because these designs: * are more complex than random selection, * potentially leak information between different client circuits, * may enable denial of service attacks, where a flood of incorrect extend cells causes a relay to believe that another relay is unreachable on an ORPort that actually works, and * require careful tuning to match the typical interval at which network connectivity is actually changing. 4. Check Relay and Bridge IPv6 ORPort Reachability We propose that relays and bridges check their own IPv6 ORPort reachability. To check IPv6 ORPort reachability, relays and bridges extend circuits via other relays, and back to their own IPv6 ORPort. IPv6 reachability failures may result in a relay or bridge refusing to publish its descriptor, if enough existing relays support IPv6 extends. 4.1. Current Reachability Implementation Relays and bridges check the reachability of their IPv4 ORPorts and DirPorts, and refuse to publish their descriptor if either reachability check fails. IPv4 ORPort reachability checks succeed when any create cell is received on any inbound OR connection. The check succeeds, even if the cell is from an IPv6 ORPort, or a circuit built by a client. Directory Authorities make direct connections to relay IPv4 and IPv6 ORPorts, to test each relay's reachability. Relays that fail either reachability test, on enough directory authorities, are excluded from the consensus. The Bridge authority makes direct connections to bridge IPv4 ORPorts, to test each bridge's reachability. Depending on its configuration, it may also test IPv6 ORPorts. Bridges that fail either reachability test are excluded from BridgeDB. 4.2. Checking IPv6 ORPort Reachability We propose that testing relays (and bridges) select some IPv6 extend-capable relays for their reachability circuits, and include their own IPv4 and IPv6 ORPorts in the final extend cells on those circuits. The final extending relay will extend to the testing relay: * using an existing authenticated connection to the testing relay (which may be over IPv4 or IPv6), or * over a new connection via the IPv4 or IPv6 ORPort in the extend cell. The testing relay will confirm that test circuits can extend to both its IPv4 and IPv6 ORPorts. 4.2.1. Selecting the Final Extending Relay IPv6 ORPort reachability checks require an IPv6 extend-capable relay as the second-last hop of reachability circuits. (The testing relay is the last hop.) IPv6-extend capable relays must have: * Relay subprotocol version 3 (or later), and * an IPv6 ORPort. (See section 5.1 for the definition of Relay subprotocol version 3.) The other relays in the path do not require any particular protocol versions. 4.2.2. Extending from the Second-Last Hop IPv6 ORPort reachability circuits should put the IPv4 and IPv6 ORPorts in the extend cell for the final extend in reachability circuits. Supplying both ORPorts makes these extend cells indistinguishable from future client extend cells. If reachability succeeds, the testing relay (or bridge) will accept the final extend on one of its ORPorts, selected at random by the extending relay (see section 3.2.1). 4.2.3. Separate IPv4 and IPv6 Reachability Flags Testing relays (and bridges) will record reachability separately for IPv4 and IPv6 ORPorts, based on the ORPort that the test circuit was received on. 4.2.4. No Changes to DirPort Reachability We do not propose any changes to relay IPv4 DirPort reachability checks at this time. The following configurations are currently not supported: * bridge DirPorts, and * relay IPv6 DirPorts. Therefore, they are also out of scope for this proposal. 4.3. Refuse to Publish Descriptor if IPv6 ORPort is Unreachable If an IPv6 ORPort reachability check fails, relays (and bridges) should log a warning. In addition, if IPv6ORPort reachability checks are reliable, based on the number of relays in the network that support IPv6 extends, relays should refuse to publish their descriptor. 4.3.1. Refusing to Publish the Descriptor We set a threshold of consensus relays for reliable IPv6 ORPort checks: * at least 30 relays, and * at least 1% of the total consensus weight, must support IPv6 extends. We chose these parameters so that the number of relays is triple the number of directory authorities, and the consensus weight is high enough to support occasional reachability circuits. In small networks with: * less than 2000 relays, or * a total consensus weight of zero, the threshold should be the minimum tor network size to test reachability: * at least 2 relays, excluding this relay. (Note: we may increase this threshold to 3 or 4 relays if we discover a higher minimum during testing.) If the current consensus satisfies this threshold, testing relays (and bridges) that fail IPv6 ORPort reachability checks should refuse to publish their descriptors. To ensure an accurate threshold, testing relays should exclude: * the testing relay itself, and * relays that they will not use in testing circuits, from the: * relay count, and * the numerator of the threshold percentage. Typically, relays will be excluded if they are in the testing relay's: * family, * IPv4 address /16 network, * IPv6 address /32 network (a requirement as of Tor 0.4.0.1-alpha), unless EnforceDistinctSubnets is 0. As a useful side-effect, these different thresholds for each relay family will reduce the likelihood of the network flapping around the threshold. If flapping has an impact on the network health, directory authorities should set the AssumeIPv6Reachable consensus parameter. (See the next section.) 4.3.2. Add AssumeIPv6Reachable Option We add an AssumeIPv6Reachable torrc option and consensus parameter. If IPv6 ORPort checks have bugs that impact the health of the network, they can be disabled by setting AssumeIPv6Reachable=1 in the consensus parameters. If IPv6 ORPort checks have bugs that impact a particular relay (or bridge), they can be disabled by setting "AssumeIPv6Reachable 1" in the relay's torrc. This option disables IPv6 ORPort reachability checks, so relays publish their descriptors if their IPv4 ORPort reachability checks succeed. The default for the torrc option is "auto", which checks the consensus parameter. If the consensus parameter is not set, the default is "0". "AssumeReachable 1" overrides all values of "AssumeIPv6Reachable", disabling both IPv4 and IPv6 ORPort reachability checks. 4.4. Optional Efficiency and Reliability Changes We propose some optional changes for efficiency and reliability, and describe their impact. Some of these changes may be more appropriate in future releases, or along with other proposed features. 4.4.1. Extend IPv6 From All Supported Second-Last Hops The testing relay (or bridge) puts both IPv4 and IPv6 ORPorts in its final extend cell, and the receiving ORPort is selected at random by the extending relay (see sections 3.2.1 and 4.2). Therefore, approximately half of IPv6 ORPort reachability circuits will actually end up confirming IPv4 ORPort reachability. We propose this optional change, to improve the rate of IPv6 ORPort reachability checks: If the second-last hop of an IPv4 ORPort reachability circuit supports IPv6 extends, testing relays may put the IPv4 and IPv6 ORPorts in the extend cell for the final extend. As the number of relays that support IPv6 extends increases, this change will increase the number of IPv6 reachability confirmations. In the ideal case, where the entire network supports IPv4 and IPv6 extends, IPv4 and IPv6 ORPort reachability checks would require a similar number of circuits. 4.4.2. Close Existing Connections Before Testing Reachability When a busy relay is performing reachability checks, it may already have established inbound or outbound connections to the second-last hop in its reachability test circuits. The extending relay may use these connections for the extend, rather than opening a connection to the target ORPort (see sections 3.2 and 4.2.2). Bridges only establish outbound connections to other relays, and only over IPv4 (except for reachability test circuits). So they are still potentially affected by this issue. We propose these optional changes, to improve the efficiency of IPv4 and IPv6 ORPort reachability checks: Testing relays (and bridges): * close any outbound connections to the second-last hop of reachability circuits, and * close inbound connections to the second-last hop of reachability circuits, if those connections are not using the target ORPort. Even though it is unlikely that bridges will have inbound connections to a non-target ORPort, bridges should still do inbound connection checks, for consistency. These changes are particularly important if a relay is connected to all other relays in the network, but only over IPv4. (Or in the future, only over IPv6.) We expect that these changes will slightly increase the number of relay re-connections, but reduce the number of reachability test circuits required to confirm reachability. 4.4.3. Accurately Identifying Test Circuits The testing relay (or bridge) may confirm that the create cells it is receiving are from its own test circuits, and that test circuits are capable of returning create cells to the origin. Currently, relays confirm reachability if any create cell is received on any inbound connection (see section 4.1). Relays do not check that the circuit is a reachability test circuit, and they do not wait to receive the return created cell. This behaviour has resulted in difficult to diagnose bugs on some rare relay configurations. We propose these optional changes, to improve the efficiency of IPv4 and IPv6 ORPort reachability checks: Testing relays may: * check that the create cell is received from a test circuit (by comparing the received cell to the cells sent by test circuits), * check that the create cell is received on an inbound connection (this is existing behaviour), * if the create cell from a test circuit is received on an outbound connection, destroy the circuit (rather than returning a created cell), and * check that the created cell is returned to the relay on a test circuit (by comparing the remote address of the final hop on the circuit, to the local IPv4 and IPv6 ORPort addresses). TODO: work out how to efficiently match inbound create cells to test circuits. 4.4.4. Allowing More Relay IPv6 Extends Currently, clients, relays, and bridges do not include IPv6 ORPorts in their extend cells. In this proposal, we only make relays (and bridges) extend over IPv6 on the final hop of test circuits. This limited use of IPv6 extends means that IPv6 connections will still be uncommon. We propose these optional changes, to increase the number of IPv6 connections between relays: To increase the number of IPv6 connections, relays that support IPv6 extends may want to use them for all hops of their own circuits. Relays make their own circuits for reachability tests, bandwidth tests, and ongoing preemptive circuits. (Bridges can not change their behaviour, because they try to imitate clients.) We propose a torrc option and consensus parameter SendIPv6CircuitExtends, which is only supported on relays (and not bridges or clients). This option makes relays send IPv4 and IPv6 ORPorts in all their extend cells, when supported by the extending and receiving relay. (See section 3.2.1.) TODO: Is there a shorter name, that isn't easily confused with enabling support for other nodes to perform IPv6 extends via this relay? The default value for this option is "auto", which checks the consensus parameter. If the consensus parameter is not set, it defaults to "0" in the initial release. Once IPv6 extends have had enough testing, we may enable SendIPv6CircuitExtends on the network. The consensus parameter will be set to 1. The default will be changed to "1" (if the consensus parameter is not set). We defer any client (and bridge) changes to a separate proposal, to be implemented when there are more IPv6 relays in the network. But we note that relay IPv6 extends will provide some cover traffic when clients eventually use IPv6 extends in their circuits. As a useful side effect, increasing the number of IPv6 connections in the network makes it more likely that an existing connection can be used for the final hop of a relay IPv6 ORPort reachability check. 4.5. Alternate Reachability Designs We briefly mention some potential reachability designs, and the reasons that they were not used in this proposal. 4.5.1. Removing IPv4 ORPorts from Extend Cells We avoid designs that only include IPv6 ORPorts in extend cells, and remove IPv4 ORPorts. Only including the IPv6 ORPort would provide slightly more specific reachability check circuits. However, we don't need IPv6-only designs, because relays continue trying different reachability circuits until they confirm reachability. IPv6-only designs also make it easy to distinguish relay reachability extend cells from other extend cells. This distinguisher will become more of an issue as IPv6 extends become more common in the network (see sections 4.2.2 and 4.4.4). Removing the IPv4 ORPort also provides no fallback, if the IPv6 ORPort is actually unreachable. IPv6-only failures do not affect reachability checks, but they will become important in the future, as other circuit types start using IPv6 extends. IPv6-only reachability designs also increase the number of special cases in the implementation. (And the likelihood of subtle bugs.) These designs may be appropriate in future, when there are IPv6-only bridges or relays. 5. New Relay Subprotocol Version We reserve Tor subprotocol "Relay=3" for: * relays that support for IPv6 extends, and * relays and bridges that support IPv6 ORPort reachability checks, as described in this proposal. 5.1. Tor Specification Changes We propose the following changes to the [Tor Specification], once this proposal is implemented. Adding a new Relay subprotocol version lets testing relays identify other relays that support IPv6 extends. It also allows us to eventually recommend or require support for IPv6 extends on all relays (and bridges). Append to the Relay version 2 subprotocol specification: Relay=2 has limited IPv6 support: * Clients do not include IPv6 ORPorts in EXTEND2 cells. * Relays do not not initiate IPv6 connections in response to EXTEND2 cells containing IPv6 ORPorts. However, relays accept inbound connections to their IPv6 ORPorts, and will extend circuits via those connections. "3" -- relays support extending over IPv6 connections in response to an EXTEND2 cell containing an IPv6 ORPort. Relays and bridges perform IPv6 ORPort reachability checks. Client behaviour does not change. This subprotocol is advertised by all relays and bridges, regardless of their configured ORPorts. But relays without a configured IPv6 ORPort may refuse to extend over IPv6. And bridges always refuse to extend over IPv6, because they try to imitate client behaviour. A successful IPv6 extend requires: * Relay subprotocol version 3 (or later) and an IPv6 ORPort on the extending relay, * an IPv6 ORPort in the EXTEND2 cell, and * an IPv6 ORPort on the accepting relay. (Because different tor instances can have different views of the network, these checks should be done when the path is selected. Extending relays should only check local IPv6 information, before attempting the extend.) This subprotocol version is described in proposal 311, and implemented in Tor 0.4.4.1-alpha. (TODO: check version after code is merged). 6. Test Plan We provide a quick summary of our testing plans. 6.1. Test IPv6 ORPort Reachability and Extends We propose to test these changes using chutney networks with AssumeReachable disabled. (Chutney currently enables AssumeReachable by default.) We also propose to test these changes on the public network with a small number of relays and bridges. Once these changes are merged, volunteer relay and bridge operators will be able to test them by: * compiling from source, * running nightly builds, or * running alpha releases. 6.2. Test Existing Features We will modify and test these existing features: * IPv4 ORPort reachability checks We do not plan on modifying these existing features: * Relay reachability retries TODO: Do relays re-check their own reachability? How often? * Relay canonical connections * "Too many connections" warning logs But we will test that they continue to function correctly, and fix any bugs triggered by the modifications in this proposal. 6.3. Test Legacy Relay Compatibility We will also test IPv6 extends from newer relays (which implement this proposal) to older relays (which do not). Although this proposal does not create these kinds of circuits, we need to check for bugs and excessive logs in older tor versions. 8. Ongoing Monitoring To monitor the impact of these changes, relays should collect basic IPv4 and IPv6 connection and bandwidth statistics (see [Proposal 313]). We may also collect separate statistics on connections from: * clients (and bridges, because they act like clients), and * other relays (and authorities, because they act like relays). We do not propose to collect additional statistics on: * bridges, * circuit counts, or * failure rates. Collecting statistics like these could impact user privacy. References: [Onion Service Protocol]: In particular, Version 3 of the Onion Service Protocol supports IPv6: https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt [Proposal 306]: One possible design for automatic client IPv4 and IPv6 connections is at https://gitweb.torproject.org/torspec.git/tree/proposals/306-ipv6-happy-eye… (TODO: modify to include bridge changes with client changes) [Proposal 312]: https://gitweb.torproject.org/torspec.git/tree/proposals/312-auto-relay-ipv… (TODO) [Proposal 313]: https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-sta… (TODO) [Relay Search] https://metrics.torproject.org/rs.html [Tor Specification]: https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt -- (End)

3 3