- tor-dev - lists.torproject.org

(Short) Arti Report 6: September 15 through September 30
by Nick Mathewson 01 Oct '21

01 Oct '21

# Arti Report 6: September 15 through September 30 ## Activities since our last report Hello! Lots of code since last time, but not so much to say about it. I'm chugging ahead on the Guard nodes implementation, but it isn't done yet. Most of the code is there, but I need to write tests, chase down a few spec issues, fix a couple of things that "don't feel right", and integrate it all with tor-circmgr. You can track progress on the guard implementation at <https://gitlab.torproject.org/tpo/core/arti/-/issues/58>. ## Thanks! Thanks to Jani Monoses for refactoring, Daniel Eades for cleanup, and Trinity Pointard for OSX reproducible builds and integration testing (!).

1 0

Re: [tor-dev] Proposal 334: A flag to mark Relays as middle-only
by ezhigp＠Safe-mail.net 18 Sep '21

18 Sep '21

-------- Original Message -------- From: Neel Chauhan <neel(a)neelc.org> Apparently from: tor-dev-bounces(a)lists.torproject.org To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Proposal 334: A flag to mark Relays as middle-only Date: Fri, 17 Sep 2021 16:09:43 -0700 > Hi nusenu (and tor-dev@), > > On 2021-09-17 16:02, nusenu wrote: > > it would be great if you could open a MR for the proposal so we can > > always see the latest version and changes > > there. > > (Over time it became unclear what comments have already been addressed > > in the text an which didn't.) > > Done: https://gitlab.torproject.org/tpo/core/torspec/-/merge_requests/46 Line 102> single directory authority servre [3]. Typo here. > > > kind regards, > > nusenu > > -Neel > _______________________________________________ > tor-dev mailing list > tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

1 0

Proposal 332: Ntor protocol with extra data, version 3.
by Nick Mathewson 18 Sep '21

18 Sep '21

``` Filename: 332-ntor-v3-with-extra-data.md Title: Ntor protocol with extra data, version 3. Author: Nick Mathewson Created: 12 July 2021 Status: Open ``` # Overview The ntor handshake is our current protocol for circuit establishment. So far we have two variants of the ntor handshake in use: the "ntor v1" that we use for everyday circuit extension (see `tor-spec.txt`) and the "hs-ntor" that we use for v3 onion service handshake (see `rend-spec-v3.txt`). This document defines a third version of ntor, adapting the improvements from hs-ntor for use in regular circuit establishment. These improvements include: * Support for sending additional encrypted and authenticated protocol-setup handshake data as part of the ntor handshake. (The information sent from the client to the relay does not receive forward secrecy.) * Support for using an external shared secret that both parties must know in order to complete the handshake. (In the HS handshake, this is the subcredential. We don't use it for circuit extension, but in theory we could.) * Providing a single specification that can, in the future, be used both for circuit extension _and_ HS introduction. # The improved protocol: an abstract view Given a client "C" that wants to construct a circuit to a relay "S": The client knows: * B: a public "onion key" for S * ID: an identity for S, represented as a fixed-length byte string. * CM: a message that it wants to send to S as part of the handshake. * An optional "verification" string. The relay knows: * A set of [(b,B)...] "onion key" keypairs. One of them is "current", the others are outdated, but still valid. * ID: Its own identity. * A function for computing a server message SM, based on a given client message. * An optional "verification" string. This must match the "verification" string from the client. Both parties have a strong source of randomness. Given this information, the client computes a "client handshake" and sends it to the relay. The relay then uses its information plus the client handshake to see if the incoming message is valid; if it is, then it computes a "server handshake" to send in reply. The client processes the server handshake, and either succeeds or fails. At this point, the client and the relay both have access to: * CM (the message the client sent) * SM (the message the relay sent) * KS (a shared byte stream of arbitrary length, used to compute keys to be used elsewhere in the protocol). Additionally, the client knows that CM was sent _only_ to the relay whose public onion key is B, and that KS is shared _only_ with that relay. The relay does not know which client participated in the handshake, but it does know that CM came from the same client that generated the key X, and that SM and KS were shared _only_ with that client. Both parties know that CM, SM, and KS were shared correctly, or not at all. Both parties know that they used the same verification string; if they did not, they do not learn what the verification string was. (This feature is required for HS handshakes.) # The handshake in detail ## Notation We use the following notation: * `|` -- concatenation * `"..."` -- a byte string, with no terminating NUL. * `ENCAP(s)` -- an encapsulation function. We define this as `htonll(len(s)) | s`. (Note that `len(ENCAP(s)) = len(s) + 8`). * `PARTITION(s, n1, n2, n3, ...)` -- a function that partitions a bytestring `s` into chunks of length `n1`, `n2`, `n3`, and so on. Extra data is put into a final chunk. If `s` is not long enough, the function fails. We require the following crypto operations: * `KDF(s,t)` -- a tweakable key derivation function, returning a keystream of arbitrary length. * `H(s,t)` -- a tweakable hash function of output length `DIGEST_LEN`. * `MAC(k, msg, t)` -- a tweakable message-authentication-code function, of output length `MAC_LEN`. * `EXP(pk,sk)` -- our Diffie Hellman group operation, taking a public key of length `PUB_KEY_LEN`. * `KEYGEN()` -- our Diffie-Hellman keypair generation algorithm, returning a (secret-key,public-key) pair. * `ENC(k, m)` -- a stream cipher with key of length `ENC_KEY_LEN`. `DEC(k, m)` is its inverse. Parameters: * `PROTOID` -- a short protocol identifier * `t_*` -- a set of "tweak" strings, used to derive distinct hashes from a single hash function. * `ID_LEN` -- the length of an identity key that uniquely identifies a relay. Given our cryptographic operations and a set of tweak strings, we define: ``` H_foo(s) = H(s, t_foo) MAC_foo(k, msg) = MAC(k, msg, t_foo) KDF_foo(s) = KDF(s, t_foo) ``` See Appendix A.1 below for a set of instantiations for these operations and constants. ## Client operation, phase 1 The client knows: B, ID -- the onion key and ID of the relay it wants to use. CM -- the message that it wants to send as part of its handshake. VER -- a verification string. First, the client generates a single-use keypair: x,X = KEYGEN() and computes: Bx = EXP(B,x) secret_input_phase1 = Bx | ID | X | B | PROTOID | ENCAP(VER) phase1_keys = KDF_msgkdf(secret_input_phase1) (ENC_K1, MAC_K1) = PARTITION(phase1_keys, ENC_KEY_LEN, MAC_KEY_LEN) encrypted_msg = ENC(ENC_K1, CM) msg_mac = MAC_msgmac(MAC_K1, ID | B | X | encrypted_msg) and sends: NODEID ID [ID_LEN bytes] KEYID B [PUB_KEY_LEN bytes] CLIENT_PK X [PUB_KEY_LEN bytes] MSG encrypted_msg [len(CM) bytes] MAC msg_mac [last MAC_LEN bytes of message] The client remembers x, X, B, ID, Bx, and msg_mac. ## Server operation The relay checks whether NODEID is as expected, and looks up the (b,B) keypair corresponding to KEYID. If the keypair is missing or the NODEID is wrong, the handshake fails. Now the relay uses `X=CLIENT_PK` to compute: Xb = EXP(X,b) secret_input_phase1 = Xb | ID | X | B | PROTOID | ENCAP(VER) phase1_keys = KDF_msgkdf(secret_input_phase1) (ENC_K1, MAC_K1) = PARTITION(phase1_keys, ENC_KEY_LEN, MAC_KEY_LEN) expected_mac = MAC_msgmac(MAC_K1, ID | B | X | MSG) If `expected_mac` is not `MAC`, the handshake fails. Otherwise the relay computes `CM` as: CM = DEC(MSG, ENC_K1) The relay then checks whether `CM` is well-formed, and in response composes `SM`, the reply that it wants to send as part of the handshake. It then generates a new ephemeral keypair: y,Y = KEYGEN() and computes the rest of the handshake: Xy = EXP(X,y) secret_input = Xy | Xb | ID | B | X | Y | PROTOID | ENCAP(VER) ntor_key_seed = H_key_seed(secret_input) verify = H_verify(secret_input) RAW_KEYSTREAM = KDF_final(ntor_key_seed) (ENC_KEY, KEYSTREAM) = PARTITION(RAW_KEYSTREAM, ENC_KEY_LKEN, ...) encrypted_msg = ENC(ENC_KEY, SM) auth_input = verify | ID | B | Y | X | MAC | ENCAP(encrypted_msg) | PROTOID | "Server" AUTH = H_auth(auth_input) The relay then sends: Y Y [PUB_KEY_LEN bytes] AUTH AUTH [DIGEST_LEN bytes] MSG encrypted_msg [len(SM) bytes, up to end of the message] The relay uses KEYSTREAM to generate the shared secrets for the newly created circuit. ## Client operation, phase 2 The client computes: Yx = EXP(Y, x) secret_input = Yx | Bx | ID | B | X | Y | PROTOID | ENCAP(VER) ntor_key_seed = H_key_seed(secret_input) verify = H_verify(secret_input) auth_input = verify | ID | B | Y | X | MAC | ENCAP(MSG) | PROTOID | "Server" AUTH_expected = H_auth(auth_input) If AUTH_expected is equal to AUTH, then the handshake has succeeded. The client can then calculate: RAW_KEYSTREAM = KDF_final(ntor_key_seed) (ENC_KEY, KEYSTREAM) = PARTITION(RAW_KEYSTREAM, ENC_KEY_LKEN, ...) SM = DEC(ENC_KEY, MSG) SM is the message from the relay, and the client uses KEYSTREAM to generate the shared secrets for the newly created circuit. # Security notes Whenever comparing bytestrings, implementations SHOULD use constant-time comparison function to avoid side-channel attacks. To avoid small-subgroup attacks against the Diffie-Hellman function, implementations SHOULD either: * Make sure that all incoming group members are in fact in the DH group. * Validate all outputs from the EXP function to make sure that they are not degenerate. # Notes on usage We don't specify what should actually be done with the resulting keystreams; that depends on the usage for which this handshake is employed. Typically, they'll be divided up into a series of tags and symmetric keys. The keystreams generated here are (conceptually) unlimited. In practice, the usage will determine the amount of key material actually needed: that's the amount that clients and relays will actually generate. The PROTOID parameter should be changed not only if the cryptographic operations change here, but also if the usage changes at all, or if the meaning of any parameters changes. (For example, if the encoding of CM and SM changed, or if ID were a different length or represented a different type of key, then we should start using a new PROTOID.) # A.1 Instantiation Here are a set of functions based on SHA3, SHAKE128, Curve25519, and AES256: ``` H(s, t) = SHA3_256(ENCAP(t) | s) MAC(k, msg, t) = SHA3_256(ENCAP(t) | ENCAP(k) | s) KDF(s, t) = SHAKE_128(ENCAP(t) | s) ENC(k, m) = AES_256_CTR(k, m) EXP(pk,sk), KEYGEN: defined as in curve25519 DIGEST_LEN = MAC_LEN = ENC_KEY_LEN = PUB_KEY_LEN = 32 ID_LEN = 32 (representing an ed25519 identity key) ``` Notes on selected operations: SHA3 can be pretty slow, and AES256 is likely overkill. I'm choosing them anyway because they are what we use in hs-ntor, and in my preliminary experiments they don't account for even 1% of the time spent on this handshake. ``` t_msgkdf = PROTOID | ":kdf_phase1" t_msgmac = PROTOID | ":msg_mac" t_key_seed = PROTOID | ":key_seed" t_verify = PROTOID | ":verify" t_final = PROTOID | ":kdf_final" t_auth = PROTOID | ":auth_final" ``` # A.2 Encoding for use with Tor circuit extension Here we give a concrete instantiation of ntor-v3 for use with circuit extension in Tor, and the parameters in A.1 above. If in use, this is a new CREATE2 type. Clients should not use it unless the relay advertises support by including an appropriate version of the `Relay=X` subprotocol in its protocols list. When the encoding and methods of this section, along with the instantiations from the previous section, are in use, we specify: PROTOID = "ntor3-curve25519-sha3_256-1" The key material is extracted as follows, unless modified by the handshake (see below). See tor-spec.txt for more info on the specific values: Df Digest authentication, forwards [20 bytes] Db Digest authentication, backwards [20 bytes] Kf Encryption key, forwards [16 bytes] Kb Encryption key, backwards [16 bytes] KH Onion service nonce [20 bytes] We use the following meta-encoding for the contents of client and server messages. [Any number of times]: TYPE [one byte] LEN [one byte] BODY [LEN bytes] We do not specify specific TYPE semantics here; we leave those for other proposals. All parties MUST reject messages that are not well-formed per the rules above. To avoid partitioning, clients MUST reject messages with TYPEs that they do not recognize. (Therefore, whenever we specify a new server message TYPE, we must say that it can only be included if the client signals that it understands it.) # A.3 How much space is available? We start with a 498-byte payload in each relay cell. The header of the EXTEND2 cell, including link specifiers and other headers, comes to 89 bytes. The client handshake requires 128 bytes (excluding CM). That leaves 281 bytes, "which should be plenty". # X.1 Negotiating proposal-324 circuit windows (We should move this section into prop324 when this proposal is finished.) We define a type value, CIRCWINDOW_INC. We define a triplet of consensus parameters: `circwindow_inc_min`, `cincwindow_inc_max`, and `circwindow_inc_dflt`. These all have range (1,65535). When the authority operators want to experiment with different values for `circwindow_inc_dflt`, they set `circwindow_inc_min` and `circwindow_inc_max` to the range in which they want to experiment, making sure that the existing `circwindow_inc_dflt` is within that range. vWhen a client sees that a relay supports the ntor3 handshake type (subprotocol `Relay=X`), and also supports the flow control algorithms of proposal 324 (subprotocol `FlowCtrl=X`), then the client sends a message, with type `CIRCWINDOW_INC`, containing a two-byte integer equal to `circwindow_inc_dflt`. The relay rejects the message if the value given is outside of the [`circwindow_inc_min`, `circwindow_inc_max`] range. Otherwise, it accepts it, and replies with the same message that the client sent.

4 7

Proposal 334: A flag to mark Relays as middle-only
by Neel Chauhan 17 Sep '21

17 Sep '21

Hi, As asked in the torspec MR [1] (42) for ticket [2] (40448), I propose a MiddleOnly dirauth flag for relays. The proposal, #334, is attached to this email, and is titled "A dirauth flag to mark Relays as Middle-only". Please comment and review it. Best, Neel Chauhan === Links: [1] - https://gitlab.torproject.org/tpo/core/torspec/-/merge_requests/42 [2] - https://gitlab.torproject.org/tpo/core/tor/-/issues/40448

7 19

Proposal 334: A flag to mark Relays as middle-only
by Tor Relays 16 Sep '21

16 Sep '21

David Goulet: > However, I'm not sure we should always let 1 authority dictate that flag > regardless of what the others think. > > I think we need to enforce majority here and not have one > single authority dictate it. > > Thoughts? > +1 I can compromise one authority and can MiddleOnly the whole Tor network.

2 1

Arti report 5: August 18 through September 15
by Nick Mathewson 15 Sep '21

15 Sep '21

# Arti Report 5: August 4 through September 15 ## Activities since our last report I'm back, with updates from the last month! We've spent a lot of time over the last on cleaning up technical debt issues in our code that had accumulated as I wrote it. There are more tests and warnings enabled than before, and we've changed some new internal APIs to make them unable to panic. We're also using the [`derive_builder`][derive_builder] tool to throw out a bunch of boilerplate code. When I wrote last month, I'd just finished up the implementation for circuit build timeout inference. As followup work to that, I worked with Mike Perry to collect all of the missing specification issues that I'd found while cloning that feature, and [roll them back][torspec-mr-40] into our `path-spec.txt` document. Mike'sw found a couple of mistakes in my implementation that I was able to resolve—fortunately, by removing code from the Arti timeout inference code. ## Getting a head start on API design API design will be the focus of our 0.1.0 release, but we still want to get it "as right as we can" in the upcoming 0.0.1 version. With that in mind, we've started enumerating the various kinds of programs and environments that we need to support down the road, and [working on a list of example programs][ticket-164] that we might be able to ship with Arti. (My own experience so far is that writing example code is just about the best way to find API deficiencies.) ## Uplifting new features from C Tor For our work on congestion control, we're getting a new [circuit extension handshake][prop332]. Going forward, as we add features to C Tor, we're going to try to add them to Arti at around the same time. With that in mind, we have the new handshake implemented (but not enabled): we used its implementation to validate our specifications double-check our test vectors. ## Towards guard support Our largest remaining priority for Arti 0.0.1 is full support for [guard nodes][guard-spec]. (In Tor, a "guard node" is a relay that a client decides to use as the first hop for all of its circuits for a long time. Using guard nodes improves resistance against many kinds of attacks based on hostile relays.) We've started laying the groundwork here: we now have support for all of the guard-related consensus parameters, and we've reworked our relay selection code to allow picking multiple weighted random items without replacement. We've also added some previously unneeded lookup-by-ID features to network directory code. (Fun fact: the Tor implementation of mapping IDs to relays has been historically ugly, since we don't really know a relay's ed25519 identity until we have a microdescriptor for it. In C, this has led to a reasonably messy pile of dangling pointer bugs…which Rust won't even let us write! Also, we've made better design choices in Arti about our directory objects, so we don't have do support all of the strange mutability that our C code had to try to handle.) We're now in the midst of coding up a guard implementation. The tricky part here is that our guard logic affects circuit building selection (since circuit paths must begin with a guard), but circuit construction affects guard selection. We're experimenting with different designs to try to keep the complexity here under control. # Thanks to awesome volunteers! Thanks to Jani Monoses for migrating Arti to the tracing crate and adding journald support; to Robin Leander Schröder for cleaning up some of our `unwrap_or()` code; to S0AndS0 for work enabling the `unwrap_used` Clippy lint; to Daniel Eades for work cleaning up Clippy lints in our test modules, and to Trinity Pointard for work on stream isolation and reproducible builds! [ticket-164]: https://gitlab.torproject.org/tpo/core/arti/-/issues/164 [prop-332]: https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/332-nt… [derive_builder]: https://crates.io/crates/derive_builder [torspec-mr-40]: https://gitlab.torproject.org/tpo/core/torspec/-/merge_requests/40 [guard-spec]: https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/guard-spec.txt

1 0

Proposal 334: A flag to mark Relays as middle-only
by ezhigp＠Safe-mail.net 14 Sep '21

14 Sep '21

Neel Chauman wrote at Sun Sep 12 19:17:37 UTC 2021: >my updated proposal >says that if one dirauth gives a relay the MiddleOnly flag, then it's >set for that relay. This is to prevent harm while all (or the majority >of) dirauths give the relay that flag. Imagine one hostile dirauth that votes for this flag on every relay it cannot control. Guessing the result is left as an exercise to the reader :). (Spoiler: IIUC, this will result in using only attacker-controlled relays as entries and exits, resulting in both-ends control of all external and hidserv circuits).

1 0

No arti report this week.
by Nick Mathewson 01 Sep '21

01 Sep '21

Hi! Usually I do Arti reports every other Wednesday, but this week I'm on vacation. I'll send the next report on September 15, so that the reports can be offset from the Arti meetings. cheers, -- Nick

1 0

Next public Arti meeting happening on September 8th at 1500 UTC
by Gaba 26 Aug '21

26 Aug '21

Hi! We are canceling the meeting on September 1st. Our next meeting will be on September 8th (and then every 2 weeks). Meeting's room: https://tor.meet.coop/gab-sis-ldw-xad Meeting notes in: http://kfahv6wfkbezjyg4r6mlhpmieydbebr5vkok5r34ya464gqz6c44bnyd.onion/p/art… Recording of previous meetings: https://www.youtube.com/playlist?list=PLwyU2dZ3LJErozq7cTtImuWkSn_34pEpq Agenda - status of Arti development and planning - user stories for Arti API https://pad.riseup.net/p/MUjnic0eRT0WrDW7M-us cheers gaba -- pronouns she/her/they GPG Fingerprint EE3F DF5C AD91 643C 21BE 8370 180D B06C 59CA BD19

1 0

Arti report 4: August 4 through August 18
by Nick Mathewson 18 Aug '21

18 Aug '21

# Arti Report 3: August 4 through August 18 At last we're done with circuit timeout inference! See the last weeks update for an overview of what this was, and why it was hard. As of last Friday, we've finally got an Arti implementation that builds testing circuits as appropriate, infers reasonable timeouts, and remembers those timeouts across runs. As a side-benefit to this work, we've found a number of points where we need to clarify the Tor specification to be more precise about correct behavior for a Tor client. (For example, how to break ties when deciding on the most populous histogram bins.) I'm taking some time to circle back and fix those in the Tor specs. Here is the first recorded persistent state from an Arti implementation, lightly reformatted: version = 1 histogram = [[415, 1], [495, 2], [515, 1], [535, 1], [545, 1], [555, 1], [625, 1], [655, 1], [665, 1], [785, 1], [835, 1], [865, 2], [885, 3], [955, 1], [965, 1], [975, 2], [985, 1], [1005, 2], [1025, 1], [1035, 2], [1055, 3], [1065, 1], [1075, 1], [1085, 2], [1095, 1], [1115, 2], [1135, 2], [1145, 2], [1175, 1], [1185, 1], [1195, 2], [1205, 3], [1215, 1], [1235, 2], [1245, 6], [1255, 1], [1265, 2], [1285, 2], [1295, 3], [1305, 1], [1315, 1], [1355, 2], [1375, 2], [1385, 2], [1405, 2], [1435, 1], [1455, 1], [1465, 1], [1495, 1], [1505, 1], [1525, 1], [1535, 1], [1555, 1], [1565, 1], [1605, 1], [1625, 1], [1665, 1], [1765, 1], [1785, 1], [1825, 1], [1965, 1], [2165, 1], [2495, 1], [3025, 1], [3855, 1], [4265, 1], [4795, 1], [4935, 1], [5435, 1], [6825, 1]] current_timeout = 1592 abandoned_circs = 0 successful_circs = 20 ## Other work We've continued to review and merge volunteer patches. Thanks, everybody! In response to a patch documenting a number of possible panics, I've taken a pass through the code to reduce the number of functions that could panic. I've also been spending some time improving test coverage: while doing so, I found an incorrect behavior when checking whether two IPv6 addresses were on the same network. (Tests are great and need to be everywhere, apparently.) ## Coming up Our current priorities are: * Starting API design work. So far we've been feeling our way forward, and it's time to get more systematic. * Improving our APIs and internal functionality for stream isolation, to support a variety of usecases more like what can be done with current Tor. * Kicking off the work on a Guard implementation. This is the last key feature we need for a basic level of privacy. * Lots more interviewing job candidates. Let's see how far we get in the next two weeks!

1 0