This is an automated email from the git hooks/post-receive script.

dgoulet pushed a commit to branch main in repository torspec.

commit 21d7e9a41f9c2449345f36f567700f8be3992f83 Author: David Goulet dgoulet@torproject.org AuthorDate: Tue May 9 11:53:19 2023 -0400

prop340: Tweak how we handle optional stream IDs

Introduce an optional header called the "message routing header" which contains a `stream_id` and is only present for commands that do require it.

We also specify that if an unrecognized command is encountered, the circuit MUST be destroyed immediately.

Signed-off-by: David Goulet dgoulet@torproject.org --- proposals/340-packed-and-fragmented.md | 238 ++++++++++++++++----------------- 1 file changed, 117 insertions(+), 121 deletions(-)

diff --git a/proposals/340-packed-and-fragmented.md b/proposals/340-packed-and-fragmented.md index 6fb1ca5..121be68 100644 --- a/proposals/340-packed-and-fragmented.md +++ b/proposals/340-packed-and-fragmented.md @@ -34,7 +34,8 @@ This proposal combines ideas from [next-generation relay cryptography](./308-counter-galois-onion).

Additionally, this proposal has been revised to incorporate another -protocol change, and remove StreamId from the relay cell header. +protocol change, and move StreamId from the relay cell header into a new, +optional header.

## A preliminary change: Relay encryption, version 1.5

@@ -54,36 +55,50 @@ The new format for a decrypted relay _cell_ will be: digest [14 bytes] body [509 - 16 = 493 bytes]

-"Digest" and "recognized" are computed as before; the only difference -is that they occur _before_ the rest of the cell, and that "digest" is -truncated to 14 bytes instead of 4. +The `recognized` and `digest` fields are computed as before; the only +difference is that they occur _before_ the rest of the cell, and that `digest` +is truncated to 14 bytes instead of 4.

If we are lucky, we won't have to build this encryption at all, and we can just move to some version of GCM-UIV or other RPRP that reserves 16 bytes for an authentication tag or similar cryptographic object.

+The `body` MUST contain exactly 493 bytes as cells have a fixed size. + ## New relay message packing

-Relay _messages_ now follow the following format: +We define this new format for a relay message which has to fit within one +relay cell. However, the body can be split accross many relay cells: + +``` + Message Header + command u8 + length u16 + Message Routing Header (optional) + stream_id u16 + Message Body + data u8[length] +```

- Header - command u8 - length u16 - Body - data u8[length] +One big change from the current tor protocol is something that has become +optional: we have moved `stream_id` into a separate inner header that only +appears sometimes named the Message Routing Header. The command value tells us +if the header is to be expected or not. + +The following message types take required stream IDs: `BEGIN`, `DATA`, `END`, +`CONNECTED`, `RESOLVE`, `RESOLVED`, and `BEGIN_DIR`, `XON`, `XOFF`.

-We require that "command" is never 0. +The following message types from proposal 339 (UDP) take required stream IDs: +`CONNECT_UDP`, `CONNECTED_UDP` and `DATAGRAM`.

-One big change from the current tor protocol is something that is _not_ -here: we have moved `stream_id` into the body of the relay message of -those commands that need it. +No other message types take stream IDs. The `stream_id` field, when present, +MUST NOT be zero.

Messages can be split across relay cells; multiple messages can occur in a single relay cell. We enforce the following rules:

* Headers may not be split across cells. - * If a 0 byte follows any relay message, there are no more messages in - that cell. + * If a 0 byte follows a message body, there are no more messages. * A relay cell may not be "empty": it must have at least some part of some message.

@@ -91,15 +106,15 @@ Unless specified elsewhere, **all** message types may be packed, and **all** message types may be fragmented.

Every command has an associated maximum length for its messages. If not -specified elsewhere, the maximum length for every message is 498 bytes -(for legacy reasons). - -Receivers MUST validate that headers are well-formed and have valid -lengths while handling the cell in which the header is encoded. If the -header is invalid, the receiver must destroy the circuit. - +specified elsewhere, the maximum length for every message is 498 bytes (for +legacy reasons).

+Receivers MUST validate that the cell `header` and the `message header` are +well-formed and have valid lengths while handling the cell in which the header +is encoded. If any of them is invalid, the circuit MUST be destroyed.

+An unrecognized `command` is considered invalid and thus MUST result in the +circuit being immediately destroyed.

## Negotiation

@@ -115,42 +130,6 @@ The `version` field is the `Relay` subprotocol version that the client wants to use. The relay must send back the same extension in its ntor3 handshake to acknowledge support.

- -## Specifying the message format with moved stream ID. - -Here, we'll specify how to adjust tor-spec to describe the `stream_id` -move correctly. - -Below, suppose that `Relay=V` denotes whatever version of the relay -message subprotocol denotes support for this proposal. - -For all relay message types that include a stream ID, we insert -the following at the beginning of their fields: - ->``` -> STREAM_ID [2 bytes] (Present when Relay protocol version >= V). ->``` - -We add a note saying: - -> STREAM_ID is part of many message types, when using Relay protocol -> version V or later. Earlier versions of the Relay protocol put -> STREAM_ID in the RELAY header. In those versions, the field is -> STREAM_ID omitted from the message. -> -> Except when noted, STREAM_ID may not be zero. - -The following message types take required stream IDs: `BEGIN`, `DATA`, `END`, -`CONNECTED`, `RESOLVE`, `RESOLVED`, and `BEGIN_DIR`, `XON`, `XOFF`. - -The following message type takes an *optional* stream ID: `SENDME`. -(*Stream-level sendmes are not a thing anymore with proposal 324, but I -want to give implementations the freedom to implement prop324 and this -proposal in either order.*) - -The following message types from proposal 339 (UDP) take required stream -IDs: `CONNECT_UDP`, `CONNECTED_UDP` and `DATAGRAM`. - ## Migration

We add a consensus parameter, "streamed-relay-messages", with default @@ -204,7 +183,7 @@ this.)

### Extending message-length maxima

-For now, the maximum length for every message is 498 bytes, except as +For now, the maximum length for every message body is 493 bytes, except as follows:

- `DATAGRAM` messages (see proposal 339) have a maximum body length @@ -218,149 +197,166 @@ saying so, and reserving a new subprotocol version.)

# Appendix: Example cells

- -Here is an example of the simplest case: one message, sent in one relay -cell. Here it is a BEGIN message. +Here is an example of the simplest case: one message, sent in one relay cell:

``` Cell 1: - Cell header + header: circid .. [4 bytes] command RELAY [1 byte] - relay cell header + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] message header: command BEGIN [1 byte] length 23 [2 bytes] - message body - stream_id (...) [2 bytes] + message routing header: + stream_id 42 [2 bytes] + message body: "www.torproject.org:443\0" [23 bytes] - end-of-messages marker + end-of-messages marker: 0 [1 byte] - padding up to end of cell + padding up to end of cell: random [464 bytes] - ```

+Total of 514 bytes which is the absolute maximum cell size. + Here's an example with fragmentation only: a large EXTEND2 message split across two relay cells.

``` Cell 1: - Cell header - circid .. [4 bytes] - command RELAY_EARLY [1 byte] - relay cell header - recognized 0 [2 bytes] - digest (...) [14 bytes] + header: + circid .. [4 bytes] + command RELAY_EARLY [1 byte] + relay cell header: + recognized 0 [2 bytes] + digest (...) [14 bytes] message header: - command EXTEND [1 byte] - length 800 [2 bytes] - message body - stream_id 0 [2 bytes] - (extend body, part 1) [488 bytes] + command EXTEND [1 byte] + length 800 [2 bytes] + message body: + (extend body, part 1) [490 bytes]

Cell 2: - Cell header - circid .. [4 bytes] - command RELAY [1 byte] - relay cell header + header: + circid .. [4 bytes] + command RELAY [1 byte] + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] - message body, continued - (extend body, part 2) [312 bytes] - end-of-messages marker + message body, continued: + (extend body, part 2) [310 bytes] (310+490=800) + end-of-messages marker: 0 [1 byte] - padding up to end of cell - random [180 bytes] + padding up to end of cell: + random [182 bytes]

```

-Here is an example with packing only: A begin_dir message and a data -message in the same cell. +Each cells are 514 bytes for a message body totalling 800 bytes. + +Here is an example with packing only: A `BEGIN_DIR` message and a data message +in the same cell.

``` Cell 1: - Cell header + header: circid .. [4 bytes] command RELAY [1 byte] - relay cell header + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] + + # First relay message message header: command BEGIN_DIR [1 byte] length 0 [2 bytes] - message body: + message routing header: stream_id 32 [2 bytes] + + # Second relay message message header: command DATA [1 byte] length 25 [2 bytes] - message body: + message routing header: stream_id 32 [2 bytes] + message body: "HTTP/1.0 GET /tor/foo\r\n\r\n" [25 bytes] - end-of-messages marker + + end-of-messages marker: 0 [1 byte] - padding up to end of cell + padding up to end of cell: random [457 bytes]

```

Here is an example with packing and fragmentation: a large DATAGRAM cell, a -SENDME cell, and an XON cell. (Note that this sequence of cells would not -actually be generated by the algorithm described in "Packing decisions" -above; this is only an example of what parties need to accept.) +SENDME cell, and an XON cell. + +(Note that this sequence of cells would not actually be generated by the +algorithm described in "Packing decisions" above; this is only an example of +what parties need to accept.)

``` Cell 1: - Cell header - circid .. [4 bytes] - command RELAY [1 byte] - relay cell header + header: + circid .. [4 bytes] + command RELAY [1 byte] + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] + + # First message message header: command DATAGRAM [1 byte] length 1200 [2 bytes] - message body + message routing header: stream_id 99 [2 bytes] + message body: (datagram body, part 1) [488 bytes]

Cell 2: - Cell header - circid .. [4 bytes] - command RELAY [1 byte] - relay cell header + header: + circid .. [4 bytes] + command RELAY [1 byte] + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] - message body, continued + message body, continued: (datagram body, part 2) [493 bytes]

Cell 3: - Cell header - circid .. [4 bytes] - command RELAY [1 byte] - relay cell header + header: + circid .. [4 bytes] + command RELAY [1 byte] + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] - message body, continued + message body, continued: (datagram body, part 3) [219 bytes] (488+493+219=1200) + + # Second message message header: command SENDME [1 byte] length 23 [2 bytes] message body: - stream_id 0 [2 bytes] version 1 [1 byte] datalen 20 [2 bytes] data (digest to ack) [20 bytes] + + # Third message message header: command XON [1 byte] length 1 [2 bytes] - message body: + message routing header: stream_id 50 [2 bytes] + message body: version 1 [1 byte] - end-of-messages marker + + end-of-messages marker: 0 [1 byte] - padding up to end of cell - random [239 bytes] + padding up to end of cell: + random [241 bytes] ```