[torspec] branch main updated (4a14d01 -> 8961bb4)
This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch main in repository torspec. from 4a14d01 Merge branch 'tor-gitlab/mr/139' new 21d7e9a prop340: Tweak how we handle optional stream IDs new 8961bb4 prop340: Change the ntorv3 data format The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: proposals/340-packed-and-fragmented.md | 257 +++++++++++++++++---------------- 1 file changed, 134 insertions(+), 123 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository torspec. commit 21d7e9a41f9c2449345f36f567700f8be3992f83 Author: David Goulet <dgoulet@torproject.org> AuthorDate: Tue May 9 11:53:19 2023 -0400 prop340: Tweak how we handle optional stream IDs Introduce an optional header called the "message routing header" which contains a `stream_id` and is only present for commands that do require it. We also specify that if an unrecognized command is encountered, the circuit MUST be destroyed immediately. Signed-off-by: David Goulet <dgoulet@torproject.org> --- proposals/340-packed-and-fragmented.md | 238 ++++++++++++++++----------------- 1 file changed, 117 insertions(+), 121 deletions(-) diff --git a/proposals/340-packed-and-fragmented.md b/proposals/340-packed-and-fragmented.md index 6fb1ca5..121be68 100644 --- a/proposals/340-packed-and-fragmented.md +++ b/proposals/340-packed-and-fragmented.md @@ -34,7 +34,8 @@ This proposal combines ideas from [next-generation relay cryptography](./308-counter-galois-onion). Additionally, this proposal has been revised to incorporate another -protocol change, and remove StreamId from the relay cell header. +protocol change, and move StreamId from the relay cell header into a new, +optional header. ## A preliminary change: Relay encryption, version 1.5 @@ -54,36 +55,50 @@ The new format for a decrypted relay _cell_ will be: digest [14 bytes] body [509 - 16 = 493 bytes] -"Digest" and "recognized" are computed as before; the only difference -is that they occur _before_ the rest of the cell, and that "digest" is -truncated to 14 bytes instead of 4. +The `recognized` and `digest` fields are computed as before; the only +difference is that they occur _before_ the rest of the cell, and that `digest` +is truncated to 14 bytes instead of 4. If we are lucky, we won't have to build this encryption at all, and we can just move to some version of GCM-UIV or other RPRP that reserves 16 bytes for an authentication tag or similar cryptographic object. +The `body` MUST contain exactly 493 bytes as cells have a fixed size. + ## New relay message packing -Relay _messages_ now follow the following format: +We define this new format for a relay message which has to fit within one +relay cell. However, the body can be split accross many relay cells: + +``` + Message Header + command u8 + length u16 + Message Routing Header (optional) + stream_id u16 + Message Body + data u8[length] +``` - Header - command u8 - length u16 - Body - data u8[length] +One big change from the current tor protocol is something that has become +optional: we have moved `stream_id` into a separate inner header that only +appears sometimes named the Message Routing Header. The command value tells us +if the header is to be expected or not. + +The following message types take required stream IDs: `BEGIN`, `DATA`, `END`, +`CONNECTED`, `RESOLVE`, `RESOLVED`, and `BEGIN_DIR`, `XON`, `XOFF`. -We require that "command" is never 0. +The following message types from proposal 339 (UDP) take required stream IDs: +`CONNECT_UDP`, `CONNECTED_UDP` and `DATAGRAM`. -One big change from the current tor protocol is something that is _not_ -here: we have moved `stream_id` into the body of the relay message of -those commands that need it. +No other message types take stream IDs. The `stream_id` field, when present, +MUST NOT be zero. Messages can be split across relay cells; multiple messages can occur in a single relay cell. We enforce the following rules: * Headers may not be split across cells. - * If a 0 byte follows any relay message, there are no more messages in - that cell. + * If a 0 byte follows a message body, there are no more messages. * A relay cell may not be "empty": it must have at least some part of some message. @@ -91,15 +106,15 @@ Unless specified elsewhere, **all** message types may be packed, and **all** message types may be fragmented. Every command has an associated maximum length for its messages. If not -specified elsewhere, the maximum length for every message is 498 bytes -(for legacy reasons). - -Receivers MUST validate that headers are well-formed and have valid -lengths while handling the cell in which the header is encoded. If the -header is invalid, the receiver must destroy the circuit. - +specified elsewhere, the maximum length for every message is 498 bytes (for +legacy reasons). +Receivers MUST validate that the cell `header` and the `message header` are +well-formed and have valid lengths while handling the cell in which the header +is encoded. If any of them is invalid, the circuit MUST be destroyed. +An unrecognized `command` is considered invalid and thus MUST result in the +circuit being immediately destroyed. ## Negotiation @@ -115,42 +130,6 @@ The `version` field is the `Relay` subprotocol version that the client wants to use. The relay must send back the same extension in its ntor3 handshake to acknowledge support. - -## Specifying the message format with moved stream ID. - -Here, we'll specify how to adjust tor-spec to describe the `stream_id` -move correctly. - -Below, suppose that `Relay=V` denotes whatever version of the relay -message subprotocol denotes support for this proposal. - -For all relay message types that include a stream ID, we insert -the following at the beginning of their fields: - ->``` -> STREAM_ID [2 bytes] (Present when Relay protocol version >= V). ->``` - -We add a note saying: - -> STREAM_ID is part of many message types, when using Relay protocol -> version V or later. Earlier versions of the Relay protocol put -> STREAM_ID in the RELAY header. In those versions, the field is -> STREAM_ID omitted from the message. -> -> Except when noted, STREAM_ID may not be zero. - -The following message types take required stream IDs: `BEGIN`, `DATA`, `END`, -`CONNECTED`, `RESOLVE`, `RESOLVED`, and `BEGIN_DIR`, `XON`, `XOFF`. - -The following message type takes an *optional* stream ID: `SENDME`. -(*Stream-level sendmes are not a thing anymore with proposal 324, but I -want to give implementations the freedom to implement prop324 and this -proposal in either order.*) - -The following message types from proposal 339 (UDP) take required stream -IDs: `CONNECT_UDP`, `CONNECTED_UDP` and `DATAGRAM`. - ## Migration We add a consensus parameter, "streamed-relay-messages", with default @@ -204,7 +183,7 @@ this.) ### Extending message-length maxima -For now, the maximum length for every message is 498 bytes, except as +For now, the maximum length for every message body is 493 bytes, except as follows: - `DATAGRAM` messages (see proposal 339) have a maximum body length @@ -218,149 +197,166 @@ saying so, and reserving a new subprotocol version.) # Appendix: Example cells - -Here is an example of the simplest case: one message, sent in one relay -cell. Here it is a BEGIN message. +Here is an example of the simplest case: one message, sent in one relay cell: ``` Cell 1: - Cell header + header: circid .. [4 bytes] command RELAY [1 byte] - relay cell header + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] message header: command BEGIN [1 byte] length 23 [2 bytes] - message body - stream_id (...) [2 bytes] + message routing header: + stream_id 42 [2 bytes] + message body: "www.torproject.org:443\0" [23 bytes] - end-of-messages marker + end-of-messages marker: 0 [1 byte] - padding up to end of cell + padding up to end of cell: random [464 bytes] - ``` +Total of 514 bytes which is the absolute maximum cell size. + Here's an example with fragmentation only: a large EXTEND2 message split across two relay cells. ``` Cell 1: - Cell header - circid .. [4 bytes] - command RELAY_EARLY [1 byte] - relay cell header - recognized 0 [2 bytes] - digest (...) [14 bytes] + header: + circid .. [4 bytes] + command RELAY_EARLY [1 byte] + relay cell header: + recognized 0 [2 bytes] + digest (...) [14 bytes] message header: - command EXTEND [1 byte] - length 800 [2 bytes] - message body - stream_id 0 [2 bytes] - (extend body, part 1) [488 bytes] + command EXTEND [1 byte] + length 800 [2 bytes] + message body: + (extend body, part 1) [490 bytes] Cell 2: - Cell header - circid .. [4 bytes] - command RELAY [1 byte] - relay cell header + header: + circid .. [4 bytes] + command RELAY [1 byte] + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] - message body, continued - (extend body, part 2) [312 bytes] - end-of-messages marker + message body, continued: + (extend body, part 2) [310 bytes] (310+490=800) + end-of-messages marker: 0 [1 byte] - padding up to end of cell - random [180 bytes] + padding up to end of cell: + random [182 bytes] ``` -Here is an example with packing only: A begin_dir message and a data -message in the same cell. +Each cells are 514 bytes for a message body totalling 800 bytes. + +Here is an example with packing only: A `BEGIN_DIR` message and a data message +in the same cell. ``` Cell 1: - Cell header + header: circid .. [4 bytes] command RELAY [1 byte] - relay cell header + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] + + # First relay message message header: command BEGIN_DIR [1 byte] length 0 [2 bytes] - message body: + message routing header: stream_id 32 [2 bytes] + + # Second relay message message header: command DATA [1 byte] length 25 [2 bytes] - message body: + message routing header: stream_id 32 [2 bytes] + message body: "HTTP/1.0 GET /tor/foo\r\n\r\n" [25 bytes] - end-of-messages marker + + end-of-messages marker: 0 [1 byte] - padding up to end of cell + padding up to end of cell: random [457 bytes] ``` Here is an example with packing and fragmentation: a large DATAGRAM cell, a -SENDME cell, and an XON cell. (Note that this sequence of cells would not -actually be generated by the algorithm described in "Packing decisions" -above; this is only an example of what parties need to accept.) +SENDME cell, and an XON cell. + +(Note that this sequence of cells would not actually be generated by the +algorithm described in "Packing decisions" above; this is only an example of +what parties need to accept.) ``` Cell 1: - Cell header - circid .. [4 bytes] - command RELAY [1 byte] - relay cell header + header: + circid .. [4 bytes] + command RELAY [1 byte] + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] + + # First message message header: command DATAGRAM [1 byte] length 1200 [2 bytes] - message body + message routing header: stream_id 99 [2 bytes] + message body: (datagram body, part 1) [488 bytes] Cell 2: - Cell header - circid .. [4 bytes] - command RELAY [1 byte] - relay cell header + header: + circid .. [4 bytes] + command RELAY [1 byte] + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] - message body, continued + message body, continued: (datagram body, part 2) [493 bytes] Cell 3: - Cell header - circid .. [4 bytes] - command RELAY [1 byte] - relay cell header + header: + circid .. [4 bytes] + command RELAY [1 byte] + relay cell header: recognized 0 [2 bytes] digest (...) [14 bytes] - message body, continued + message body, continued: (datagram body, part 3) [219 bytes] (488+493+219=1200) + + # Second message message header: command SENDME [1 byte] length 23 [2 bytes] message body: - stream_id 0 [2 bytes] version 1 [1 byte] datalen 20 [2 bytes] data (digest to ack) [20 bytes] + + # Third message message header: command XON [1 byte] length 1 [2 bytes] - message body: + message routing header: stream_id 50 [2 bytes] + message body: version 1 [1 byte] - end-of-messages marker + + end-of-messages marker: 0 [1 byte] - padding up to end of cell - random [239 bytes] + padding up to end of cell: + random [241 bytes] ``` -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository torspec. commit 8961bb4d83fccb2b987f9899ca83aa430f84ab0c Author: David Goulet <dgoulet@torproject.org> AuthorDate: Thu May 11 12:48:45 2023 -0400 prop340: Change the ntorv3 data format Reason behind this is to simplify the negotiation of this feature. A ntorv3 handshake is in the fast path of circuit creation and minimizing any parsing and on the wire binary size is a win. Furthermore, this prevents us to go into a complex code path of version negotiation where either side can be tricked into using another older version. We also do NOT expect to ever end up in a situation where one side sends packed cells but not the other. And so, this change simplifies everything and reflects the approach we took with congestion control as well. Future change of packed/fragmented relay cells will be possible through a new ntorv3 extension and a Relay protover. Signed-off-by: David Goulet <dgoulet@torproject.org> --- proposals/340-packed-and-fragmented.md | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/proposals/340-packed-and-fragmented.md b/proposals/340-packed-and-fragmented.md index 121be68..74af5ca 100644 --- a/proposals/340-packed-and-fragmented.md +++ b/proposals/340-packed-and-fragmented.md @@ -118,17 +118,32 @@ circuit being immediately destroyed. ## Negotiation -This message format requires a new `Relay` subprotocol version to -indicate support. If a client wants to indicate support for this -format, it sends the following extension as part of its `ntor3` -handshake: +This message format requires a new `Relay` subprotocol version to indicate +support. If a client wants to indicate support for this format, it sends the +following extension as part of its `ntor3` handshake: - RELAY_PROTOCOL - version u8 + EXT_FIELD_TYPE: -The `version` field is the `Relay` subprotocol version that the client -wants to use. The relay must send back the same extension in its ntor3 -handshake to acknowledge support. + [03] -- Packed and Fragmented Cell Request + +This field is zero payload length. Its presence signifies that the client +wants to use packed and fragmented cells on the circuit. + +The Exit side ntorv3 response payload is encoded as: + + EXT_FIELD_TYPE: + + [04] -- Packed and Fragmented Cell Response + +Again, the extension presence indicates to the client that the Exit has +acknowledged the feature and is ready to use it. If the extension is not +present, the client MUST not use the packed and fragmented feature even though +the Exit has advertised the correct protover. + +The client MUST reject the handshake and thus close the circuit if: + + - The response extension is seen for a non-ntorv3 handshake. + - The response extension is seen but no request was made initially. ## Migration -- To stop receiving notification emails like this one, please contact the administrator of this repository.
participants (1)
-
gitolite role