- tor-commits - lists.torproject.org

[tor] branch release-0.4.7 updated (e234df6098 -> e98995bb08)
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch release-0.4.7 in repository tor. from e234df6098 version: Bump version to 0.4.7.8-dev new 8d8afc4efa relay: Send DESTROY cell instead of TRUNCATED cell new 5260b4ef34 Merge branch 'maint-0.4.5' into maint-0.4.6 new e98995bb08 Merge branch 'maint-0.4.6' into maint-0.4.7 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: changes/ticket40623 | 4 ++++ src/core/or/command.c | 10 +++++----- 2 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changes/ticket40623 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[tor] branch release-0.4.6 updated (5568961cbf -> 2dee45267b)
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch release-0.4.6 in repository tor. from 5568961cbf Merge branch 'maint-0.4.6' into release-0.4.6 new 8d8afc4efa relay: Send DESTROY cell instead of TRUNCATED cell new 5260b4ef34 Merge branch 'maint-0.4.5' into maint-0.4.6 new 2dee45267b Merge branch 'maint-0.4.6' into release-0.4.6 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: changes/ticket40623 | 4 ++++ src/core/or/command.c | 10 +++++----- 2 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changes/ticket40623 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 1

[tor] branch release-0.4.5 updated (621c922134 -> 611ee0d707)
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch release-0.4.5 in repository tor. from 621c922134 Merge branch 'maint-0.4.5' into release-0.4.5 new 8d8afc4efa relay: Send DESTROY cell instead of TRUNCATED cell new 611ee0d707 Merge branch 'maint-0.4.5' into release-0.4.5 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: changes/ticket40623 | 4 ++++ src/core/or/command.c | 10 +++++----- 2 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changes/ticket40623 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 1

[tor] branch maint-0.4.7 updated (e234df6098 -> e98995bb08)
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch maint-0.4.7 in repository tor. from e234df6098 version: Bump version to 0.4.7.8-dev new 8d8afc4efa relay: Send DESTROY cell instead of TRUNCATED cell new 5260b4ef34 Merge branch 'maint-0.4.5' into maint-0.4.6 new e98995bb08 Merge branch 'maint-0.4.6' into maint-0.4.7 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: changes/ticket40623 | 4 ++++ src/core/or/command.c | 10 +++++----- 2 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changes/ticket40623 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[tor] branch maint-0.4.6 updated (4ba89c0ccc -> 5260b4ef34)
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch maint-0.4.6 in repository tor. from 4ba89c0ccc Merge branch 'maint-0.4.5' into maint-0.4.6 new 8d8afc4efa relay: Send DESTROY cell instead of TRUNCATED cell new 5260b4ef34 Merge branch 'maint-0.4.5' into maint-0.4.6 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: changes/ticket40623 | 4 ++++ src/core/or/command.c | 10 +++++----- 2 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changes/ticket40623 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[tor] branch maint-0.4.5 updated (c213c1b0d1 -> 8d8afc4efa)
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch maint-0.4.5 in repository tor. from c213c1b0d1 Add changes entry to tor!575. new 8d8afc4efa relay: Send DESTROY cell instead of TRUNCATED cell The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: changes/ticket40623 | 4 ++++ src/core/or/command.c | 10 +++++----- 2 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changes/ticket40623 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[tor] branch main updated (ed74c52158 -> 028f5fd6ea)
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch main in repository tor. from ed74c52158 cmux: Remove a log bug that is actually an acceptable race new 8d8afc4efa relay: Send DESTROY cell instead of TRUNCATED cell new 5260b4ef34 Merge branch 'maint-0.4.5' into maint-0.4.6 new e98995bb08 Merge branch 'maint-0.4.6' into maint-0.4.7 new 028f5fd6ea Merge branch 'maint-0.4.7' The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: changes/ticket40623 | 4 ++++ src/core/or/command.c | 10 +++++----- 2 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changes/ticket40623 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 4

[tor] branch main updated: cmux: Remove a log bug that is actually an acceptable race
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. The following commit(s) were added to refs/heads/main by this push: new ed74c52158 cmux: Remove a log bug that is actually an acceptable race ed74c52158 is described below commit ed74c5215862e1464e4f72b9c5995613ea00e9c8 Author: David Goulet <dgoulet(a)torproject.org> AuthorDate: Tue Jul 26 11:45:43 2022 -0400 cmux: Remove a log bug that is actually an acceptable race Closes #40647 Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- changes/ticket40647 | 4 ++++ src/core/or/relay.c | 21 +++++++++++++++++---- 2 files changed, 21 insertions(+), 4 deletions(-) diff --git a/changes/ticket40647 b/changes/ticket40647 new file mode 100644 index 0000000000..ae20aae3f3 --- /dev/null +++ b/changes/ticket40647 @@ -0,0 +1,4 @@ + o Minor bugfixes (relay): + - Remove a "BUG" warning for an acceptable race between a circuit close + and considering that circuit active. Fixes bug 40647; bugfix on + 0.3.5.1-alpha. diff --git a/src/core/or/relay.c b/src/core/or/relay.c index 68fddd1ae7..332878a8ea 100644 --- a/src/core/or/relay.c +++ b/src/core/or/relay.c @@ -3038,10 +3038,23 @@ channel_flush_from_first_active_circuit, (channel_t *chan, int max)) streams_blocked = circ->streams_blocked_on_p_chan; } - /* Circuitmux told us this was active, so it should have cells */ - if (/*BUG(*/ queue->n == 0 /*)*/) { - log_warn(LD_BUG, "Found a supposedly active circuit with no cells " - "to send. Trying to recover."); + /* Circuitmux told us this was active, so it should have cells. + * + * Note: In terms of logic and coherence, this should never happen but the + * cmux dragon is powerful. Reason is that when the OOM is triggered, when + * cleaning up circuits, we mark them for close and then clear their cell + * queues. And so, we can have a circuit considered active by the cmux + * dragon but without cells. The cmux subsystem is only notified of this + * when the circuit is freed which leaves a tiny window between close and + * free to end up here. + * + * We are accepting this as an "ok" race else the changes are likely non + * trivial to make the mark for close to set the num cells to 0 and change + * the free functions to detach the circuit conditionnaly without creating + * a chain effect of madness. + * + * The lesson here is arti will prevail and leave the cmux dragon alone. */ + if (queue->n == 0) { circuitmux_set_num_cells(cmux, circ, 0); if (! circ->marked_for_close) circuit_mark_for_close(circ, END_CIRC_REASON_INTERNAL); -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[tor-browser] branch tor-browser-91.11.0esr-12.0-1 updated (4ff5808811ac9 -> ef0f0ae9f0e56)
by gitolite role 26 Jul '22

26 Jul '22

This is an automated email from the git hooks/post-receive script. pierov pushed a change to branch tor-browser-91.11.0esr-12.0-1 in repository tor-browser. from 4ff5808811ac9 fixup! Bug 31286: Implementation of bridge, proxy, and firewall settings in about:preferences#connection new 16a1af2604d74 fixup! Bug 31286: Implementation of bridge, proxy, and firewall settings in about:preferences#connection new 55991a59b5c28 fixup! Bug 31286: Implementation of bridge, proxy, and firewall settings in about:preferences#connection new a1c57b4b4a76d fixup! Bug 31286: Implementation of bridge, proxy, and firewall settings in about:preferences#connection new ef0f0ae9f0e56 fixup! Bug 31286: Implementation of bridge, proxy, and firewall settings in about:preferences#connection The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../torpreferences/content/builtinBridgeDialog.jsm | 17 ++----- .../torpreferences/content/connectionPane.js | 57 ++++++++++++---------- .../torpreferences/content/provideBridgeDialog.jsm | 15 ++---- .../torpreferences/content/requestBridgeDialog.jsm | 13 ++--- 4 files changed, 44 insertions(+), 58 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 4

[torspec] branch main updated: Add proposal 341-better-oos.md
by gitolite role 25 Jul '22

25 Jul '22

This is an automated email from the git hooks/post-receive script. nickm pushed a commit to branch main in repository torspec. The following commit(s) were added to refs/heads/main by this push: new 6f9ec72 Add proposal 341-better-oos.md 6f9ec72 is described below commit 6f9ec72e279c837a07101b37c8790117cdfaf8a5 Author: Nick Mathewson <nickm(a)torproject.org> AuthorDate: Mon Jul 25 15:04:42 2022 -0400 Add proposal 341-better-oos.md --- proposals/000-index.txt | 2 + proposals/341-better-oos.md | 171 ++++++++++++++++++++++++++++++++++++++++++++ proposals/BY_INDEX.md | 1 + proposals/README.md | 1 + 4 files changed, 175 insertions(+) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 7d518c1..0f82741 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -261,6 +261,7 @@ Proposals by number: 338 Use an 8-byte timestamp in NETINFO cells [ACCEPTED] 339 UDP traffic over Tor [ACCEPTED] 340 Packed and fragmented relay messages [OPEN] +341 A better algorithm for out-of-sockets eviction [OPEN] Proposals by status: @@ -301,6 +302,7 @@ Proposals by status: 326 The "tor-relay" Well-Known Resource Identifier 330 Modernizing authority contact entries 340 Packed and fragmented relay messages + 341 A better algorithm for out-of-sockets eviction ACCEPTED: 265 Load Balancing with Overhead Parameters [for 0.2.9.x] 282 Remove "Named" and "Unnamed" handling from consensus voting [for 0.3.3.x] diff --git a/proposals/341-better-oos.md b/proposals/341-better-oos.md new file mode 100644 index 0000000..0715c14 --- /dev/null +++ b/proposals/341-better-oos.md @@ -0,0 +1,171 @@ +``` +Filename: 341-better-oos.md +Title: A better algorithm for out-of-sockets eviction +Author: Nick Mathewson +Created: 25 July 2022 +Status: Open +``` + +# Introduction + +Our existing algorithm for handling an out-of-sockets condition needs +improvement. It only handles sockets used for OR connections, and +prioritizes those with more circuits. Because of these weaknesses, the +algorithm is trivial to circumvent, and it's disabled by default with +`DisableOOSCheck`. + +Here we propose a new algorithm for choosing which connections to close +when we're out of sockets. In summary, the new algorithm works by +deciding which kinds of connections we have "too many" of, and then by +closing excess connections of each kind. The algorithm for selecting +connections of each kind is different. + + + +# Intuitions behind the algorithm below + +We want to keep a healthy mix of connections running; favoring one kind +of connection over another gives the attacker a fine way to starve the +disfavored connections by making a bunch of the favored kind. + +The correct mix of connections depends on the type of service we are +providing. Everywhere _except_ authorities, for example, inbound +directory connections are perfectly fine to close, since nothing in our +protocol actually generates them. + +In general, we would prefer to close DirPort connections, then Exit +connections, then OR connections. + +The priority with which to close connections is different depending on +the connection type. "Age of connection" or "number of circuits" may be +a fine metric for how truly used an OR connection is, but for a DirPort +connection, high age is suspicious. + +# The algorithm + +Define a "candidate" connection as one that has a socket, and is either +an exit stream, an _inbound_ directory stream, or an OR connection. + +(Note that OR connections can be from clients, relays, or bridges. Note +that ordinary relays should not get directory streams that use sockets, +since clients always use `BEGIN_DIR` to create tunneled directory +streams.) + +In all of the following, treat subtraction as saturating at zero. In +other words, when you see "A - B" below, read it as "MAX(A-B, 0)". + +## Phase 1: Deciding how many connections to close + +When we are find that we are low on sockets, we pick a number of sockets +that we want to close according to our existing algorithm. (That is, we +try to close 1/4 of our maximum sockets if we have reached our upper +limit, or 1/10 of our maximum sockets if we have encountered a failure +from socket(2).) Call this `N_CLOSE`. + +Then we decide which sockets to target based on this algorithm. + +1. Consider the total number of sockets used for exit streams + (`N_EXIT`), the total number used for _inbound_ directory streams + (`N_DIR`), and the total number used for OR connections (`N_OR`). + (In these calculations, we exclude connections that are already + marked to be closed.) Call the total `N_CONN = N_DIR + N_OR + + N_EXIT`. Define `N_RETAIN = N_CONN - N_CLOSE`. + +2. Compute how many connections of each type are "in excess". First, + calculate our target proportions: + + * If we are an authority, let `T_DIR` = 1. Otherwise set `T_DIR = 0.1`. + * If we are an exit or we are running an onion service, let `T_EXIT = + 2`. Otherwise let `T_EXIT = 0.1`. + * Let `T_OR` = 1. + + > TODO: Should those numbers be consensus parameters? + + These numbers define the relative proportions of connections that + we would be willing to retain retain in our final mix. Compute a + number of _excess_ connections of each type by calculating. + + ``` + T_TOTAL = T_OR + T_DIR + T_EXIT. + EXCESS_DIR = N_DIR - N_RETAIN * (T_DIR / T_TOTAL) + EXCESS_EXIT = N_EXIT - N_RETAIN * (T_EXIT / T_TOTAL) + EXCESS_OR = N_OR - N_RETAIN * (T_OR / T_TOTAL) + ``` + +3. Finally, divide N_CLOSE among the different types of excess + connections, assigning first to excess directory connections, then + excess exit connections, and finally to excess OR connections. + + ``` + CLOSE_DIR = MIN(EXCESS_DIR, N_CLOSE) + N_CLOSE := N_CLOSE - CLOSE_DIR + CLOSE_EXIT = MIN(EXCESS_EXIT, N_CLOSE) + N_CLOSE := N_CLOSE - CLOSE_EXIT + CLOSE_OR = MIN(EXCESS_OR, N_CLOSE) + ``` + +We will try to close `CLOSE_DIR` directory connections, `CLOSE_EXIT` +exit connections, and `CLOSE_OR` OR connections. + +## Phase 2: Closing directory connections + +We want to close a certain number of directory connections. To select +our targets, we sort first by the number of directory connections from +a similar address (see "similar address" below) and then by their age, +preferring to close the oldest ones first. + +> This approach defeats "many requests from the same address" and "Open +> a connection and hold it open, and do so from many addresses". It +> doesn't do such a great job with defeating "open and close frequently +> and do so on many addresses." + +> Note that fallback directories do not typically use sockets for +> handling directory connections: theirs are usually created with +> BEGIN_DIR. + +## Phase 3: Closing exit connections. + +We want to close a certain number of exit connections. To do this, we +pick an exit connection at random, then close its circuit _along with +all the other exit connections on the same circuit_. Then we repeat +until we have closed at least our target number of exit connections. + +> This approach probabilistically favors closing circuits with a large +> number of sockets open, regardless of how long those sockets have been +> open. This defeats the easiest way of opening a large number of exit +> streams ("open them all on once circuit") without making the +> counter-approach ("open each exit stream on a its own circuit") much +> more attractive. + +## Phase 3: Closing OR connections. + +We want to close a certain number of OR connections, to clients, bridges, or +relays. + +To do this, we first close OR connections with zero circuits. Then we +close all OR connections but the most recent 2 from each "similar +address". Then we close OR connections at random from among those _not_ +to a recognized relay in the latest directory. Finally, we close OR +connections at random. + +> We used to unconditionally prefer to close connections with fewer +> circuits. That's trivial for an adversary to circumvent, though: they +> can just open a bunch of circuits on their bogus OR connections, and +> force us to preferentially close circuits from real clients, bridges, +> and relays. + +> Note that some connections that seem like client connections ("not +> from relays in the latest directory") are actually those created by +> bridges. + +## What is "A similar address"? + +We define two connections as having a similar address if they are in the +same IPv4 /30, or if they are in the same IPv6 /90. + + +# Acknowledgments + +This proposal was inspired by a set of +[OOS improvements](https://gitlab.torproject.org/tpo/core/tor/-/issues/32794) +from `starlight`. diff --git a/proposals/BY_INDEX.md b/proposals/BY_INDEX.md index 9383351..b64ae55 100644 --- a/proposals/BY_INDEX.md +++ b/proposals/BY_INDEX.md @@ -258,4 +258,5 @@ Below are a list of proposals sorted by their proposal number. See * [`338-netinfo-y2038.md`](/proposals/338-netinfo-y2038.md): Use an 8-byte timestamp in NETINFO cells [ACCEPTED] * [`339-udp-over-tor.md`](/proposals/339-udp-over-tor.md): UDP traffic over Tor [ACCEPTED] * [`340-packed-and-fragmented.md`](/proposals/340-packed-and-fragmented.md): Packed and fragmented relay messages [OPEN] +* [`341-better-oos.md`](/proposals/341-better-oos.md): A better algorithm for out-of-sockets eviction [OPEN] diff --git a/proposals/README.md b/proposals/README.md index 1431e8d..4127329 100644 --- a/proposals/README.md +++ b/proposals/README.md @@ -40,6 +40,7 @@ for discussion. * [`326-tor-relay-well-known-uri-rfc8615.md`](/proposals/326-tor-relay-well-known-uri-rfc8615.md): The "tor-relay" Well-Known Resource Identifier * [`330-authority-contact.md`](/proposals/330-authority-contact.md): Modernizing authority contact entries * [`340-packed-and-fragmented.md`](/proposals/340-packed-and-fragmented.md): Packed and fragmented relay messages +* [`341-better-oos.md`](/proposals/341-better-oos.md): A better algorithm for out-of-sockets eviction ## ACCEPTED proposals: slated for implementation -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0