January 2021 - tor-commits - lists.torproject.org

[tor/release-0.4.5] Remove BUG() when checking TOO_MANY_OUTDATED_DIRSERVERS.
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit 4d6d3b3c056bd886226ba6c55ef00e06b9c86ae6 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Jan 11 15:02:23 2021 -0500 Remove BUG() when checking TOO_MANY_OUTDATED_DIRSERVERS. Fixes bug #40234; bugfix on 0.3.2.5-alpha. --- src/feature/nodelist/microdesc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/feature/nodelist/microdesc.c b/src/feature/nodelist/microdesc.c index dafaabb5e5..90eb938f6d 100644 --- a/src/feature/nodelist/microdesc.c +++ b/src/feature/nodelist/microdesc.c @@ -124,8 +124,9 @@ microdesc_note_outdated_dirserver(const char *relay_digest) tor_assert(outdated_dirserver_list); /* If the list grows too big, clean it up */ - if (BUG(smartlist_len(outdated_dirserver_list) > - TOO_MANY_OUTDATED_DIRSERVERS)) { + if (smartlist_len(outdated_dirserver_list) > TOO_MANY_OUTDATED_DIRSERVERS) { + log_info(LD_GENERAL,"Too many outdated directory servers (%d). Resetting.", + smartlist_len(outdated_dirserver_list)); microdesc_reset_outdated_dirservers_list(); }

1 0

[tor/release-0.4.5] Merge branch 'maint-0.4.5' into release-0.4.5
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit 429d9659ca6699c18f53cae04b5dfece421d1f93 Merge: 21cd621a24 c931eae981 Author: George Kadianakis <desnacked(a)riseup.net> Date: Wed Jan 13 16:01:41 2021 +0200 Merge branch 'maint-0.4.5' into release-0.4.5 src/feature/nodelist/microdesc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)

1 0

[tor/release-0.4.5] Merge branch 'mr/252' into maint-0.4.5
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit c931eae9811a3bba3720f22159f43af8ad3a1b1b Merge: 01be7cc535 4d6d3b3c05 Author: George Kadianakis <desnacked(a)riseup.net> Date: Wed Jan 13 16:01:11 2021 +0200 Merge branch 'mr/252' into maint-0.4.5 src/feature/nodelist/microdesc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)

1 0

[torspec/master] prop328: Initial draft
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit 17df80f84b3425b13c71eb8ab29d23e3a59863d3 Author: David Goulet <dgoulet(a)torproject.org> Date: Tue Nov 3 11:29:38 2020 -0500 prop328: Initial draft Signed-off-by: Mike Perry <mikeprry(a)torproject.org> Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- proposals/000-index.txt | 2 + proposals/328-relay-overload-report.md | 199 +++++++++++++++++++++++++++++++++ proposals/BY_INDEX.md | 1 + proposals/README.md | 1 + 4 files changed, 203 insertions(+) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index f6ff770..6851cff 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -248,6 +248,7 @@ Proposals by number: 325 Packed relay cells: saving space on small commands [OPEN] 326 The "tor-relay" Well-Known Resource Identifier [OPEN] 327 A First Take at PoW Over Introduction Circuits [DRAFT] +328 Make Relays Report When They Are Overloaded [DRAFT] Proposals by status: @@ -257,6 +258,7 @@ Proposals by status: 294 TLS 1.3 Migration 316 FlashFlow: A Secure Speed Test for Tor (Parent Proposal) 327 A First Take at PoW Over Introduction Circuits + 328 Make Relays Report When They Are Overloaded NEEDS-REVISION: 212 Increase Acceptable Consensus Age [for 0.2.4.x+] 219 Support for full DNS and DNSSEC resolution in Tor [for 0.2.5.x] diff --git a/proposals/328-relay-overload-report.md b/proposals/328-relay-overload-report.md new file mode 100644 index 0000000..b05d289 --- /dev/null +++ b/proposals/328-relay-overload-report.md @@ -0,0 +1,199 @@ +``` +Filename: 328-relay-overload-report.md +Title: Make Relays Report When They Are Overloaded +Author: David Goulet, Mike Perry +Created: November 3rd 2020 +Status: Draft +``` + +# 0. Introduction + +Many relays are likely sometimes under heavy load in terms of memory, CPU or +network resources which in turns diminishes their ability to efficiently relay +data through the network. + +Having the capability of learning if a relay is overloaded would allow us to +make better informed load balancing decisions. For instance, we can make our +bandwidth scanners more intelligent on how they allocate bandwidth based on +such metrics from relays. + +We could furthermore improve our network health monitoring and pinpoint relays +possibly misbehaving or under DDoS attack. + +# 1. Metrics to Report + +We propose that relays start collecting several metrics (see section 2) +reflecting their loads from different component of tor. + +Then, we propose that 3 new lines be added to the extra-info document (see +dir-spec.txt, section 2.1.2) if only the overload case arrise. + +This following describes a series of metrics to collect but more might come in +the future and thus this is not an exhaustive list. + +# 1.1. General Overload + +The general overload line indicates that a relay has reached an "overloaded +state" which can be one or many of the following load metrics: + + - Any OOMkiller invocation due to memory pressure + - Any onionskins are dropped + - CPU utilization of Tor's mainloop CPU core above 90% for 60 sec + - TCP port exhaustion + +The format of the overloaded line added in the extra-info document is as +follow: + +``` +"overload-reached" YYYY-MM-DD HH:MM:SS NL + [At most once.] +``` + +The timestamp is when a at least one metrics was detected. It should always be +at the hour and thus, as an example, "2020-01-10 13:00:00" is an expected +timestamp. Because this is a binary state, if the line is present, we consider +that it was hit at the very least once somewhere between the provided +timestamp and the "published" timestamp of the document which is when the +document was generated. + +The overload field should remain in place for 72 hours since last triggered. +If the limits are reached again in this period, the timestamp is updated, and +this 72 hour period restarts. + +# 1.2. Token bucket size + +Relays should report the 'BandwidthBurst' and 'BandwidthRate' limits in their +descriptor, as well as the number of times these limits were reached, for read +and write, in the past 24 hours starting at the provided timestamp rounded +down to the hour. + +``` +"overload-ratelimits" SP YYYY-MM-DD SP HH:MM:SS + SP rate-limit SP burst-limit + SP read-rate-count SP read-burst-count + SP write-rate-count SP write-burst-count NL + [At most once.] +``` + +The "rate-limit" and "burst-limit" are the raw values from the BandwidthRate +and BandwidthBurst found in the torrc configuration file. + +The "{read|write}-rate-count" and "{read|write}-burst-count" are the counts of +how many times the reported limits were exhausted and thus the maximum between +the read and write count occurances. + +# 1.3. File Descriptor Exhaustion + +Not having enough file descriptors in this day of age is really a +misconfiguration or a too old operation system. That way, we can very quickly +notice which relay has a value too small and we can notify them. + +This should be published in this format: + +``` +"overload-fd-exhausted" YYYY-MM-DD HH:MM:SS NL + [At most once.] +``` + +As the overloaded line, the timestamp indicates that the maximum was reached +between the this timestamp and the "published" timestamp of the document. + +This overload field should remain in place for 72 hours since last triggered. +If the limits are reached again in this period, the timestamp is updated, and +this 72 hour period restarts. + +# 2. Load Metrics + +This section proposes a series of metrics that should be collected and +reported to the MetricsPort. The Prometheus format (only one supported for +now) is described for each metrics but each of them are prefixed with the +following in order to have a proper namespace for "load" events: + +`tor_load_` + +## 2.1 Out-Of-Memory (OOM) Invocation + +Tor's OOM manages caches and queues of all sorts. Relays have many of them and +so any invocation of the OOM should be reported. + +``` +# HELP Total number of bytes the OOM has cleaned up +# TYPE counter +tor_load_oom_bytes_total{<LABEL>} <VALUE> +``` + +Running counter of how many bytes were cleaned up by the OOM for a tor +component identified by a label (see list below). To make sense, this should +be visualized with the rate() function. + +Possible LABELs for which the OOM was triggered: + - `cell`: Circuit cell queue + - `dns`: DNS resolution cache + - `geoip`: GeoIP cache + - `hsdir`: Onion service descriptors + +## 2.2 Onionskin Queues + +Onionskins handling is one of the few items that tor processes in parallel but +they can be dropped for various reasons when under load. For this metrics to +make sense, we also need to gather how many onionskins are we processing and +thus one can provide a total processed versus dropped ratio: + +``` +# HELP Total number of onionskins +# TYPE counter +tor_load_onionskin_total{<LABEL>} <NUM> +``` + +Possible LABELs are: + - `processed`: Indicating how many were processed. + - `dropped`: Indicating how many were dropped due to load. + +## 2.3 File Descriptor Exhaustion + +Relays can reach a "ulimit" (on Linux) cap that is the number of allowed +opened file descriptors. In Tor's use case, this is mostly sockets. File +descriptors should be reported as follow: + +``` +# HELP Total number of file descriptors +# TYPE gauge +tor_load_fd_total{<LABEL>} <NUM> +``` + +Possible LABELs are: + - `remaining`: How many file descriptors remains that is can be opened. + +Note: since tor does track that value in order to reserve a block for critical +port such as the Control Port, that value can easily be exported. + +## 2.4 TCP Port Exhaustion + +TCP protocol is capped at 65535 ports and thus if the relay ever is unable to +open more outbound sockets, that is an overloaded state. It should be +reported: + +``` +# HELP Total number of opened outbound connections. +# TYPE gauge +tor_load_socket_total{<LABEL>} <NUM> +``` + +Possible LABELs are: + - `outbound`: Sockets used for outbound connections. + +## 2.5 Connection Bucket Limit + +Rate limited connections track bandwidth using a bucket system. Once the +bucket is filled and tor wants to send more, it pauses until it is refilled a +second later. Once that is hit, it should be reported: + +``` +# HELP Total number of global connection bucket limit reached +# TYPE counter +tor_load_global_rate_limit_reached_total{<LABEL>} <NUM> +``` + +Possible LABELs are: + - `read`: Read side of the global rate limit bucket. + - `write`: Write side of the global rate limit bucket. diff --git a/proposals/BY_INDEX.md b/proposals/BY_INDEX.md index 90a151d..3d80c1c 100644 --- a/proposals/BY_INDEX.md +++ b/proposals/BY_INDEX.md @@ -245,4 +245,5 @@ Below are a list of proposals sorted by their proposal number. See * [`325-packed-relay-cells.md`](/proposals/325-packed-relay-cells.md): Packed relay cells: saving space on small commands [OPEN] * [`326-tor-relay-well-known-uri-rfc8615.md`](/proposals/326-tor-relay-well-known-uri-rfc8615.md): The "tor-relay" Well-Known Resource Identifier [OPEN] * [`327-pow-over-intro.txt`](/proposals/327-pow-over-intro.txt): A First Take at PoW Over Introduction Circuits [DRAFT] +* [`328-relay-overload-report.md`](/proposals/328-relay-overload-report.md): Make Relays Report When They Are Overloaded [DRAFT] diff --git a/proposals/README.md b/proposals/README.md index 904009e..44996ad 100644 --- a/proposals/README.md +++ b/proposals/README.md @@ -108,6 +108,7 @@ discussion. * [`294-tls-1.3.txt`](/proposals/294-tls-1.3.txt): TLS 1.3 Migration * [`316-flashflow.md`](/proposals/316-flashflow.md): FlashFlow: A Secure Speed Test for Tor (Parent Proposal) * [`327-pow-over-intro.txt`](/proposals/327-pow-over-intro.txt): A First Take at PoW Over Introduction Circuits +* [`328-relay-overload-report.md`](/proposals/328-relay-overload-report.md): Make Relays Report When They Are Overloaded ## NEEDS-REVISION proposals: ideas that we can't implement as-is

1 0

[torspec/master] Merge remote-tracking branch 'tor-gitlab/mr/22'
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit 3540c8f32227bbe27231a435e240de62d07952f3 Merge: 6a8e540 17df80f Author: George Kadianakis <desnacked(a)riseup.net> Date: Wed Jan 13 15:40:32 2021 +0200 Merge remote-tracking branch 'tor-gitlab/mr/22' proposals/000-index.txt | 2 + proposals/328-relay-overload-report.md | 199 +++++++++++++++++++++++++++++++++ proposals/BY_INDEX.md | 1 + proposals/README.md | 1 + 4 files changed, 203 insertions(+)

1 0

[torspec/master] dir-spec: Remove the notion of a "current" location in our ed diffs
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit 839348cc65dc3bb4caa06ea4a8f3ed3c25d184b7 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 17 15:40:51 2020 -0500 dir-spec: Remove the notion of a "current" location in our ed diffs Tor has never generated or implemented this diff command. Found while working on arti. --- dir-spec.txt | 7 ------- 1 file changed, 7 deletions(-) diff --git a/dir-spec.txt b/dir-spec.txt index 148163d..cdbe00b 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -4049,7 +4049,6 @@ E. Limited ed diff format - "<n1>,<n2>c" Replace lines n1 through n2, inclusive, with the following block. - "<n1>a" Append the following block after line n1. - - "a" Append the following block after the current line. Note that line numbers always apply to the file after all previous commands have already been applied. Note also that line numbers @@ -4065,12 +4064,6 @@ E. Limited ed diff format successfully apply the diff even if they have an unusual encoding for the signatures. - The "current line" is either the first line of the file, if this is - the first command, the last line of a block we added in an append or - change command, or the line immediate following a set of lines we just - deleted (or the last line of the file if there are no lines after - that). - The replace and append command take blocks. These blocks are simply appended to the diff after the line with the command. A line with just a period (".") ends the block (and is not part of the lines

1 0

[torspec/master] Merge remote-tracking branch 'tor-gitlab/mr/24'
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit 6a8e5400bd797415df559897297b7409e9322721 Merge: c2ea456 839348c Author: George Kadianakis <desnacked(a)riseup.net> Date: Wed Jan 13 15:38:52 2021 +0200 Merge remote-tracking branch 'tor-gitlab/mr/24' dir-spec.txt | 7 ------- 1 file changed, 7 deletions(-)

1 0

[torspec/master] Merge remote-tracking branch 'tor-gitlab/mr/23'
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit c2ea45638af748d0a2d31eaf037d0b38c758595f Merge: 12271f0 a28dc71 Author: George Kadianakis <desnacked(a)riseup.net> Date: Wed Jan 13 15:27:08 2021 +0200 Merge remote-tracking branch 'tor-gitlab/mr/23' dir-spec.txt | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)

1 0

[torspec/master] Document consensus method 31 as a fix for bug #19011.
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit a28dc7176d4e70313d45361e3e1f9d3999098a75 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Dec 9 13:18:43 2020 -0500 Document consensus method 31 as a fix for bug #19011. --- dir-spec.txt | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/dir-spec.txt b/dir-spec.txt index 148163d..668ff32 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2805,6 +2805,10 @@ * If consensus method 30 or later is used, the base64 encoded ntor-onion-key does not include the trailing = sign. + * If consensus method 31 or later is used, authorities parse the + "bwweightscale" and "maxunmeasuredbw" parameters correctly when + computing votes. + The signatures at the end of a consensus document are sorted in ascending order by identity digest. @@ -2891,6 +2895,8 @@ "28" -- Removes "a" lines from microdescriptors. "29" -- Canonicalizes families in microdescriptors. "30" -- Removes padding from ntor-onion-key. + "31" -- Uses correct parsing for bwweightscale and maxunmeasuredbw + when computing weights Before generating a consensus, an authority must decide which consensus method to use. To do this, it looks for the highest version number @@ -2938,7 +2944,11 @@ 3.8.3. Computing Bandwidth Weights - Let weight_scale = 10000 + Let weight_scale = 10000, or the value of the "bwweightscale" parameter. + (Before consensus method 31 there was a bug in parsing bwweightscale, so + that if there were any consensus parameters after it alphabetically, it + would always be treated as 10000. A similar bug existed for + "maxunmeasuredbw".) Starting with consensus method 26, G, M, E, and D are initialized to 1 and T to 4. Prior consensus methods initialize them all to 0. With this change, @@ -3091,7 +3101,8 @@ To ensure consensus, all calculations are performed using integer math with a fixed precision determined by the bwweightscale consensus - parameter (defaults at 10000, Min: 1, Max: INT32_MAX). + parameter (defaults at 10000, Min: 1, Max: INT32_MAX). (See note above + about parsing bug in bwweightscale before consensus method 31.) For future balancing improvements, Tor clients support 11 additional weights for directory requests and middle weighting. These weights are currently

1 0

[tor/master] Merge branch 'mr/236'
by asn＠torproject.org 13 Jan '21

13 Jan '21

commit cc30c09f7c92fff7bf9e5621d31b2f9538ec0466 Merge: 1949afef98 fb3704b459 Author: George Kadianakis <desnacked(a)riseup.net> Date: Wed Jan 13 15:23:54 2021 +0200 Merge branch 'mr/236' changes/bug19011 | 7 +++ src/feature/dirauth/dirvote.c | 115 +++++++++++++++++++++++------------------- src/feature/dirauth/dirvote.h | 10 +++- src/test/test_dirvote.c | 25 +++++++++ 4 files changed, 105 insertions(+), 52 deletions(-)

1 0