July 2020 - tor-commits - lists.torproject.org

[torspec/master] Add proposal 325-packed-relay-cells.md
by nickm＠torproject.org 10 Jul '20

10 Jul '20

commit 3997e7a8a5a6ff0cf5b7cdce5f850a59abc12162 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 10 13:51:21 2020 -0400 Add proposal 325-packed-relay-cells.md --- proposals/000-index.txt | 2 + proposals/325-packed-relay-cells.md | 145 ++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 3ddeccd..86f2a16 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -245,6 +245,7 @@ Proposals by number: 322 Extending link specifiers to include the directory port [OPEN] 323 Specification for Walking Onions [DRAFT] 324 RTT-based Congestion Control for Tor [OPEN] +325 Packed relay cells: saving space on small commands [DRAFT] Proposals by status: @@ -263,6 +264,7 @@ Proposals by status: 313 Tor Relay IPv6 Statistics 316 FlashFlow: A Secure Speed Test for Tor (Parent Proposal) 323 Specification for Walking Onions + 325 Packed relay cells: saving space on small commands NEEDS-REVISION: 212 Increase Acceptable Consensus Age [for 0.2.4.x+] 219 Support for full DNS and DNSSEC resolution in Tor [for 0.2.5.x] diff --git a/proposals/325-packed-relay-cells.md b/proposals/325-packed-relay-cells.md new file mode 100644 index 0000000..455e7a2 --- /dev/null +++ b/proposals/325-packed-relay-cells.md @@ -0,0 +1,145 @@ +``` +Filename: 325-packed-relay-cells.md +Title: Packed relay cells: saving space on small commands +Author: Nick Mathewson +Created: 10 July 2020 +Status: Draft +``` + +# Introduction + +In proposal 319 I suggested a way to fragment long commands across +multiple RELAY cells. In this proposal, I suggest a new format for +RELAY cells that can be used to pack multiple relay commands into a +single cell. + +Why would we want to do this? As we move towards improved +congestion-control and flow-control algorithms, we might not want to +use an entire 498-byte relay payload just to send a one-byte +flow-control message. + +We already have some cases where we'd benefit from this feature. +For example, when we send SENDME messages, END cells, or BEGIN_DIR +cells, most of the cell body is wasted with padding. + +As a side benefit, packing cells in this way may make the job of the +traffic analyst a little more tricky, as cell contents become less +predictable. + +# The basic design + +Let's use the term "Relay Message" to mean the kind of thing that a +relay cell used to hold. Thus, this proposal is about packing +multiple "Relay Messages" in to a cell. + +I'll use "Packed relay cell" to mean a relay cell in this new +format, that supports multiple messages. + +I'll use "client" to mean the initiator of a circuit, and "relay" to +refer to the parties through who a circuit is created. Note that +each "relay" (as used here) may be the "client" on circuits of its own. + +When a relay supports relay message packing, it advertises the fact +using a new Relay protocol version. Clients must opt-in to using +this protocol version (see XXX below) before they can send any +packed relay cells, and before the relay will send them any packed +relay cells. + +When packed cells are in use, multiple cell messages can be +concatenated in a single relay cell. + +Only some relay commands are supported for relay cell packing, +listed here: + - SENDME + - DROP + - DATA + - BEGIN + - BEGIN_DIR + - END + - CONNECTED + - PADDING_NEGOTIATE + - PADDING_NEGOTIATED + +If any relay message with a relay command _not_ listed above appears +in a packed relay cell with another relay message, then the +receiving party MUST tear down the circuit. + +(Note that relay cell fragments (proposal 319) are not supported for +packing.) + +The command byte "0" is now used to explicitly indicate "end of +cell". If the byte "0" appears after a relay message, the rest of +the cell MUST be ignored. + +When generating RELAY cells, implementations SHOULD (as they do +today) fill in the unused bytes with four 0-valued bytes, followed by +a sequence of random bytes up to the end of the cell. If there are +fewer than 4 unused bytes at the end of the cell, those unused bytes +should all be filled with 0-valued bytes. + +# Negotiation and migration + +After receiving a packed relay cell, the relay know that the client +supports this proposal: Relays SHOULD send packed relay +cells on any circuit on which they have received a packed relay +cell. Relays MUST NOT send packed relay cells otherwise. + +Clients, in turn, MAY send packed relay cells to any relay whose +"Relay" subprotocol version indicates that it supports this +protocol. To avoid fingerprinting, this client behavior should +controlled with a tristate (1/0/auto) torrc configuration value, +with the default set to use a consensus parameter. + +The parameter is: + + "relay-cell-packing" + + Boolean: if 1, clients should send packed relay cells. + (Min: 0, Max 1, Default: 0) + +To handle migration, first the parameter should be set to 0 and the +configuration setting should be "auto". To test the feature, individual +clients can set the tristate to "1". + +Once enough clients have support for the parameter, the parameter +can be set to 1. + + +# A new relay message format + +(This section is optional and should be considered separately; we +may decide it is too complex.) + +Currently, every relay message uses 5 bytes of header to hold a +relay command, a length field, and a stream ID. This is wasteful: +the stream ID is often redundant, and the top 7 bits of the length +field are always zero. + +I propose a new relay message format, described here (with `ux` +denoting an x-bit bitfield). This format is 2 bytes or 4 bytes, +depending on its first bit. + + struct relay_header { + u1 stream_id_included; // Is the stream_id included? + u6 relay_command; // as before + u9 relay_data_len; // as before + u8 optional_stream_id[]; // 0 bytes or two bytes. + } + +Alternatively, you can view the first three fields as a 16-bit +value, computed as: + + (stream_id_included<<15) | (relay_command << 9) | (relay_data_len). + +If the optional_stream_id field is not present, then the default +value for the stream_id is computed as follows. We use stream_id 0 +for any command that doesn't take a stream ID. For commands that +_do_ take a steam_id, we use whichever nonzero stream_id appeared +last in this cell. + +This format limits the space of possible relay commands. That's +probably okay: after 20 years of Tor development, we have defined 25 +relay command values. But in case 2^6==64 commands will not be +enough, we reserve command values 48 through 63 for future formats +that need more command bits. +

1 0

[tor/master] Add an extra comment about keeping rust and c protover in sync.
by nickm＠torproject.org 10 Jul '20

10 Jul '20

commit 8ebbf629404a97b6110c1916d554009dc8f42c5a Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 10 13:32:25 2020 -0400 Add an extra comment about keeping rust and c protover in sync. --- src/core/or/protover.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/core/or/protover.c b/src/core/or/protover.c index c6b024369..1ac264925 100644 --- a/src/core/or/protover.c +++ b/src/core/or/protover.c @@ -393,6 +393,11 @@ protocol_list_supports_protocol_or_later(const char *list, const char * protover_get_supported_protocols(void) { + /* WARNING! + * + * Remember to edit the SUPPORTED_PROTOCOLS list in protover.rs if you + * are editing this list. + */ return "Cons=1-2 " "Desc=1-2 "

1 0

[tor/master] Merge remote-tracking branch 'tor-gitlab/mr/42'
by nickm＠torproject.org 10 Jul '20

10 Jul '20

commit 6a647be54a8e88f94461f82e16b78a9113a1c351 Merge: 0c2bb9eac 401b3a8a3 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 10 13:20:14 2020 -0400 Merge remote-tracking branch 'tor-gitlab/mr/42' src/app/config/resolve_addr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

1 0

[tor/master] addr: Remove BUG() that always triggers
by nickm＠torproject.org 10 Jul '20

10 Jul '20

commit 401b3a8a3c823c6d0642cc323464e05a140f0351 Author: David Goulet <dgoulet(a)torproject.org> Date: Fri Jul 10 13:09:46 2020 -0400 addr: Remove BUG() that always triggers Fix on unreleased code. Logical || in the BUG() made it that it would always trigger the BUG(). Fixes #40034 Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- src/app/config/resolve_addr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/app/config/resolve_addr.c b/src/app/config/resolve_addr.c index 172f69854..7ec5ae565 100644 --- a/src/app/config/resolve_addr.c +++ b/src/app/config/resolve_addr.c @@ -85,7 +85,7 @@ resolved_addr_get_suggested(int family, tor_addr_t *addr_out) void resolved_addr_set_suggested(const tor_addr_t *addr) { - if (BUG(tor_addr_family(addr) != AF_INET || + if (BUG(tor_addr_family(addr) != AF_INET && tor_addr_family(addr) != AF_INET6)) { return; }

1 0

[tor/master] Merge remote-tracking branch 'tor-gitlab/mr/41'
by nickm＠torproject.org 10 Jul '20

10 Jul '20

commit 0c2bb9eac576d899b2ff8a8fa1be91762b4ee1be Merge: 063cdb314 46b86b22e Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 10 13:19:28 2020 -0400 Merge remote-tracking branch 'tor-gitlab/mr/41' src/feature/relay/relay_find_addr.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)

1 0

[tor/master] addr: Remove a BUG() that can normally occur
by nickm＠torproject.org 10 Jul '20

10 Jul '20

commit 46b86b22e3e554da6dcef77c00b622507f6c4b42 Author: David Goulet <dgoulet(a)torproject.org> Date: Fri Jul 10 13:06:20 2020 -0400 addr: Remove a BUG() that can normally occur Fix on unreleased code. The relay_new_address_suggestion() is called when a NETINFO cell is received thus not only for relay or bridges. Remove the BUG() that made sure only in server mode we could handle the suggested address. Fixes #40032 Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- src/feature/relay/relay_find_addr.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/feature/relay/relay_find_addr.c b/src/feature/relay/relay_find_addr.c index 28b5985bb..16d0a4733 100644 --- a/src/feature/relay/relay_find_addr.c +++ b/src/feature/relay/relay_find_addr.c @@ -64,8 +64,9 @@ relay_address_new_suggestion(const tor_addr_t *suggested_addr, tor_assert(peer_addr); tor_assert(identity_digest); - /* This should never be called on a non Tor relay. */ - if (BUG(!server_mode(options))) { + /* Non server should just ignore this suggestion. Clients don't need to + * learn their address let alone cache it. */ + if (!server_mode(options)) { return; }

1 0

[tor/master] connstats: add and clarify some documentation.
by dgoulet＠torproject.org 10 Jul '20

10 Jul '20

commit 515cc49cb7fce62db8256f2729e4473015574d1a Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 10 09:17:18 2020 -0400 connstats: add and clarify some documentation. --- src/feature/stats/connstats.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/src/feature/stats/connstats.c b/src/feature/stats/connstats.c index 32987ecf9..82b1adcd9 100644 --- a/src/feature/stats/connstats.c +++ b/src/feature/stats/connstats.c @@ -8,8 +8,12 @@ * @file connstats.c * @brief Count bidirectional vs one-way connections. * - * Connection statistics, used by relays to count connections, and track - * one-way and bidirectional connections. + * Connection statistics, use to track one-way and bidirectional connections. + * + * Note that this code counts concurrent connections in each + * BIDI_INTERVAL-second interval, not total connections. It can tell you what + * fraction of connections are bidirectional at each time, not necessarily + * what number are bidirectional. **/ #include "orconfig.h" @@ -28,16 +32,16 @@ conn_stats_init(time_t now) start_of_conn_stats_interval = now; } -/* Count connections that we read and wrote less than these many bytes - * from/to as below threshold. */ +/** Count connections on which we read and wrote less than this many bytes + * as "below threshold." */ #define BIDI_THRESHOLD 20480 -/* Count connections that we read or wrote at least this factor as many +/** Count connections that we read or wrote at least this factor as many * bytes from/to than we wrote or read to/from as mostly reading or * writing. */ #define BIDI_FACTOR 10 -/* Interval length in seconds for considering read and written bytes for +/** Interval length in seconds for considering read and written bytes for * connection stats. */ #define BIDI_INTERVAL 10 @@ -75,13 +79,14 @@ typedef struct bidi_map_entry_t { static HT_HEAD(bidimap, bidi_map_entry_t) bidi_map = HT_INITIALIZER(); +/** Hashtable helper: return true if @a a and @a b have the same key. */ static int bidi_map_ent_eq(const bidi_map_entry_t *a, const bidi_map_entry_t *b) { return a->conn_id == b->conn_id; } -/* DOCDOC bidi_map_ent_hash */ +/** Hashtable helper: compute a digest for the key of @a entry. */ static unsigned bidi_map_ent_hash(const bidi_map_entry_t *entry) {

1 0

[tor/master] Change connstats.c identifers to start with conn_stats
by dgoulet＠torproject.org 10 Jul '20

10 Jul '20

commit 3f2de0bcca301785c5f1911354f2d3f4d5ba1708 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 10 09:11:12 2020 -0400 Change connstats.c identifers to start with conn_stats This is an automated commit, generated by this command: ./scripts/maint/rename_c_identifier.py \ rep_hist_conn_stats_init conn_stats_init \ rep_hist_note_or_conn_bytes conn_stats_note_or_conn_bytes \ rep_hist_reset_conn_stats conn_stats_reset \ rep_hist_format_conn_stats conn_stats_format \ rep_hist_conn_stats_write conn_stats_save \ rep_hist_conn_stats_term conn_stats_terminate \ bidi_map_free_all conn_stats_free_all --- src/core/mainloop/connection.c | 2 +- src/core/mainloop/mainloop.c | 2 +- src/feature/relay/relay_config.c | 4 ++-- src/feature/stats/connstats.c | 24 ++++++++++++------------ src/feature/stats/connstats.h | 14 +++++++------- src/feature/stats/rephist.c | 2 +- src/test/test_stats.c | 36 ++++++++++++++++++------------------ 7 files changed, 42 insertions(+), 42 deletions(-) diff --git a/src/core/mainloop/connection.c b/src/core/mainloop/connection.c index 2d6f02fc6..0d2e6a220 100644 --- a/src/core/mainloop/connection.c +++ b/src/core/mainloop/connection.c @@ -3364,7 +3364,7 @@ record_num_bytes_transferred_impl(connection_t *conn, return; if (conn->type == CONN_TYPE_OR) - rep_hist_note_or_conn_bytes(conn->global_identifier, num_read, + conn_stats_note_or_conn_bytes(conn->global_identifier, num_read, num_written, now); const bool is_ipv6 = (conn->socket_family == AF_INET6); diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index edea036fa..3bf9be566 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -1951,7 +1951,7 @@ write_stats_file_callback(time_t now, const or_options_t *options) next_time_to_write_stats_files = next_write; } if (options->ConnDirectionStatistics) { - time_t next_write = rep_hist_conn_stats_write(now); + time_t next_write = conn_stats_save(now); if (next_write && next_write < next_time_to_write_stats_files) next_time_to_write_stats_files = next_write; } diff --git a/src/feature/relay/relay_config.c b/src/feature/relay/relay_config.c index 4ba83d4bc..7cb7f2ccf 100644 --- a/src/feature/relay/relay_config.c +++ b/src/feature/relay/relay_config.c @@ -1308,7 +1308,7 @@ options_act_relay_stats(const or_options_t *old_options, } if ((!old_options || !old_options->ConnDirectionStatistics) && options->ConnDirectionStatistics) { - rep_hist_conn_stats_init(now); + conn_stats_init(now); } if ((!old_options || !old_options->HiddenServiceStatistics) && options->HiddenServiceStatistics) { @@ -1338,7 +1338,7 @@ options_act_relay_stats(const or_options_t *old_options, rep_hist_exit_stats_term(); if (old_options && old_options->ConnDirectionStatistics && !options->ConnDirectionStatistics) - rep_hist_conn_stats_term(); + conn_stats_terminate(); return 0; } diff --git a/src/feature/stats/connstats.c b/src/feature/stats/connstats.c index f8df26ecc..32987ecf9 100644 --- a/src/feature/stats/connstats.c +++ b/src/feature/stats/connstats.c @@ -23,7 +23,7 @@ static time_t start_of_conn_stats_interval; /** Initialize connection stats. */ void -rep_hist_conn_stats_init(time_t now) +conn_stats_init(time_t now) { start_of_conn_stats_interval = now; } @@ -95,7 +95,7 @@ HT_GENERATE2(bidimap, bidi_map_entry_t, node, bidi_map_ent_hash, /** Release all storage held in connstats.c */ void -bidi_map_free_all(void) +conn_stats_free_all(void) { bidi_map_entry_t **ptr, **next, *ent; for (ptr = HT_START(bidimap, &bidi_map); ptr; ptr = next) { @@ -108,22 +108,22 @@ bidi_map_free_all(void) /** Reset counters for conn statistics. */ void -rep_hist_reset_conn_stats(time_t now) +conn_stats_reset(time_t now) { start_of_conn_stats_interval = now; below_threshold = 0; mostly_read = 0; mostly_written = 0; both_read_and_written = 0; - bidi_map_free_all(); + conn_stats_free_all(); } /** Stop collecting connection stats in a way that we can re-start doing - * so in rep_hist_conn_stats_init(). */ + * so in conn_stats_init(). */ void -rep_hist_conn_stats_term(void) +conn_stats_terminate(void) { - rep_hist_reset_conn_stats(0); + conn_stats_reset(0); } /** We read num_read bytes and wrote num_written from/to OR @@ -131,7 +131,7 @@ rep_hist_conn_stats_term(void) * observation in a new interval, sum up the last observations. Add bytes * for this connection. */ void -rep_hist_note_or_conn_bytes(uint64_t conn_id, size_t num_read, +conn_stats_note_or_conn_bytes(uint64_t conn_id, size_t num_read, size_t num_written, time_t when) { if (!start_of_conn_stats_interval) @@ -184,7 +184,7 @@ rep_hist_note_or_conn_bytes(uint64_t conn_id, size_t num_read, * until now, or NULL if we're not collecting conn stats. Caller must * ensure start_of_conn_stats_interval is in the past. */ char * -rep_hist_format_conn_stats(time_t now) +conn_stats_format(time_t now) { char *result, written[ISO_TIME_LEN+1]; @@ -209,7 +209,7 @@ rep_hist_format_conn_stats(time_t now) * overwriting an existing file) and reset counters. Return when we would * next want to write conn stats or 0 if we never want to write. */ time_t -rep_hist_conn_stats_write(time_t now) +conn_stats_save(time_t now) { char *str = NULL; @@ -219,10 +219,10 @@ rep_hist_conn_stats_write(time_t now) goto done; /* Not ready to write */ /* Generate history string. */ - str = rep_hist_format_conn_stats(now); + str = conn_stats_format(now); /* Reset counters. */ - rep_hist_reset_conn_stats(now); + conn_stats_reset(now); /* Try to write to disk. */ if (!check_or_create_data_subdir("stats")) { diff --git a/src/feature/stats/connstats.h b/src/feature/stats/connstats.h index 2abe8d736..26ff74cfd 100644 --- a/src/feature/stats/connstats.h +++ b/src/feature/stats/connstats.h @@ -12,13 +12,13 @@ #ifndef TOR_FEATURE_STATS_CONNSTATS_H #define TOR_FEATURE_STATS_CONNSTATS_H -void rep_hist_conn_stats_init(time_t now); -void rep_hist_note_or_conn_bytes(uint64_t conn_id, size_t num_read, +void conn_stats_init(time_t now); +void conn_stats_note_or_conn_bytes(uint64_t conn_id, size_t num_read, size_t num_written, time_t when); -void rep_hist_reset_conn_stats(time_t now); -char *rep_hist_format_conn_stats(time_t now); -time_t rep_hist_conn_stats_write(time_t now); -void rep_hist_conn_stats_term(void); -void bidi_map_free_all(void); +void conn_stats_reset(time_t now); +char *conn_stats_format(time_t now); +time_t conn_stats_save(time_t now); +void conn_stats_terminate(void); +void conn_stats_free_all(void); #endif /* !defined(TOR_FEATURE_STATS_CONNSTATS_H) */ diff --git a/src/feature/stats/rephist.c b/src/feature/stats/rephist.c index 5535a3ab0..b6730e122 100644 --- a/src/feature/stats/rephist.c +++ b/src/feature/stats/rephist.c @@ -2141,7 +2141,7 @@ rep_hist_free_all(void) tor_free(exit_bytes_written); tor_free(exit_streams); predicted_ports_free_all(); - bidi_map_free_all(); + conn_stats_free_all(); if (circuits_for_buffer_stats) { SMARTLIST_FOREACH(circuits_for_buffer_stats, circ_buffer_stats_t *, s, diff --git a/src/test/test_stats.c b/src/test/test_stats.c index c8a76e27d..288757229 100644 --- a/src/test/test_stats.c +++ b/src/test/test_stats.c @@ -112,36 +112,36 @@ test_stats(void *arg) /* Continue with testing connection statistics; we shouldn't collect * conn stats without initializing them. */ - rep_hist_note_or_conn_bytes(1, 20, 400, now); - s = rep_hist_format_conn_stats(now + 86400); + conn_stats_note_or_conn_bytes(1, 20, 400, now); + s = conn_stats_format(now + 86400); tt_ptr_op(s, OP_EQ, NULL); /* Initialize stats, note bytes, and generate history string. */ - rep_hist_conn_stats_init(now); - rep_hist_note_or_conn_bytes(1, 30000, 400000, now); - rep_hist_note_or_conn_bytes(1, 30000, 400000, now + 5); - rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 10); - rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 15); - s = rep_hist_format_conn_stats(now + 86400); + conn_stats_init(now); + conn_stats_note_or_conn_bytes(1, 30000, 400000, now); + conn_stats_note_or_conn_bytes(1, 30000, 400000, now + 5); + conn_stats_note_or_conn_bytes(2, 400000, 30000, now + 10); + conn_stats_note_or_conn_bytes(2, 400000, 30000, now + 15); + s = conn_stats_format(now + 86400); tt_str_op("conn-bi-direct 2010-08-12 13:27:30 (86400 s) 0,0,1,0\n",OP_EQ, s); tor_free(s); /* Stop collecting stats, add some bytes, and ensure we don't generate * a history string. */ - rep_hist_conn_stats_term(); - rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 15); - s = rep_hist_format_conn_stats(now + 86400); + conn_stats_terminate(); + conn_stats_note_or_conn_bytes(2, 400000, 30000, now + 15); + s = conn_stats_format(now + 86400); tt_ptr_op(s, OP_EQ, NULL); /* Re-start stats, add some bytes, reset stats, and see what history we * get when observing no bytes at all. */ - rep_hist_conn_stats_init(now); - rep_hist_note_or_conn_bytes(1, 30000, 400000, now); - rep_hist_note_or_conn_bytes(1, 30000, 400000, now + 5); - rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 10); - rep_hist_note_or_conn_bytes(2, 400000, 30000, now + 15); - rep_hist_reset_conn_stats(now); - s = rep_hist_format_conn_stats(now + 86400); + conn_stats_init(now); + conn_stats_note_or_conn_bytes(1, 30000, 400000, now); + conn_stats_note_or_conn_bytes(1, 30000, 400000, now + 5); + conn_stats_note_or_conn_bytes(2, 400000, 30000, now + 10); + conn_stats_note_or_conn_bytes(2, 400000, 30000, now + 15); + conn_stats_reset(now); + s = conn_stats_format(now + 86400); tt_str_op("conn-bi-direct 2010-08-12 13:27:30 (86400 s) 0,0,0,0\n",OP_EQ, s); tor_free(s);

1 0

[tor/master] connstats: extract functions for summarizing a connection's status
by dgoulet＠torproject.org 10 Jul '20

10 Jul '20

commit 54141d66e27a47719fe81a886d5e92d2b730f374 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 10 09:36:07 2020 -0400 connstats: extract functions for summarizing a connection's status --- src/feature/stats/connstats.c | 58 +++++++++++++++++++++++++++++-------------- 1 file changed, 39 insertions(+), 19 deletions(-) diff --git a/src/feature/stats/connstats.c b/src/feature/stats/connstats.c index 2879a3018..7d7c67361 100644 --- a/src/feature/stats/connstats.c +++ b/src/feature/stats/connstats.c @@ -134,6 +134,44 @@ conn_stats_terminate(void) conn_stats_reset(0); } +/** + * Record a single entry @a ent in the counts structure @cnt. + */ +static void +add_entry_to_count(conn_counts_t *cnt, const bidi_map_entry_t *ent) +{ + if (ent->read + ent->written < BIDI_THRESHOLD) + cnt->below_threshold++; + else if (ent->read >= ent->written * BIDI_FACTOR) + cnt->mostly_read++; + else if (ent->written >= ent->read * BIDI_FACTOR) + cnt->mostly_written++; + else + cnt->both_read_and_written++; +} + +/** + * Count all the connection information we've received during the current + * period in 'bidimap', and store that information in the appropriate count + * structures. + **/ +static void +collect_period_statistics(void) +{ + bidi_map_entry_t **ptr, **next, *ent; + conn_counts_t *cnt = &counts; + for (ptr = HT_START(bidimap, &bidi_map); ptr; ptr = next) { + ent = *ptr; + add_entry_to_count(cnt, ent); + next = HT_NEXT_RMV(bidimap, &bidi_map, ptr); + tor_free(ent); + } + log_info(LD_GENERAL, "%d below threshold, %d mostly read, " + "%d mostly written, %d both read and written.", + cnt->below_threshold, cnt->mostly_read, cnt->mostly_written, + cnt->both_read_and_written); +} + /** We read num_read bytes and wrote num_written from/to OR * connection conn_id in second when. If this is the first * observation in a new interval, sum up the last observations. Add bytes @@ -149,27 +187,9 @@ conn_stats_note_or_conn_bytes(uint64_t conn_id, size_t num_read, bidi_next_interval = when + BIDI_INTERVAL; /* Sum up last period's statistics */ if (when >= bidi_next_interval) { - bidi_map_entry_t **ptr, **next, *ent; - conn_counts_t *cnt = &counts; - for (ptr = HT_START(bidimap, &bidi_map); ptr; ptr = next) { - ent = *ptr; - if (ent->read + ent->written < BIDI_THRESHOLD) - cnt->below_threshold++; - else if (ent->read >= ent->written * BIDI_FACTOR) - cnt->mostly_read++; - else if (ent->written >= ent->read * BIDI_FACTOR) - cnt->mostly_written++; - else - cnt->both_read_and_written++; - next = HT_NEXT_RMV(bidimap, &bidi_map, ptr); - tor_free(ent); - } + collect_period_statistics(); while (when >= bidi_next_interval) bidi_next_interval += BIDI_INTERVAL; - log_info(LD_GENERAL, "%d below threshold, %d mostly read, " - "%d mostly written, %d both read and written.", - cnt->below_threshold, cnt->mostly_read, cnt->mostly_written, - cnt->both_read_and_written); } /* Add this connection's bytes. */ if (num_read > 0 || num_written > 0) {

1 0

[tor/master] connstats: extract connection type counts into a structure.
by dgoulet＠torproject.org 10 Jul '20

10 Jul '20

commit 0b5e19d22312291d5c9f8a5e8f8f3e6ad2ef4920 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Jul 10 09:32:30 2020 -0400 connstats: extract connection type counts into a structure. --- src/feature/stats/connstats.c | 58 +++++++++++++++++++++++-------------------- src/feature/stats/connstats.h | 2 +- 2 files changed, 32 insertions(+), 28 deletions(-) diff --git a/src/feature/stats/connstats.c b/src/feature/stats/connstats.c index 82b1adcd9..2879a3018 100644 --- a/src/feature/stats/connstats.c +++ b/src/feature/stats/connstats.c @@ -48,22 +48,28 @@ conn_stats_init(time_t now) /** Start of next BIDI_INTERVAL second interval. */ static time_t bidi_next_interval = 0; -/** Number of connections that we read and wrote less than BIDI_THRESHOLD - * bytes from/to in BIDI_INTERVAL seconds. */ -static uint32_t below_threshold = 0; +/** A single grouped set of connection type counts. */ +typedef struct conn_counts_t { + /** Number of connections that we read and wrote less than BIDI_THRESHOLD + * bytes from/to in BIDI_INTERVAL seconds. */ + uint32_t below_threshold; -/** Number of connections that we read at least BIDI_FACTOR times more - * bytes from than we wrote to in BIDI_INTERVAL seconds. */ -static uint32_t mostly_read = 0; + /** Number of connections that we read at least BIDI_FACTOR times more + * bytes from than we wrote to in BIDI_INTERVAL seconds. */ + uint32_t mostly_read; -/** Number of connections that we wrote at least BIDI_FACTOR times more - * bytes to than we read from in BIDI_INTERVAL seconds. */ -static uint32_t mostly_written = 0; + /** Number of connections that we wrote at least BIDI_FACTOR times more + * bytes to than we read from in BIDI_INTERVAL seconds. */ + uint32_t mostly_written; -/** Number of connections that we read and wrote at least BIDI_THRESHOLD - * bytes from/to, but not BIDI_FACTOR times more in either direction in - * BIDI_INTERVAL seconds. */ -static uint32_t both_read_and_written = 0; + /** Number of connections that we read and wrote at least BIDI_THRESHOLD + * bytes from/to, but not BIDI_FACTOR times more in either direction in + * BIDI_INTERVAL seconds. */ + uint32_t both_read_and_written; +} conn_counts_t ; + +/** A collection of connection counts, over all OR connections. */ +static conn_counts_t counts; /** Entry in a map from connection ID to the number of read and written * bytes on this connection in a BIDI_INTERVAL second interval. */ @@ -116,10 +122,7 @@ void conn_stats_reset(time_t now) { start_of_conn_stats_interval = now; - below_threshold = 0; - mostly_read = 0; - mostly_written = 0; - both_read_and_written = 0; + memset(&counts, 0, sizeof(counts)); conn_stats_free_all(); } @@ -147,16 +150,17 @@ conn_stats_note_or_conn_bytes(uint64_t conn_id, size_t num_read, /* Sum up last period's statistics */ if (when >= bidi_next_interval) { bidi_map_entry_t **ptr, **next, *ent; + conn_counts_t *cnt = &counts; for (ptr = HT_START(bidimap, &bidi_map); ptr; ptr = next) { ent = *ptr; if (ent->read + ent->written < BIDI_THRESHOLD) - below_threshold++; + cnt->below_threshold++; else if (ent->read >= ent->written * BIDI_FACTOR) - mostly_read++; + cnt->mostly_read++; else if (ent->written >= ent->read * BIDI_FACTOR) - mostly_written++; + cnt->mostly_written++; else - both_read_and_written++; + cnt->both_read_and_written++; next = HT_NEXT_RMV(bidimap, &bidi_map, ptr); tor_free(ent); } @@ -164,8 +168,8 @@ conn_stats_note_or_conn_bytes(uint64_t conn_id, size_t num_read, bidi_next_interval += BIDI_INTERVAL; log_info(LD_GENERAL, "%d below threshold, %d mostly read, " "%d mostly written, %d both read and written.", - below_threshold, mostly_read, mostly_written, - both_read_and_written); + cnt->below_threshold, cnt->mostly_read, cnt->mostly_written, + cnt->both_read_and_written); } /* Add this connection's bytes. */ if (num_read > 0 || num_written > 0) { @@ -202,10 +206,10 @@ conn_stats_format(time_t now) tor_asprintf(&result, "conn-bi-direct %s (%d s) %d,%d,%d,%d\n", written, (unsigned) (now - start_of_conn_stats_interval), - below_threshold, - mostly_read, - mostly_written, - both_read_and_written); + counts.below_threshold, + counts.mostly_read, + counts.mostly_written, + counts.both_read_and_written); return result; } diff --git a/src/feature/stats/connstats.h b/src/feature/stats/connstats.h index 26ff74cfd..43b654974 100644 --- a/src/feature/stats/connstats.h +++ b/src/feature/stats/connstats.h @@ -14,7 +14,7 @@ void conn_stats_init(time_t now); void conn_stats_note_or_conn_bytes(uint64_t conn_id, size_t num_read, - size_t num_written, time_t when); + size_t num_written, time_t when); void conn_stats_reset(time_t now); char *conn_stats_format(time_t now); time_t conn_stats_save(time_t now);

1 0