February 2018 - tor-commits - lists.torproject.org

[tor-browser-build/master] Bug 25266: PT config should include full names of executable files
by boklm＠torproject.org 16 Feb '18

16 Feb '18

commit 729ad3b42b612829301a1231879296a77c95a35c Author: Kathy Brade <brade(a)pearlcrescent.com> Date: Thu Feb 15 15:56:28 2018 +0000 Bug 25266: PT config should include full names of executable files Tor Launcher's Moat client implementation uses Mozilla's Subprocess.jsm module to start the meek client, and that module requires a complete file path including the .exe suffix. For consistency, we now include the .exe suffix everywhere. --- .../Bundle-Data/PTConfigs/windows/torrc-defaults-appendix | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/projects/tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix b/projects/tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix index 2c1b636..94d7fd9 100644 --- a/projects/tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix +++ b/projects/tor-browser/Bundle-Data/PTConfigs/windows/torrc-defaults-appendix @@ -1,8 +1,8 @@ ## fteproxy configuration -ClientTransportPlugin fte exec TorBrowser\Tor\PluggableTransports\fteproxy --managed +ClientTransportPlugin fte exec TorBrowser\Tor\PluggableTransports\fteproxy.exe --managed ## obfs4proxy configuration -ClientTransportPlugin obfs2,obfs3,obfs4,scramblesuit exec TorBrowser\Tor\PluggableTransports\obfs4proxy +ClientTransportPlugin obfs2,obfs3,obfs4,scramblesuit exec TorBrowser\Tor\PluggableTransports\obfs4proxy.exe ## meek configuration -ClientTransportPlugin meek exec TorBrowser\Tor\PluggableTransports\terminateprocess-buffer TorBrowser\Tor\PluggableTransports\meek-client-torbrowser -- TorBrowser\Tor\PluggableTransports\meek-client +ClientTransportPlugin meek exec TorBrowser\Tor\PluggableTransports\terminateprocess-buffer.exe TorBrowser\Tor\PluggableTransports\meek-client-torbrowser.exe -- TorBrowser\Tor\PluggableTransports\meek-client.exe

1 0

[tor/release-0.3.2] dos: Man page entry for DoS mitigation
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit a3714268f659998dc879ed723852440cd8be1b04 Author: David Goulet <dgoulet(a)torproject.org> Date: Fri Jan 26 09:00:17 2018 -0500 dos: Man page entry for DoS mitigation Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- doc/tor.1.txt | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 4c5d5359a..a2bbb8ab6 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -2441,6 +2441,96 @@ The following options are used to configure a hidden service. including setting SOCKSPort to "0". (Default: 0) +DENIAL OF SERVICE MITIGATION OPTIONS +------------------------------------ + +The following options are useful only for a public relay. They control the +Denial of Service mitigation subsystem. + +[[DoSCircuitCreationEnabled]] **DoSCircuitCreationEnabled** **0**|**1**|**auto**:: + + Enable circuit creation DoS mitigation. If enabled, tor will cache client + IPs along with statistics in order to detect circuit DoS attacks. If an + address is positively identified, tor will activate defenses against the + address. See the DoSCircuitCreationDefenseType option for more details. + This is a client to relay detection only. "auto" means use the consensus + parameter. + (Default: auto) + +[[DoSCircuitCreationMinConnections]] **DoSCircuitCreationMinConnections** __NUM__:: + + Minimum threshold of concurrent connections before a client address can be + flagged as executing a circuit creation DoS. In other words, once a client + address reaches the circuit rate and has a minimum of NUM concurrent + connections, a detection is positive. "0" means use the consensus + parameter. + (Default: 0) + +[[DoSCircuitCreationRateTenths]] **DoSCircuitCreationRateTenths** __NUM__:: + + The allowed circuit creation rate in tenths of circuit per second applied + per client IP address. For example, if you want to set a rate of 5 + circuits per second allowed per IP address, this value should be set to + 50. If this option is 0, it obeys a consensus parameter. (Default: 0) + +[[DoSCircuitCreationBurst]] **DoSCircuitCreationBurst** __NUM__:: + + The allowed circuit creation burst per client IP address. If the circuit + rate and the burst are reached, a client is marked as executing a circuit + creation DoS. "0" means use the consensus parameter. + (Default: 0) + +[[DoSCircuitCreationDefenseType]] **DoSCircuitCreationDefenseType** __NUM__:: + + This is the type of defense applied to a detected client address. The + possible values are: + + 1: No defense. + 2: Refuse circuit creation for the DoSCircuitCreationDefenseTimePeriod period of time. ++ + "0" means use the consensus parameter. + (Default: 0) + +[[DoSCircuitCreationDefenseTimePeriod]] **DoSCircuitCreationDefenseTimePeriod** __NUM__:: + + The base time period that the DoS defense is activated for. The actual + value is selected randomly for each activation from NUM+1 to 3/2 * NUM. + "0" means use the consensus parameter. + (Default: 0) + +[[DoSConnectionEnabled]] **DoSConnectionEnabled** **0**|**1**|**auto**:: + + Enable the connection DoS mitigation. For client address only, this allows + tor to mitigate against large number of concurrent connections made by a + single IP address. "auto" means use the consensus parameter. + (Default: auto) + +[[DoSConnectionMaxConcurrentCount]] **DoSConnectionMaxConcurrentCount** __NUM__:: + + The maximum threshold of concurrent connection from a client IP address. + Above this limit, a defense selected by DoSConnectionDefenseType is + applied. "0" means use the consensus parameter. + (Default: 0) + +[[DoSConnectionDefenseType]] **DoSConnectionDefenseType** __NUM__:: + + This is the type of defense applied to a detected client address for the + connection mitigation. The possible values are: + + 1: No defense. + 2: Immediately close new connections. ++ + "0" means use the consensus parameter. + (Default: 0) + +[[DoSRefuseSingleHopClientRendezvous]] **DoSRefuseSingleHopClientRendezvous** **0**|**1**|**auto**:: + + Refuse establishment of rendezvous points for single hop clients. In other + words, if a client directly connects to the relay and sends an + ESTABLISH_RENDEZVOUS cell, it is silently dropped. "auto" means use the + consensus parameter. + (Default: auto) + TESTING NETWORK OPTIONS -----------------------

1 0

[tor/release-0.3.2] test: Remove a redundant round from test_dos_bucket_refill
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit b45ae1b00237f0209c4ccce777de59581fda5e39 Author: teor <teor2345(a)gmail.com> Date: Wed Jan 31 11:11:08 2018 +1100 test: Remove a redundant round from test_dos_bucket_refill This round is left over from the tenths of a second code. Part of #25094. --- src/test/test_dos.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/test/test_dos.c b/src/test/test_dos.c index 5a8474ad8..80abc1937 100644 --- a/src/test/test_dos.c +++ b/src/test/test_dos.c @@ -176,7 +176,7 @@ test_dos_bucket_refill(void *arg) /* Initialize DoS subsystem and get relevant limits */ dos_init(); uint32_t max_circuit_count = get_param_cc_circuit_burst(NULL); - int circ_rate = tor_lround(get_circuit_rate_per_second()); + int circ_rate = get_circuit_rate_per_second(); /* Check that the circuit rate is a positive number and smaller than the max * circuit count */ tt_int_op(circ_rate, OP_GT, 1);

1 0

[tor/release-0.3.2] dos: Make circuit rate limit per second, not tenths anymore
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit e58a4fc6cfcdeafc2ebfb61fd3cf6d163ce2436c Author: David Goulet <dgoulet(a)torproject.org> Date: Mon Jan 29 11:50:11 2018 -0500 dos: Make circuit rate limit per second, not tenths anymore Because this touches too many commits at once, it is made into one single commit. Remove the use of "tenths" for the circuit rate to simplify things. We can only refill the buckets at best once every second because of the use of approx_time() and our token system is set to be 1 token = 1 circuit so make the rate a flat integer of circuit per second. Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- doc/tor.1.txt | 8 +++----- src/or/config.c | 2 +- src/or/dos.c | 32 ++++++++------------------------ src/or/dos.h | 2 +- src/or/or.h | 5 ++--- 5 files changed, 15 insertions(+), 34 deletions(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index a2bbb8ab6..58997cdf3 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -2466,12 +2466,10 @@ Denial of Service mitigation subsystem. parameter. (Default: 0) -[[DoSCircuitCreationRateTenths]] **DoSCircuitCreationRateTenths** __NUM__:: +[[DoSCircuitCreationRate]] **DoSCircuitCreationRate** __NUM__:: - The allowed circuit creation rate in tenths of circuit per second applied - per client IP address. For example, if you want to set a rate of 5 - circuits per second allowed per IP address, this value should be set to - 50. If this option is 0, it obeys a consensus parameter. (Default: 0) + The allowed circuit creation rate per second applied per client IP + address. If this option is 0, it obeys a consensus parameter. (Default: 0) [[DoSCircuitCreationBurst]] **DoSCircuitCreationBurst** __NUM__:: diff --git a/src/or/config.c b/src/or/config.c index c651c202e..3b4027433 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -245,7 +245,7 @@ static config_var_t option_vars_[] = { /* DoS circuit creation options. */ V(DoSCircuitCreationEnabled, AUTOBOOL, "auto"), V(DoSCircuitCreationMinConnections, UINT, "0"), - V(DoSCircuitCreationRateTenths, UINT, "0"), + V(DoSCircuitCreationRate, UINT, "0"), V(DoSCircuitCreationBurst, UINT, "0"), V(DoSCircuitCreationDefenseType, INT, "0"), V(DoSCircuitCreationDefenseTimePeriod, INTERVAL, "0"), diff --git a/src/or/dos.c b/src/or/dos.c index 5af75ca57..a614d1231 100644 --- a/src/or/dos.c +++ b/src/or/dos.c @@ -31,7 +31,7 @@ static unsigned int dos_cc_enabled = 0; /* Consensus parameters. They can be changed when a new consensus arrives. * They are initialized with the hardcoded default values. */ static uint32_t dos_cc_min_concurrent_conn; -static uint32_t dos_cc_circuit_rate_tenths; +static uint32_t dos_cc_circuit_rate; static uint32_t dos_cc_circuit_burst; static dos_cc_defense_type_t dos_cc_defense_type; static int32_t dos_cc_defense_time_period; @@ -93,14 +93,14 @@ get_param_cc_min_concurrent_connection(const networkstatus_t *ns) /* Return the parameter for the time rate that is how many circuits over this * time span. */ static uint32_t -get_param_cc_circuit_rate_tenths(const networkstatus_t *ns) +get_param_cc_circuit_rate(const networkstatus_t *ns) { /* This is in seconds. */ - if (get_options()->DoSCircuitCreationRateTenths) { - return get_options()->DoSCircuitCreationRateTenths; + if (get_options()->DoSCircuitCreationRate) { + return get_options()->DoSCircuitCreationRate; } - return networkstatus_get_param(ns, "DoSCircuitCreationRateTenths", - DOS_CC_CIRCUIT_RATE_TENTHS_DEFAULT, + return networkstatus_get_param(ns, "DoSCircuitCreationRate", + DOS_CC_CIRCUIT_RATE_DEFAULT, 1, INT32_MAX); } @@ -189,7 +189,7 @@ set_dos_parameters(const networkstatus_t *ns) /* Get the default consensus param values. */ dos_cc_enabled = get_param_cc_enabled(ns); dos_cc_min_concurrent_conn = get_param_cc_min_concurrent_connection(ns); - dos_cc_circuit_rate_tenths = get_param_cc_circuit_rate_tenths(ns); + dos_cc_circuit_rate = get_param_cc_circuit_rate(ns); dos_cc_circuit_burst = get_param_cc_circuit_burst(ns); dos_cc_defense_time_period = get_param_cc_defense_time_period(ns); dos_cc_defense_type = get_param_cc_defense_type(ns); @@ -225,23 +225,7 @@ cc_consensus_has_changed(const networkstatus_t *ns) STATIC uint32_t get_circuit_rate_per_second(void) { - int64_t circ_rate; - - /* We take the burst divided by the rate which is in tenths of a second so - * convert to get a circuit rate per second. */ - circ_rate = dos_cc_circuit_rate_tenths / 10; - if (circ_rate < 0) { - /* Safety check, never allow it to go below 0 else the bucket will always - * be empty resulting in every address to be detected. */ - circ_rate = 1; - } - - /* Clamp it down to a 32 bit value because a rate of 2^32 circuits per - * second is just too much in any circumstances. */ - if (circ_rate > UINT32_MAX) { - circ_rate = UINT32_MAX; - } - return (uint32_t) circ_rate; + return dos_cc_circuit_rate; } /* Given the circuit creation client statistics object, refill the circuit diff --git a/src/or/dos.h b/src/or/dos.h index 9ce1baddb..8695512ea 100644 --- a/src/or/dos.h +++ b/src/or/dos.h @@ -70,7 +70,7 @@ void dos_note_refuse_single_hop_client(void); /* DoSCircuitCreationMinConnections default */ #define DOS_CC_MIN_CONCURRENT_CONN_DEFAULT 3 /* DoSCircuitCreationRateTenths is 3 per seconds. */ -#define DOS_CC_CIRCUIT_RATE_TENTHS_DEFAULT (3 * 10) +#define DOS_CC_CIRCUIT_RATE_DEFAULT 3 /* DoSCircuitCreationBurst default. */ #define DOS_CC_CIRCUIT_BURST_DEFAULT 90 /* DoSCircuitCreationDefenseTimePeriod in seconds. */ diff --git a/src/or/or.h b/src/or/or.h index 454d05ed5..024a9cff0 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -4520,9 +4520,8 @@ typedef struct { /** Minimum concurrent connection needed from one single address before any * defense is used. */ int DoSCircuitCreationMinConnections; - /** Circuit rate, in tenths of a second, that is used to refill the token - * bucket at this given rate. */ - int DoSCircuitCreationRateTenths; + /** Circuit rate used to refill the token bucket. */ + int DoSCircuitCreationRate; /** Maximum allowed burst of circuits. Reaching that value, the address is * detected as malicious and a defense might be used. */ int DoSCircuitCreationBurst;

1 0

[tor/release-0.3.2] dos: Add changes file for ticket 24902
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit 9aca7d47306222f2870ec16a7291a8215d6c3316 Author: David Goulet <dgoulet(a)torproject.org> Date: Tue Jan 30 09:15:33 2018 -0500 dos: Add changes file for ticket 24902 Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- changes/ticket24902 | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/changes/ticket24902 b/changes/ticket24902 new file mode 100644 index 000000000..1a2ef95cc --- /dev/null +++ b/changes/ticket24902 @@ -0,0 +1,13 @@ + o Major features (denial of service mitigation): + - Give relays some defenses against the recent network overload. We start + with three defenses (default parameters in parentheses). First: if a + single client address makes too many concurrent connections (>100), hang + up on further connections. Second: if a single client address makes + circuits too quickly (more than 3 per second, with an allowed burst of + 90) while also having too many connections open (3), refuse new create + cells for the next while (1-2 hours). Third: if a client asks to + establish a rendezvous point to you directly, ignore the request. These + defenses can be manually controlled by new torrc options, but relays + will also take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. +

1 0

[tor/release-0.3.2] test: Add unit tests for overflows and underflows in cc_stats_refill_bucket
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit 1f4a73133cf864774c017e2c50b347727519c18f Author: teor <teor2345(a)gmail.com> Date: Wed Jan 31 11:22:20 2018 +1100 test: Add unit tests for overflows and underflows in cc_stats_refill_bucket Closes #25094. Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- src/test/test_dos.c | 146 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 146 insertions(+) diff --git a/src/test/test_dos.c b/src/test/test_dos.c index 7fe549560..071926c28 100644 --- a/src/test/test_dos.c +++ b/src/test/test_dos.c @@ -234,6 +234,152 @@ test_dos_bucket_refill(void *arg) current_circ_count += max_circuit_count; tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now use a very large time, and check that the token bucket does not have + * more than max_circs allowance, even tho we let it simmer for so long. */ + now = INT32_MAX; /* 2038? */ + update_approx_time(now); + cc_stats_refill_bucket(&dos_stats->cc_stats, addr); + current_circ_count += max_circuit_count; + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now use a very small time, and check that the token bucket has exactly + * the max_circs allowance, because backward clock jumps are rare. */ + now = INT32_MIN; /* 19?? */ + update_approx_time(now); + cc_stats_refill_bucket(&dos_stats->cc_stats, addr); + current_circ_count += max_circuit_count; + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Progress time forward one sec again, refill the bucket and check that the + * refill happened correctly. */ + now += 1; + update_approx_time(now); + cc_stats_refill_bucket(&dos_stats->cc_stats, addr); + /* check refill */ + current_circ_count += circ_rate; + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now use a very large time (again), and check that the token bucket does + * not have more than max_circs allowance, even tho we let it simmer for so + * long. */ + now = INT32_MAX; /* 2038? */ + update_approx_time(now); + cc_stats_refill_bucket(&dos_stats->cc_stats, addr); + current_circ_count += max_circuit_count; + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* This code resets the time to zero with 32-bit time_t, which triggers the + * code that initialises the bucket. */ +#if SIZEOF_TIME_T == 8 + /* Now use a very very small time, and check that the token bucket has + * exactly the max_circs allowance, because backward clock jumps are rare. + */ + now = (time_t)INT64_MIN; /* ???? */ + update_approx_time(now); + cc_stats_refill_bucket(&dos_stats->cc_stats, addr); + current_circ_count += max_circuit_count; + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Progress time forward one sec again, refill the bucket and check that the + * refill happened correctly. */ + now += 1; + update_approx_time(now); + cc_stats_refill_bucket(&dos_stats->cc_stats, addr); + /* check refill */ + current_circ_count += circ_rate; + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now use a very very small time, and check that the token bucket has + * exactly the max_circs allowance, because backward clock jumps are rare. + */ + now = (time_t)INT64_MIN; /* ???? */ + update_approx_time(now); + cc_stats_refill_bucket(&dos_stats->cc_stats, addr); + current_circ_count += max_circuit_count; + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now use a very very large time, and check that the token bucket does not + * have more than max_circs allowance, even tho we let it simmer for so + * long. */ + now = (time_t)INT64_MAX; /* ???? */ + update_approx_time(now); + cc_stats_refill_bucket(&dos_stats->cc_stats, addr); + current_circ_count += max_circuit_count; + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); + + /* Now send as many CREATE cells as needed to deplete our token bucket + * completely */ + for (; current_circ_count != 0; current_circ_count--) { + dos_cc_new_create_cell(chan); + } + tt_uint_op(current_circ_count, OP_EQ, 0); + tt_uint_op(dos_stats->cc_stats.circuit_bucket, OP_EQ, current_circ_count); +#endif + done: tor_free(chan); dos_free_all();

1 0

[tor/release-0.3.2] Merge branch 'ticket25122_029_02' into ticket24902_029_05
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit 475218c108ad84aa302f0daec013faab9ff778f2 Merge: 33d9889a2 e758d659a Author: David Goulet <dgoulet(a)torproject.org> Date: Fri Feb 2 14:55:01 2018 -0500 Merge branch 'ticket25122_029_02' into ticket24902_029_05 changes/ticket25122 | 4 ++ src/or/geoip.c | 148 ++++++++++++++++++++++++++++++++++++++++++++++++++-- src/or/geoip.h | 2 + src/or/relay.c | 16 ++++-- src/test/test.c | 18 +++++++ 5 files changed, 180 insertions(+), 8 deletions(-) diff --cc src/or/geoip.c index 4e4f6e639,76fca43f6..20dad5f15 --- a/src/or/geoip.c +++ b/src/or/geoip.c @@@ -516,9 -574,7 +557,10 @@@ clientmap_entry_free(clientmap_entry_t if (!ent) return; + /* This entry is about to be freed so pass it to the DoS subsystem to see if + * any actions can be taken about it. */ + dos_geoip_entry_about_to_free(ent); + geoip_decrement_client_history_cache_size(clientmap_entry_size(ent)); tor_free(ent->transport_name); tor_free(ent); @@@ -573,13 -651,13 +640,9 @@@ geoip_note_client_seen(geoip_client_act safe_str_client(fmt_addr((addr))), transport_name ? transport_name : "<no transport>"); - tor_addr_copy(&lookup.addr, addr); - lookup.action = (int)action; - lookup.transport_name = (char*) transport_name; - ent = HT_FIND(clientmap, &client_history, &lookup); - + ent = geoip_lookup_client(addr, transport_name, action); if (! ent) { - ent = tor_malloc_zero(sizeof(clientmap_entry_t)); - tor_addr_copy(&ent->addr, addr); - if (transport_name) - ent->transport_name = tor_strdup(transport_name); - ent->action = (int)action; + ent = clientmap_entry_new(action, addr, transport_name); HT_INSERT(clientmap, &client_history, ent); } if (now / 60 <= (int)MAX_LAST_SEEN_IN_MINUTES && now >= 0) @@@ -621,25 -699,81 +684,100 @@@ geoip_remove_old_clients(time_t cutoff &cutoff); } +/* Return a client entry object matching the given address, transport name and + * geoip action from the clientmap. NULL if not found. The transport_name can + * be NULL. */ +clientmap_entry_t * +geoip_lookup_client(const tor_addr_t *addr, const char *transport_name, + geoip_client_action_t action) +{ + clientmap_entry_t lookup; + + tor_assert(addr); + + /* We always look for a client connection with no transport. */ + tor_addr_copy(&lookup.addr, addr); + lookup.action = action; + lookup.transport_name = (char *) transport_name; + + return HT_FIND(clientmap, &client_history, &lookup); +} + + /* Cleanup client entries older than the cutoff. Used for the OOM. Return the + * number of bytes freed. If 0 is returned, nothing was freed. */ + static size_t + oom_clean_client_entries(time_t cutoff) + { + size_t bytes = 0; + clientmap_entry_t **ent, **ent_next; + + for (ent = HT_START(clientmap, &client_history); ent; ent = ent_next) { + clientmap_entry_t *entry = *ent; + if (entry->last_seen_in_minutes < (cutoff / 60)) { + ent_next = HT_NEXT_RMV(clientmap, &client_history, ent); + bytes += clientmap_entry_size(entry); + clientmap_entry_free(entry); + } else { + ent_next = HT_NEXT(clientmap, &client_history, ent); + } + } + return bytes; + } + + /* Below this minimum lifetime, the OOM won't cleanup any entries. */ + #define GEOIP_CLIENT_CACHE_OOM_MIN_CUTOFF (4 * 60 * 60) + /* The OOM moves the cutoff by that much every run. */ + #define GEOIP_CLIENT_CACHE_OOM_STEP (15 * 50) + + /* Cleanup the geoip client history cache called from the OOM handler. Return + * the amount of bytes removed. This can return a value below or above + * min_remove_bytes but will stop as oon as the min_remove_bytes has been + * reached. */ + size_t + geoip_client_cache_handle_oom(time_t now, size_t min_remove_bytes) + { + time_t k; + size_t bytes_removed = 0; + + /* Our OOM handler called with 0 bytes to remove is a code flow error. */ + tor_assert(min_remove_bytes != 0); + + /* Set k to the initial cutoff of an entry. We then going to move it by step + * to try to remove as much as we can. */ + k = WRITE_STATS_INTERVAL; + + do { + time_t cutoff; + + /* If k has reached the minimum lifetime, we have to stop else we might + * remove every single entries which would be pretty bad for the DoS + * mitigation subsystem if by just filling the geoip cache, it was enough + * to trigger the OOM and clean every single entries. */ + if (k <= GEOIP_CLIENT_CACHE_OOM_MIN_CUTOFF) { + break; + } + + cutoff = now - k; + bytes_removed += oom_clean_client_entries(cutoff); + k -= GEOIP_CLIENT_CACHE_OOM_STEP; + } while (bytes_removed < min_remove_bytes); + + return bytes_removed; + } + + /* Return the total size in bytes of the client history cache. */ + size_t + geoip_client_cache_total_allocation(void) + { + size_t bytes = 0; + clientmap_entry_t **ent; + + HT_FOREACH(ent, clientmap, &client_history) { + bytes += clientmap_entry_size(*ent); + } + return bytes; + } + /** How many responses are we giving to clients requesting v3 network * statuses? */ static uint32_t ns_v3_responses[GEOIP_NS_RESPONSE_NUM]; diff --cc src/or/geoip.h index aa0fca50f,42d0c1cfd..c8ea9f85e --- a/src/or/geoip.h +++ b/src/or/geoip.h @@@ -57,9 -33,8 +57,11 @@@ void geoip_note_client_seen(geoip_clien const tor_addr_t *addr, const char *transport_name, time_t now); void geoip_remove_old_clients(time_t cutoff); +clientmap_entry_t *geoip_lookup_client(const tor_addr_t *addr, + const char *transport_name, + geoip_client_action_t action); + size_t geoip_client_cache_total_allocation(void); + size_t geoip_client_cache_handle_oom(time_t now, size_t min_remove_bytes); void geoip_note_ns_response(geoip_ns_response_t response); char *geoip_get_transport_history(void);

1 0

[tor/release-0.3.2] geoip: Increment and decrement functions for the geoip client cache
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit 4d812e29b9b1ec88fe268c150a826466b23a8762 Author: David Goulet <dgoulet(a)torproject.org> Date: Fri Feb 2 13:14:50 2018 -0500 geoip: Increment and decrement functions for the geoip client cache These functions protect againts over and underflow. They BUG() in case we overflow the counter. Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- src/or/geoip.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/src/or/geoip.c b/src/or/geoip.c index c5e8cdab9..92db9742e 100644 --- a/src/or/geoip.c +++ b/src/or/geoip.c @@ -76,6 +76,34 @@ static char geoip6_digest[DIGEST_LEN]; * handler. */ static size_t geoip_client_history_cache_size; +/* Increment the geoip client history cache size counter with the given bytes. + * This prevents an overflow and set it to its maximum in that case. */ +static inline void +geoip_increment_client_history_cache_size(size_t bytes) +{ + /* This is shockingly high, lets log it so it can be reported. */ + IF_BUG_ONCE(geoip_client_history_cache_size > (SIZE_MAX - bytes)) { + geoip_client_history_cache_size = SIZE_MAX; + return; + } + geoip_client_history_cache_size += bytes; +} + +/* Decrement the geoip client history cache size counter with the given bytes. + * This prevents an underflow and set it to 0 in that case. */ +static inline void +geoip_decrement_client_history_cache_size(size_t bytes) +{ + /* Going below 0 means that we either allocated an entry without + * incrementing the counter or we have different sizes when allocating and + * freeing. It shouldn't happened so log it. */ + IF_BUG_ONCE(geoip_client_history_cache_size < bytes) { + geoip_client_history_cache_size = 0; + return; + } + geoip_client_history_cache_size -= bytes; +} + /** Return the index of the <b>country</b>'s entry in the GeoIP * country list if it is a valid 2-letter country code, otherwise * return -1. */ @@ -546,7 +574,7 @@ clientmap_entry_free(clientmap_entry_t *ent) if (!ent) return; - geoip_client_history_cache_size -= clientmap_entry_size(ent); + geoip_decrement_client_history_cache_size(clientmap_entry_size(ent)); tor_free(ent->transport_name); tor_free(ent); @@ -610,7 +638,7 @@ geoip_note_client_seen(geoip_client_action_t action, ent->transport_name = tor_strdup(transport_name); ent->action = (int)action; HT_INSERT(clientmap, &client_history, ent); - geoip_client_history_cache_size += clientmap_entry_size(ent); + geoip_increment_client_history_cache_size(clientmap_entry_size(ent)); } if (now / 60 <= (int)MAX_LAST_SEEN_IN_MINUTES && now >= 0) ent->last_seen_in_minutes = (unsigned)(now/60);

1 0

[tor/release-0.3.2] remove a redundant semicolon
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit 22a5d3dd2ab793336e911d3aceb8cacd278e0f48 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jan 30 18:11:16 2018 -0500 remove a redundant semicolon --- src/test/test_dos.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/test/test_dos.c b/src/test/test_dos.c index 9496b0735..6db98b9ed 100644 --- a/src/test/test_dos.c +++ b/src/test/test_dos.c @@ -78,7 +78,7 @@ static int mock_channel_get_addr_if_possible(channel_t *chan, tor_addr_t *addr_out) { (void)chan; - tt_int_op(AF_INET,OP_EQ, tor_addr_parse(addr_out, "18.0.0.1"));; + tt_int_op(AF_INET,OP_EQ, tor_addr_parse(addr_out, "18.0.0.1")); return 1; done:

1 0

[tor/release-0.3.2] geoip: Add clientmap_entry_new() function
by nickm＠torproject.org 16 Feb '18

16 Feb '18

commit e758d659a0bc8b9a0e2bd6a0126755fd1fb58e0a Author: David Goulet <dgoulet(a)torproject.org> Date: Fri Feb 2 13:24:37 2018 -0500 geoip: Add clientmap_entry_new() function Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- src/or/geoip.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/src/or/geoip.c b/src/or/geoip.c index 92db9742e..76fca43f6 100644 --- a/src/or/geoip.c +++ b/src/or/geoip.c @@ -580,6 +580,31 @@ clientmap_entry_free(clientmap_entry_t *ent) tor_free(ent); } +/* Return a newly allocated clientmap entry with the given action and address + * that are mandatory. The transport_name can be optional. This can't fail. */ +static clientmap_entry_t * +clientmap_entry_new(geoip_client_action_t action, const tor_addr_t *addr, + const char *transport_name) +{ + clientmap_entry_t *entry; + + tor_assert(action == GEOIP_CLIENT_CONNECT || + action == GEOIP_CLIENT_NETWORKSTATUS); + tor_assert(addr); + + entry = tor_malloc_zero(sizeof(clientmap_entry_t)); + entry->action = action; + tor_addr_copy(&entry->addr, addr); + if (transport_name) { + entry->transport_name = tor_strdup(transport_name); + } + + /* Allocated and initialized, note down its size for the OOM handler. */ + geoip_increment_client_history_cache_size(clientmap_entry_size(entry)); + + return entry; +} + /** Clear history of connecting clients used by entry and bridge stats. */ static void client_history_clear(void) @@ -632,13 +657,8 @@ geoip_note_client_seen(geoip_client_action_t action, ent = HT_FIND(clientmap, &client_history, &lookup); if (! ent) { - ent = tor_malloc_zero(sizeof(clientmap_entry_t)); - tor_addr_copy(&ent->addr, addr); - if (transport_name) - ent->transport_name = tor_strdup(transport_name); - ent->action = (int)action; + ent = clientmap_entry_new(action, addr, transport_name); HT_INSERT(clientmap, &client_history, ent); - geoip_increment_client_history_cache_size(clientmap_entry_size(ent)); } if (now / 60 <= (int)MAX_LAST_SEEN_IN_MINUTES && now >= 0) ent->last_seen_in_minutes = (unsigned)(now/60);

1 0