January 2021 - tor-commits - lists.torproject.org

[tor/maint-0.4.3] hs-v3: Require reasonably live consensus
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit 04b0263974c7ad1327e7a193884cf31d55f7949a Author: David Goulet <dgoulet(a)torproject.org> Date: Mon Jan 11 16:01:22 2021 -0500 hs-v3: Require reasonably live consensus Some days before this commit, the network experienced a DDoS on the directory authorities that prevented them to generate a consensus for more than 5 hours straight. That in turn entirely disabled onion service v3, client and service side, due to the subsystem requiring a live consensus to function properly. We know require a reasonably live consensus which means that the HSv3 subsystem will to its job for using the best consensus tor can find. If the entire network is using an old consensus, than this should be alright. If the service happens to use a live consensus while a client is not, it should still work because the client will use the current SRV it sees which might be the previous SRV for the service for which it still publish descriptors for. If the service is using an old one and somehow can't get a new one while clients are on a new one, then reachability issues might arise. However, this is a situation we already have at the moment since the service will simply not work if it doesn't have a live consensus while a client has one. Fixes #40237 Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- changes/ticket40237 | 5 ++++ src/core/mainloop/mainloop.c | 3 ++- src/feature/hs/hs_cache.c | 5 +++- src/feature/hs/hs_client.c | 8 ++++--- src/feature/hs/hs_common.c | 12 +++++++--- src/feature/hs/hs_service.c | 7 ++++-- src/feature/hs_common/shared_random_client.c | 23 ++++++++++++------ src/feature/nodelist/nodelist.c | 2 +- src/test/test_hs_cache.c | 7 +++--- src/test/test_hs_client.c | 36 +++++++++++++++------------- src/test/test_hs_common.c | 36 ++++++++++++++++++---------- src/test/test_hs_service.c | 21 ++++++++-------- src/test/test_shared_random.c | 22 +++++++++++++---- 13 files changed, 122 insertions(+), 65 deletions(-) diff --git a/changes/ticket40237 b/changes/ticket40237 new file mode 100644 index 0000000000..fc32f59cd4 --- /dev/null +++ b/changes/ticket40237 @@ -0,0 +1,5 @@ + o Major bugfixes (onion service v3): + - Stop requiring a live consensus for v3 clients and services to work. The + use of a reasonably live consensus will allow v3 to work properly in most + cases if the network failed to generate a consensus for more than 2 hours + in a row. Fixes bug 40237; bugfix on 0.3.5.1-alpha. diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index f0aa37e8da..e67e1299b2 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -2511,7 +2511,8 @@ hs_service_callback(time_t now, const or_options_t *options) /* We need to at least be able to build circuits and that we actually have * a working network. */ if (!have_completed_a_circuit() || net_is_disabled() || - networkstatus_get_live_consensus(now) == NULL) { + !networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor())) { goto end; } diff --git a/src/feature/hs/hs_cache.c b/src/feature/hs/hs_cache.c index 05f9940ae6..042ec55fa4 100644 --- a/src/feature/hs/hs_cache.c +++ b/src/feature/hs/hs_cache.c @@ -17,6 +17,7 @@ #include "feature/hs/hs_common.h" #include "feature/hs/hs_client.h" #include "feature/hs/hs_descriptor.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "feature/rend/rendcache.h" @@ -673,7 +674,9 @@ cached_client_descriptor_has_expired(time_t now, /* We use the current consensus time to see if we should expire this * descriptor since we use consensus time for all other parts of the protocol * as well (e.g. to build the blinded key and compute time periods). */ - const networkstatus_t *ns = networkstatus_get_live_consensus(now); + const networkstatus_t *ns = + networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor()); /* If we don't have a recent consensus, consider this entry expired since we * will want to fetch a new HS desc when we get a live consensus. */ if (!ns) { diff --git a/src/feature/hs/hs_client.c b/src/feature/hs/hs_client.c index c65f857419..e25919ecb7 100644 --- a/src/feature/hs/hs_client.c +++ b/src/feature/hs/hs_client.c @@ -29,6 +29,7 @@ #include "feature/hs/hs_descriptor.h" #include "feature/hs/hs_ident.h" #include "feature/nodelist/describe.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "feature/nodelist/nodelist.h" #include "feature/nodelist/routerset.h" @@ -1181,9 +1182,10 @@ can_client_refetch_desc(const ed25519_public_key_t *identity_pk, goto cannot; } - /* Without a live consensus we can't do any client actions. It is needed to - * compute the hashring for a service. */ - if (!networkstatus_get_live_consensus(approx_time())) { + /* Without a usable consensus we can't do any client actions. It is needed + * to compute the hashring for a service. */ + if (!networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor())) { log_info(LD_REND, "Can't fetch descriptor for service %s because we " "are missing a live consensus. Stalling connection.", safe_str_client(ed25519_fmt(identity_pk))); diff --git a/src/feature/hs/hs_common.c b/src/feature/hs/hs_common.c index ebe49f09a5..de653037d1 100644 --- a/src/feature/hs/hs_common.c +++ b/src/feature/hs/hs_common.c @@ -25,6 +25,7 @@ #include "feature/hs/hs_service.h" #include "feature/hs_common/shared_random_client.h" #include "feature/nodelist/describe.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "feature/nodelist/nodelist.h" #include "feature/nodelist/routerset.h" @@ -272,7 +273,9 @@ hs_get_time_period_num(time_t now) if (now != 0) { current_time = now; } else { - networkstatus_t *ns = networkstatus_get_live_consensus(approx_time()); + networkstatus_t *ns = + networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); current_time = ns ? ns->valid_after : approx_time(); } @@ -1098,7 +1101,8 @@ hs_in_period_between_tp_and_srv,(const networkstatus_t *consensus, time_t now)) time_t srv_start_time, tp_start_time; if (!consensus) { - consensus = networkstatus_get_live_consensus(now); + consensus = networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor()); if (!consensus) { return 0; } @@ -1343,7 +1347,9 @@ hs_get_responsible_hsdirs(const ed25519_public_key_t *blinded_pk, sorted_nodes = smartlist_new(); /* Make sure we actually have a live consensus */ - networkstatus_t *c = networkstatus_get_live_consensus(approx_time()); + networkstatus_t *c = + networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); if (!c || smartlist_len(c->routerstatus_list) == 0) { log_warn(LD_REND, "No live consensus so we can't get the responsible " "hidden service directories."); diff --git a/src/feature/hs/hs_service.c b/src/feature/hs/hs_service.c index 6d32cae86c..e820ce9d0b 100644 --- a/src/feature/hs/hs_service.c +++ b/src/feature/hs/hs_service.c @@ -23,6 +23,7 @@ #include "feature/hs_common/shared_random_client.h" #include "feature/keymgt/loadkey.h" #include "feature/nodelist/describe.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "feature/nodelist/nickname.h" #include "feature/nodelist/node_select.h" @@ -2500,7 +2501,8 @@ should_rotate_descriptors(hs_service_t *service, time_t now) tor_assert(service); - ns = networkstatus_get_live_consensus(now); + ns = networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor()); if (ns == NULL) { goto no_rotation; } @@ -3100,7 +3102,8 @@ should_service_upload_descriptor(const hs_service_t *service, } /* Don't upload desc if we don't have a live consensus */ - if (!networkstatus_get_live_consensus(now)) { + if (!networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor())) { goto cannot; } diff --git a/src/feature/hs_common/shared_random_client.c b/src/feature/hs_common/shared_random_client.c index 3d6be94080..ead5d681a9 100644 --- a/src/feature/hs_common/shared_random_client.c +++ b/src/feature/hs_common/shared_random_client.c @@ -13,6 +13,7 @@ #include "app/config/config.h" #include "feature/dircommon/voting_schedule.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "lib/encoding/binascii.h" @@ -37,7 +38,9 @@ int get_voting_interval(void) { int interval; - networkstatus_t *consensus = networkstatus_get_live_consensus(time(NULL)); + networkstatus_t *consensus = + networkstatus_get_reasonably_live_consensus(time(NULL), + usable_consensus_flavor()); if (consensus) { interval = (int)(consensus->fresh_until - consensus->valid_after); @@ -142,7 +145,8 @@ sr_get_current(const networkstatus_t *ns) if (ns) { consensus = ns; } else { - consensus = networkstatus_get_live_consensus(approx_time()); + consensus = networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); } /* Ideally we would never be asked for an SRV without a live consensus. Make * sure this assumption is correct. */ @@ -165,7 +169,8 @@ sr_get_previous(const networkstatus_t *ns) if (ns) { consensus = ns; } else { - consensus = networkstatus_get_live_consensus(approx_time()); + consensus = networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); } /* Ideally we would never be asked for an SRV without a live consensus. Make * sure this assumption is correct. */ @@ -237,10 +242,14 @@ sr_state_get_start_time_of_current_protocol_run(void) int voting_interval = get_voting_interval(); time_t beginning_of_curr_round; - /* This function is not used for voting purposes, so if we have a live - consensus, use its valid-after as the beginning of the current round, - otherwise resort to the voting schedule which should always exist. */ - networkstatus_t *ns = networkstatus_get_live_consensus(approx_time()); + /* This function is not used for voting purposes, so if we have a reasonably + * live consensus, use its valid-after as the beginning of the current + * round. If we have no consensus but we're an authority, use our own + * schedule. Otherwise, try using our view of the voting interval to figure + * out when the current round _should_ be starting. */ + networkstatus_t *ns = + networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); if (ns) { beginning_of_curr_round = ns->valid_after; } else { diff --git a/src/feature/nodelist/nodelist.c b/src/feature/nodelist/nodelist.c index 99d7f746a8..8974d95db6 100644 --- a/src/feature/nodelist/nodelist.c +++ b/src/feature/nodelist/nodelist.c @@ -361,7 +361,7 @@ node_set_hsdir_index(node_t *node, const networkstatus_t *ns) tor_assert(node); tor_assert(ns); - if (!networkstatus_is_live(ns, now)) { + if (!networkstatus_consensus_reasonably_live(ns, now)) { static struct ratelim_t live_consensus_ratelim = RATELIM_INIT(30 * 60); log_fn_ratelim(&live_consensus_ratelim, LOG_INFO, LD_GENERAL, "Not setting hsdir index with a non-live consensus."); diff --git a/src/test/test_hs_cache.c b/src/test/test_hs_cache.c index 9182829116..742fa349d9 100644 --- a/src/test/test_hs_cache.c +++ b/src/test/test_hs_cache.c @@ -443,9 +443,10 @@ test_hsdir_revision_counter_check(void *arg) static networkstatus_t mock_ns; static networkstatus_t * -mock_networkstatus_get_live_consensus(time_t now) +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { (void) now; + (void) flavor; return &mock_ns; } @@ -466,8 +467,8 @@ test_client_cache(void *arg) /* Initialize HSDir cache subsystem */ init_test(); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Set consensus time */ parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", diff --git a/src/test/test_hs_client.c b/src/test/test_hs_client.c index 2f2bb45581..53ee3c53d2 100644 --- a/src/test/test_hs_client.c +++ b/src/test/test_hs_client.c @@ -62,16 +62,18 @@ static networkstatus_t mock_ns; /* Always return NULL. */ static networkstatus_t * -mock_networkstatus_get_live_consensus_false(time_t now) +mock_networkstatus_get_reasonably_live_consensus_false(time_t now, int flavor) { (void) now; + (void) flavor; return NULL; } static networkstatus_t * -mock_networkstatus_get_live_consensus(time_t now) +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { (void) now; + (void) flavor; return &mock_ns; } @@ -340,8 +342,8 @@ test_client_pick_intro(void *arg) ed25519_keypair_t service_kp; hs_descriptor_t *desc = NULL; - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); (void) arg; @@ -565,15 +567,15 @@ test_descriptor_fetch(void *arg) get_options_mutable()->FetchHidServDescriptors = 1; /* 2. We don't have a live consensus. */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus_false); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus_false); ret = hs_client_refetch_hsdesc(&service_pk); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); tt_int_op(ret, OP_EQ, HS_CLIENT_FETCH_MISSING_INFO); /* From now on, return a live consensus. */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* 3. Not enough dir information. */ MOCK(router_have_minimum_dir_info, @@ -615,7 +617,7 @@ test_descriptor_fetch(void *arg) done: connection_free_minimal(ENTRY_TO_CONN(ec)); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); UNMOCK(router_have_minimum_dir_info); hs_free_all(); } @@ -808,8 +810,8 @@ test_desc_has_arrived_cleanup(void *arg) hs_init(); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); MOCK(connection_mark_unattached_ap_, mock_connection_mark_unattached_ap_); MOCK(router_have_minimum_dir_info, @@ -880,7 +882,7 @@ test_desc_has_arrived_cleanup(void *arg) tor_free(desc_str); hs_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); UNMOCK(connection_mark_unattached_ap_); UNMOCK(router_have_minimum_dir_info); } @@ -900,8 +902,8 @@ test_close_intro_circuits_new_desc(void *arg) /* This is needed because of the client cache expiration timestamp is based * on having a consensus. See cached_client_descriptor_has_expired(). */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Set consensus time */ parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", @@ -968,7 +970,7 @@ test_close_intro_circuits_new_desc(void *arg) tt_int_op(ret, OP_EQ, 0); tt_assert(encoded); - hs_cache_store_as_client(encoded, &service_kp.pubkey); + ret = hs_cache_store_as_client(encoded, &service_kp.pubkey); tt_int_op(ret, OP_EQ, 0); tor_free(encoded); tt_assert(hs_cache_lookup_as_client(&service_kp.pubkey)); @@ -983,7 +985,7 @@ test_close_intro_circuits_new_desc(void *arg) hs_descriptor_free(desc1); hs_descriptor_free(desc2); hs_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } struct testcase_t hs_client_tests[] = { diff --git a/src/test/test_hs_common.c b/src/test/test_hs_common.c index 2aff179687..c6104301e1 100644 --- a/src/test/test_hs_common.c +++ b/src/test/test_hs_common.c @@ -359,9 +359,10 @@ mock_networkstatus_get_latest_consensus(void) } static networkstatus_t * -mock_networkstatus_get_live_consensus(time_t now) +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { (void) now; + (void) flavor; tt_assert(mock_ns); @@ -381,6 +382,8 @@ test_responsible_hsdirs(void *arg) MOCK(networkstatus_get_latest_consensus, mock_networkstatus_get_latest_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); ns = networkstatus_get_latest_consensus(); @@ -417,6 +420,8 @@ test_responsible_hsdirs(void *arg) smartlist_clear(ns->routerstatus_list); networkstatus_vote_free(mock_ns); cleanup_nodelist(); + + UNMOCK(networkstatus_get_reasonably_live_consensus); } static void @@ -466,6 +471,8 @@ test_desc_reupload_logic(void *arg) hs_init(); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); MOCK(router_have_minimum_dir_info, mock_router_have_minimum_dir_info); MOCK(get_or_state, @@ -877,9 +884,11 @@ static smartlist_t *service_responsible_hsdirs = NULL; static smartlist_t *client_responsible_hsdirs = NULL; static networkstatus_t * -mock_networkstatus_get_live_consensus_service(time_t now) +mock_networkstatus_get_reasonably_live_consensus_service(time_t now, + int flavor) { (void) now; + (void) flavor; if (mock_service_ns) { return mock_service_ns; @@ -895,13 +904,14 @@ mock_networkstatus_get_live_consensus_service(time_t now) static networkstatus_t * mock_networkstatus_get_latest_consensus_service(void) { - return mock_networkstatus_get_live_consensus_service(0); + return mock_networkstatus_get_reasonably_live_consensus_service(0, 0); } static networkstatus_t * -mock_networkstatus_get_live_consensus_client(time_t now) +mock_networkstatus_get_reasonably_live_consensus_client(time_t now, int flavor) { (void) now; + (void) flavor; if (mock_client_ns) { return mock_client_ns; @@ -917,7 +927,7 @@ mock_networkstatus_get_live_consensus_client(time_t now) static networkstatus_t * mock_networkstatus_get_latest_consensus_client(void) { - return mock_networkstatus_get_live_consensus_client(0); + return mock_networkstatus_get_reasonably_live_consensus_client(0, 0); } /* Mock function because we are not trying to test the close circuit that does @@ -1377,8 +1387,8 @@ run_reachability_scenario(const reachability_cfg_t *cfg, int num_scenario) * === Client setup === */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus_client); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus_client); MOCK(networkstatus_get_latest_consensus, mock_networkstatus_get_latest_consensus_client); @@ -1402,14 +1412,14 @@ run_reachability_scenario(const reachability_cfg_t *cfg, int num_scenario) tt_int_op(smartlist_len(client_responsible_hsdirs), OP_EQ, 6); UNMOCK(networkstatus_get_latest_consensus); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); /* * === Service setup === */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus_service); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus_service); MOCK(networkstatus_get_latest_consensus, mock_networkstatus_get_latest_consensus_service); @@ -1436,7 +1446,7 @@ run_reachability_scenario(const reachability_cfg_t *cfg, int num_scenario) tt_int_op(smartlist_len(service_responsible_hsdirs), OP_EQ, 8); UNMOCK(networkstatus_get_latest_consensus); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); /* Some testing of the values we just got from the client and service. */ tt_mem_op(&client_blinded_pk, OP_EQ, &service_blinded_pk, @@ -1687,8 +1697,8 @@ test_client_service_hsdir_set_sync(void *arg) MOCK(networkstatus_get_latest_consensus, mock_networkstatus_get_latest_consensus); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); MOCK(get_or_state, get_or_state_replacement); MOCK(hs_desc_encode_descriptor, diff --git a/src/test/test_hs_service.c b/src/test/test_hs_service.c index 32b08ecf37..c60ab6c930 100644 --- a/src/test/test_hs_service.c +++ b/src/test/test_hs_service.c @@ -79,9 +79,10 @@ static networkstatus_t mock_ns; static networkstatus_t * -mock_networkstatus_get_live_consensus(time_t now) +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { (void) now; + (void) flavor; return &mock_ns; } @@ -1302,8 +1303,8 @@ test_rotate_descriptors(void *arg) hs_init(); MOCK(get_or_state, get_or_state_replacement); MOCK(circuit_mark_for_close_, mock_circuit_mark_for_close); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Descriptor rotation happens with a consensus with a new SRV. */ @@ -1387,7 +1388,7 @@ test_rotate_descriptors(void *arg) hs_free_all(); UNMOCK(get_or_state); UNMOCK(circuit_mark_for_close_); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } /** Test building descriptors: picking intro points, setting up their link @@ -1408,8 +1409,8 @@ test_build_update_descriptors(void *arg) MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); dummy_state = tor_malloc_zero(sizeof(or_state_t)); @@ -1633,8 +1634,8 @@ test_build_descriptors(void *arg) MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); dummy_state = tor_malloc_zero(sizeof(or_state_t)); @@ -1714,8 +1715,8 @@ test_upload_descriptors(void *arg) hs_init(); MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); dummy_state = tor_malloc_zero(sizeof(or_state_t)); diff --git a/src/test/test_shared_random.c b/src/test/test_shared_random.c index b4fe6eef64..413dfbeb03 100644 --- a/src/test/test_shared_random.c +++ b/src/test/test_shared_random.c @@ -164,6 +164,15 @@ mock_networkstatus_get_live_consensus(time_t now) return &mock_consensus; } +/* Mock function to immediately return our local 'mock_consensus'. */ +static networkstatus_t * +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) +{ + (void) now; + (void) flavor; + return &mock_consensus; +} + static void test_get_state_valid_until_time(void *arg) { @@ -176,6 +185,8 @@ test_get_state_valid_until_time(void *arg) MOCK(networkstatus_get_live_consensus, mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); retval = parse_rfc1123_time("Mon, 20 Apr 2015 01:00:00 UTC", &mock_consensus.fresh_until); @@ -232,7 +243,7 @@ test_get_state_valid_until_time(void *arg) } done: - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } /** Test the function that calculates the start time of the current SRV @@ -248,6 +259,8 @@ test_get_start_time_of_current_run(void *arg) MOCK(networkstatus_get_live_consensus, mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); retval = parse_rfc1123_time("Mon, 20 Apr 2015 01:00:00 UTC", &mock_consensus.fresh_until); @@ -331,6 +344,7 @@ test_get_start_time_of_current_run(void *arg) /* Next test is testing it without a consensus to use the testing voting * interval . */ UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); /* Now let's alter the voting schedule and check the correctness of the * function. Voting interval of 10 seconds, means that an SRV protocol run @@ -362,8 +376,8 @@ test_get_start_time_functions(void *arg) (void) arg; int retval; - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); retval = parse_rfc1123_time("Mon, 20 Apr 2015 01:00:00 UTC", &mock_consensus.fresh_until); @@ -384,7 +398,7 @@ test_get_start_time_functions(void *arg) start_time_of_protocol_run); done: - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } static void

1 0

[tor/maint-0.4.4] hs-v3: Require reasonably live consensus
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit 04b0263974c7ad1327e7a193884cf31d55f7949a Author: David Goulet <dgoulet(a)torproject.org> Date: Mon Jan 11 16:01:22 2021 -0500 hs-v3: Require reasonably live consensus Some days before this commit, the network experienced a DDoS on the directory authorities that prevented them to generate a consensus for more than 5 hours straight. That in turn entirely disabled onion service v3, client and service side, due to the subsystem requiring a live consensus to function properly. We know require a reasonably live consensus which means that the HSv3 subsystem will to its job for using the best consensus tor can find. If the entire network is using an old consensus, than this should be alright. If the service happens to use a live consensus while a client is not, it should still work because the client will use the current SRV it sees which might be the previous SRV for the service for which it still publish descriptors for. If the service is using an old one and somehow can't get a new one while clients are on a new one, then reachability issues might arise. However, this is a situation we already have at the moment since the service will simply not work if it doesn't have a live consensus while a client has one. Fixes #40237 Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- changes/ticket40237 | 5 ++++ src/core/mainloop/mainloop.c | 3 ++- src/feature/hs/hs_cache.c | 5 +++- src/feature/hs/hs_client.c | 8 ++++--- src/feature/hs/hs_common.c | 12 +++++++--- src/feature/hs/hs_service.c | 7 ++++-- src/feature/hs_common/shared_random_client.c | 23 ++++++++++++------ src/feature/nodelist/nodelist.c | 2 +- src/test/test_hs_cache.c | 7 +++--- src/test/test_hs_client.c | 36 +++++++++++++++------------- src/test/test_hs_common.c | 36 ++++++++++++++++++---------- src/test/test_hs_service.c | 21 ++++++++-------- src/test/test_shared_random.c | 22 +++++++++++++---- 13 files changed, 122 insertions(+), 65 deletions(-) diff --git a/changes/ticket40237 b/changes/ticket40237 new file mode 100644 index 0000000000..fc32f59cd4 --- /dev/null +++ b/changes/ticket40237 @@ -0,0 +1,5 @@ + o Major bugfixes (onion service v3): + - Stop requiring a live consensus for v3 clients and services to work. The + use of a reasonably live consensus will allow v3 to work properly in most + cases if the network failed to generate a consensus for more than 2 hours + in a row. Fixes bug 40237; bugfix on 0.3.5.1-alpha. diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index f0aa37e8da..e67e1299b2 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -2511,7 +2511,8 @@ hs_service_callback(time_t now, const or_options_t *options) /* We need to at least be able to build circuits and that we actually have * a working network. */ if (!have_completed_a_circuit() || net_is_disabled() || - networkstatus_get_live_consensus(now) == NULL) { + !networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor())) { goto end; } diff --git a/src/feature/hs/hs_cache.c b/src/feature/hs/hs_cache.c index 05f9940ae6..042ec55fa4 100644 --- a/src/feature/hs/hs_cache.c +++ b/src/feature/hs/hs_cache.c @@ -17,6 +17,7 @@ #include "feature/hs/hs_common.h" #include "feature/hs/hs_client.h" #include "feature/hs/hs_descriptor.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "feature/rend/rendcache.h" @@ -673,7 +674,9 @@ cached_client_descriptor_has_expired(time_t now, /* We use the current consensus time to see if we should expire this * descriptor since we use consensus time for all other parts of the protocol * as well (e.g. to build the blinded key and compute time periods). */ - const networkstatus_t *ns = networkstatus_get_live_consensus(now); + const networkstatus_t *ns = + networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor()); /* If we don't have a recent consensus, consider this entry expired since we * will want to fetch a new HS desc when we get a live consensus. */ if (!ns) { diff --git a/src/feature/hs/hs_client.c b/src/feature/hs/hs_client.c index c65f857419..e25919ecb7 100644 --- a/src/feature/hs/hs_client.c +++ b/src/feature/hs/hs_client.c @@ -29,6 +29,7 @@ #include "feature/hs/hs_descriptor.h" #include "feature/hs/hs_ident.h" #include "feature/nodelist/describe.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "feature/nodelist/nodelist.h" #include "feature/nodelist/routerset.h" @@ -1181,9 +1182,10 @@ can_client_refetch_desc(const ed25519_public_key_t *identity_pk, goto cannot; } - /* Without a live consensus we can't do any client actions. It is needed to - * compute the hashring for a service. */ - if (!networkstatus_get_live_consensus(approx_time())) { + /* Without a usable consensus we can't do any client actions. It is needed + * to compute the hashring for a service. */ + if (!networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor())) { log_info(LD_REND, "Can't fetch descriptor for service %s because we " "are missing a live consensus. Stalling connection.", safe_str_client(ed25519_fmt(identity_pk))); diff --git a/src/feature/hs/hs_common.c b/src/feature/hs/hs_common.c index ebe49f09a5..de653037d1 100644 --- a/src/feature/hs/hs_common.c +++ b/src/feature/hs/hs_common.c @@ -25,6 +25,7 @@ #include "feature/hs/hs_service.h" #include "feature/hs_common/shared_random_client.h" #include "feature/nodelist/describe.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "feature/nodelist/nodelist.h" #include "feature/nodelist/routerset.h" @@ -272,7 +273,9 @@ hs_get_time_period_num(time_t now) if (now != 0) { current_time = now; } else { - networkstatus_t *ns = networkstatus_get_live_consensus(approx_time()); + networkstatus_t *ns = + networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); current_time = ns ? ns->valid_after : approx_time(); } @@ -1098,7 +1101,8 @@ hs_in_period_between_tp_and_srv,(const networkstatus_t *consensus, time_t now)) time_t srv_start_time, tp_start_time; if (!consensus) { - consensus = networkstatus_get_live_consensus(now); + consensus = networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor()); if (!consensus) { return 0; } @@ -1343,7 +1347,9 @@ hs_get_responsible_hsdirs(const ed25519_public_key_t *blinded_pk, sorted_nodes = smartlist_new(); /* Make sure we actually have a live consensus */ - networkstatus_t *c = networkstatus_get_live_consensus(approx_time()); + networkstatus_t *c = + networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); if (!c || smartlist_len(c->routerstatus_list) == 0) { log_warn(LD_REND, "No live consensus so we can't get the responsible " "hidden service directories."); diff --git a/src/feature/hs/hs_service.c b/src/feature/hs/hs_service.c index 6d32cae86c..e820ce9d0b 100644 --- a/src/feature/hs/hs_service.c +++ b/src/feature/hs/hs_service.c @@ -23,6 +23,7 @@ #include "feature/hs_common/shared_random_client.h" #include "feature/keymgt/loadkey.h" #include "feature/nodelist/describe.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "feature/nodelist/nickname.h" #include "feature/nodelist/node_select.h" @@ -2500,7 +2501,8 @@ should_rotate_descriptors(hs_service_t *service, time_t now) tor_assert(service); - ns = networkstatus_get_live_consensus(now); + ns = networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor()); if (ns == NULL) { goto no_rotation; } @@ -3100,7 +3102,8 @@ should_service_upload_descriptor(const hs_service_t *service, } /* Don't upload desc if we don't have a live consensus */ - if (!networkstatus_get_live_consensus(now)) { + if (!networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor())) { goto cannot; } diff --git a/src/feature/hs_common/shared_random_client.c b/src/feature/hs_common/shared_random_client.c index 3d6be94080..ead5d681a9 100644 --- a/src/feature/hs_common/shared_random_client.c +++ b/src/feature/hs_common/shared_random_client.c @@ -13,6 +13,7 @@ #include "app/config/config.h" #include "feature/dircommon/voting_schedule.h" +#include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "lib/encoding/binascii.h" @@ -37,7 +38,9 @@ int get_voting_interval(void) { int interval; - networkstatus_t *consensus = networkstatus_get_live_consensus(time(NULL)); + networkstatus_t *consensus = + networkstatus_get_reasonably_live_consensus(time(NULL), + usable_consensus_flavor()); if (consensus) { interval = (int)(consensus->fresh_until - consensus->valid_after); @@ -142,7 +145,8 @@ sr_get_current(const networkstatus_t *ns) if (ns) { consensus = ns; } else { - consensus = networkstatus_get_live_consensus(approx_time()); + consensus = networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); } /* Ideally we would never be asked for an SRV without a live consensus. Make * sure this assumption is correct. */ @@ -165,7 +169,8 @@ sr_get_previous(const networkstatus_t *ns) if (ns) { consensus = ns; } else { - consensus = networkstatus_get_live_consensus(approx_time()); + consensus = networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); } /* Ideally we would never be asked for an SRV without a live consensus. Make * sure this assumption is correct. */ @@ -237,10 +242,14 @@ sr_state_get_start_time_of_current_protocol_run(void) int voting_interval = get_voting_interval(); time_t beginning_of_curr_round; - /* This function is not used for voting purposes, so if we have a live - consensus, use its valid-after as the beginning of the current round, - otherwise resort to the voting schedule which should always exist. */ - networkstatus_t *ns = networkstatus_get_live_consensus(approx_time()); + /* This function is not used for voting purposes, so if we have a reasonably + * live consensus, use its valid-after as the beginning of the current + * round. If we have no consensus but we're an authority, use our own + * schedule. Otherwise, try using our view of the voting interval to figure + * out when the current round _should_ be starting. */ + networkstatus_t *ns = + networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); if (ns) { beginning_of_curr_round = ns->valid_after; } else { diff --git a/src/feature/nodelist/nodelist.c b/src/feature/nodelist/nodelist.c index 99d7f746a8..8974d95db6 100644 --- a/src/feature/nodelist/nodelist.c +++ b/src/feature/nodelist/nodelist.c @@ -361,7 +361,7 @@ node_set_hsdir_index(node_t *node, const networkstatus_t *ns) tor_assert(node); tor_assert(ns); - if (!networkstatus_is_live(ns, now)) { + if (!networkstatus_consensus_reasonably_live(ns, now)) { static struct ratelim_t live_consensus_ratelim = RATELIM_INIT(30 * 60); log_fn_ratelim(&live_consensus_ratelim, LOG_INFO, LD_GENERAL, "Not setting hsdir index with a non-live consensus."); diff --git a/src/test/test_hs_cache.c b/src/test/test_hs_cache.c index 9182829116..742fa349d9 100644 --- a/src/test/test_hs_cache.c +++ b/src/test/test_hs_cache.c @@ -443,9 +443,10 @@ test_hsdir_revision_counter_check(void *arg) static networkstatus_t mock_ns; static networkstatus_t * -mock_networkstatus_get_live_consensus(time_t now) +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { (void) now; + (void) flavor; return &mock_ns; } @@ -466,8 +467,8 @@ test_client_cache(void *arg) /* Initialize HSDir cache subsystem */ init_test(); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Set consensus time */ parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", diff --git a/src/test/test_hs_client.c b/src/test/test_hs_client.c index 2f2bb45581..53ee3c53d2 100644 --- a/src/test/test_hs_client.c +++ b/src/test/test_hs_client.c @@ -62,16 +62,18 @@ static networkstatus_t mock_ns; /* Always return NULL. */ static networkstatus_t * -mock_networkstatus_get_live_consensus_false(time_t now) +mock_networkstatus_get_reasonably_live_consensus_false(time_t now, int flavor) { (void) now; + (void) flavor; return NULL; } static networkstatus_t * -mock_networkstatus_get_live_consensus(time_t now) +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { (void) now; + (void) flavor; return &mock_ns; } @@ -340,8 +342,8 @@ test_client_pick_intro(void *arg) ed25519_keypair_t service_kp; hs_descriptor_t *desc = NULL; - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); (void) arg; @@ -565,15 +567,15 @@ test_descriptor_fetch(void *arg) get_options_mutable()->FetchHidServDescriptors = 1; /* 2. We don't have a live consensus. */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus_false); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus_false); ret = hs_client_refetch_hsdesc(&service_pk); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); tt_int_op(ret, OP_EQ, HS_CLIENT_FETCH_MISSING_INFO); /* From now on, return a live consensus. */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* 3. Not enough dir information. */ MOCK(router_have_minimum_dir_info, @@ -615,7 +617,7 @@ test_descriptor_fetch(void *arg) done: connection_free_minimal(ENTRY_TO_CONN(ec)); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); UNMOCK(router_have_minimum_dir_info); hs_free_all(); } @@ -808,8 +810,8 @@ test_desc_has_arrived_cleanup(void *arg) hs_init(); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); MOCK(connection_mark_unattached_ap_, mock_connection_mark_unattached_ap_); MOCK(router_have_minimum_dir_info, @@ -880,7 +882,7 @@ test_desc_has_arrived_cleanup(void *arg) tor_free(desc_str); hs_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); UNMOCK(connection_mark_unattached_ap_); UNMOCK(router_have_minimum_dir_info); } @@ -900,8 +902,8 @@ test_close_intro_circuits_new_desc(void *arg) /* This is needed because of the client cache expiration timestamp is based * on having a consensus. See cached_client_descriptor_has_expired(). */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Set consensus time */ parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", @@ -968,7 +970,7 @@ test_close_intro_circuits_new_desc(void *arg) tt_int_op(ret, OP_EQ, 0); tt_assert(encoded); - hs_cache_store_as_client(encoded, &service_kp.pubkey); + ret = hs_cache_store_as_client(encoded, &service_kp.pubkey); tt_int_op(ret, OP_EQ, 0); tor_free(encoded); tt_assert(hs_cache_lookup_as_client(&service_kp.pubkey)); @@ -983,7 +985,7 @@ test_close_intro_circuits_new_desc(void *arg) hs_descriptor_free(desc1); hs_descriptor_free(desc2); hs_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } struct testcase_t hs_client_tests[] = { diff --git a/src/test/test_hs_common.c b/src/test/test_hs_common.c index 2aff179687..c6104301e1 100644 --- a/src/test/test_hs_common.c +++ b/src/test/test_hs_common.c @@ -359,9 +359,10 @@ mock_networkstatus_get_latest_consensus(void) } static networkstatus_t * -mock_networkstatus_get_live_consensus(time_t now) +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { (void) now; + (void) flavor; tt_assert(mock_ns); @@ -381,6 +382,8 @@ test_responsible_hsdirs(void *arg) MOCK(networkstatus_get_latest_consensus, mock_networkstatus_get_latest_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); ns = networkstatus_get_latest_consensus(); @@ -417,6 +420,8 @@ test_responsible_hsdirs(void *arg) smartlist_clear(ns->routerstatus_list); networkstatus_vote_free(mock_ns); cleanup_nodelist(); + + UNMOCK(networkstatus_get_reasonably_live_consensus); } static void @@ -466,6 +471,8 @@ test_desc_reupload_logic(void *arg) hs_init(); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); MOCK(router_have_minimum_dir_info, mock_router_have_minimum_dir_info); MOCK(get_or_state, @@ -877,9 +884,11 @@ static smartlist_t *service_responsible_hsdirs = NULL; static smartlist_t *client_responsible_hsdirs = NULL; static networkstatus_t * -mock_networkstatus_get_live_consensus_service(time_t now) +mock_networkstatus_get_reasonably_live_consensus_service(time_t now, + int flavor) { (void) now; + (void) flavor; if (mock_service_ns) { return mock_service_ns; @@ -895,13 +904,14 @@ mock_networkstatus_get_live_consensus_service(time_t now) static networkstatus_t * mock_networkstatus_get_latest_consensus_service(void) { - return mock_networkstatus_get_live_consensus_service(0); + return mock_networkstatus_get_reasonably_live_consensus_service(0, 0); } static networkstatus_t * -mock_networkstatus_get_live_consensus_client(time_t now) +mock_networkstatus_get_reasonably_live_consensus_client(time_t now, int flavor) { (void) now; + (void) flavor; if (mock_client_ns) { return mock_client_ns; @@ -917,7 +927,7 @@ mock_networkstatus_get_live_consensus_client(time_t now) static networkstatus_t * mock_networkstatus_get_latest_consensus_client(void) { - return mock_networkstatus_get_live_consensus_client(0); + return mock_networkstatus_get_reasonably_live_consensus_client(0, 0); } /* Mock function because we are not trying to test the close circuit that does @@ -1377,8 +1387,8 @@ run_reachability_scenario(const reachability_cfg_t *cfg, int num_scenario) * === Client setup === */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus_client); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus_client); MOCK(networkstatus_get_latest_consensus, mock_networkstatus_get_latest_consensus_client); @@ -1402,14 +1412,14 @@ run_reachability_scenario(const reachability_cfg_t *cfg, int num_scenario) tt_int_op(smartlist_len(client_responsible_hsdirs), OP_EQ, 6); UNMOCK(networkstatus_get_latest_consensus); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); /* * === Service setup === */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus_service); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus_service); MOCK(networkstatus_get_latest_consensus, mock_networkstatus_get_latest_consensus_service); @@ -1436,7 +1446,7 @@ run_reachability_scenario(const reachability_cfg_t *cfg, int num_scenario) tt_int_op(smartlist_len(service_responsible_hsdirs), OP_EQ, 8); UNMOCK(networkstatus_get_latest_consensus); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); /* Some testing of the values we just got from the client and service. */ tt_mem_op(&client_blinded_pk, OP_EQ, &service_blinded_pk, @@ -1687,8 +1697,8 @@ test_client_service_hsdir_set_sync(void *arg) MOCK(networkstatus_get_latest_consensus, mock_networkstatus_get_latest_consensus); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); MOCK(get_or_state, get_or_state_replacement); MOCK(hs_desc_encode_descriptor, diff --git a/src/test/test_hs_service.c b/src/test/test_hs_service.c index 32b08ecf37..c60ab6c930 100644 --- a/src/test/test_hs_service.c +++ b/src/test/test_hs_service.c @@ -79,9 +79,10 @@ static networkstatus_t mock_ns; static networkstatus_t * -mock_networkstatus_get_live_consensus(time_t now) +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) { (void) now; + (void) flavor; return &mock_ns; } @@ -1302,8 +1303,8 @@ test_rotate_descriptors(void *arg) hs_init(); MOCK(get_or_state, get_or_state_replacement); MOCK(circuit_mark_for_close_, mock_circuit_mark_for_close); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Descriptor rotation happens with a consensus with a new SRV. */ @@ -1387,7 +1388,7 @@ test_rotate_descriptors(void *arg) hs_free_all(); UNMOCK(get_or_state); UNMOCK(circuit_mark_for_close_); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } /** Test building descriptors: picking intro points, setting up their link @@ -1408,8 +1409,8 @@ test_build_update_descriptors(void *arg) MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); dummy_state = tor_malloc_zero(sizeof(or_state_t)); @@ -1633,8 +1634,8 @@ test_build_descriptors(void *arg) MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); dummy_state = tor_malloc_zero(sizeof(or_state_t)); @@ -1714,8 +1715,8 @@ test_upload_descriptors(void *arg) hs_init(); MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); dummy_state = tor_malloc_zero(sizeof(or_state_t)); diff --git a/src/test/test_shared_random.c b/src/test/test_shared_random.c index b4fe6eef64..413dfbeb03 100644 --- a/src/test/test_shared_random.c +++ b/src/test/test_shared_random.c @@ -164,6 +164,15 @@ mock_networkstatus_get_live_consensus(time_t now) return &mock_consensus; } +/* Mock function to immediately return our local 'mock_consensus'. */ +static networkstatus_t * +mock_networkstatus_get_reasonably_live_consensus(time_t now, int flavor) +{ + (void) now; + (void) flavor; + return &mock_consensus; +} + static void test_get_state_valid_until_time(void *arg) { @@ -176,6 +185,8 @@ test_get_state_valid_until_time(void *arg) MOCK(networkstatus_get_live_consensus, mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); retval = parse_rfc1123_time("Mon, 20 Apr 2015 01:00:00 UTC", &mock_consensus.fresh_until); @@ -232,7 +243,7 @@ test_get_state_valid_until_time(void *arg) } done: - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } /** Test the function that calculates the start time of the current SRV @@ -248,6 +259,8 @@ test_get_start_time_of_current_run(void *arg) MOCK(networkstatus_get_live_consensus, mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); retval = parse_rfc1123_time("Mon, 20 Apr 2015 01:00:00 UTC", &mock_consensus.fresh_until); @@ -331,6 +344,7 @@ test_get_start_time_of_current_run(void *arg) /* Next test is testing it without a consensus to use the testing voting * interval . */ UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); /* Now let's alter the voting schedule and check the correctness of the * function. Voting interval of 10 seconds, means that an SRV protocol run @@ -362,8 +376,8 @@ test_get_start_time_functions(void *arg) (void) arg; int retval; - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); retval = parse_rfc1123_time("Mon, 20 Apr 2015 01:00:00 UTC", &mock_consensus.fresh_until); @@ -384,7 +398,7 @@ test_get_start_time_functions(void *arg) start_time_of_protocol_run); done: - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } static void

1 0

[tor/maint-0.4.4] Merge branch 'ticket40237_043_01' into ticket40237_044_01
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit b3652f2104c880bf626cb4e97e6088bca5a1571f Merge: 5a822b462a 0485c7ddba Author: David Goulet <dgoulet(a)torproject.org> Date: Tue Jan 12 10:54:31 2021 -0500 Merge branch 'ticket40237_043_01' into ticket40237_044_01 changes/ticket40237 | 5 +++ src/core/mainloop/mainloop.c | 3 +- src/feature/hs/hs_cache.c | 5 ++- src/feature/hs/hs_client.c | 8 +++-- src/feature/hs/hs_common.c | 12 ++++++-- src/feature/hs/hs_service.c | 9 ++++-- src/feature/hs_common/shared_random_client.c | 26 ++++++++++------ src/feature/nodelist/nodelist.c | 2 +- src/test/test_hs_cache.c | 19 ++++++------ src/test/test_hs_client.c | 46 +++++++++++++++------------- src/test/test_hs_common.c | 36 ++++++++++++++-------- src/test/test_hs_service.c | 40 ++++++++++++------------ src/test/test_shared_random.c | 22 ++++++++++--- 13 files changed, 144 insertions(+), 89 deletions(-) diff --cc src/feature/hs/hs_service.c index a42879a48f,54bc572d11..c29f39c6b4 --- a/src/feature/hs/hs_service.c +++ b/src/feature/hs/hs_service.c @@@ -3188,10 -3085,8 +3190,11 @@@ should_service_upload_descriptor(const } /* Don't upload desc if we don't have a live consensus */ - if (!networkstatus_get_live_consensus(now)) { - msg = tor_strdup("No live consensus"); + if (!networkstatus_get_reasonably_live_consensus(now, + usable_consensus_flavor())) { ++ msg = tor_strdup("No reasonably live consensus"); + log_cant_upload_desc(service, desc, msg, + LOG_DESC_UPLOAD_REASON_NO_LIVE_CONSENSUS); goto cannot; } diff --cc src/feature/hs_common/shared_random_client.c index c2ea5afe32,b4b2f2c9d3..4e8a2942fc --- a/src/feature/hs_common/shared_random_client.c +++ b/src/feature/hs_common/shared_random_client.c @@@ -11,8 -11,8 +11,9 @@@ #include "feature/hs_common/shared_random_client.h" #include "app/config/config.h" -#include "feature/dircommon/voting_schedule.h" +#include "feature/dirauth/authmode.h" +#include "feature/dirauth/voting_schedule.h" + #include "feature/nodelist/microdesc.h" #include "feature/nodelist/networkstatus.h" #include "lib/encoding/binascii.h" @@@ -55,25 -37,17 +56,27 @@@ in get_voting_interval(void) { int interval; - networkstatus_t *consensus = networkstatus_get_live_consensus(time(NULL)); + networkstatus_t *consensus = + networkstatus_get_reasonably_live_consensus(time(NULL), + usable_consensus_flavor()); if (consensus) { + /* Ideally we have a live consensus and we can just use that. */ + interval = (int)(consensus->fresh_until - consensus->valid_after); + } else if (authdir_mode(get_options()) || ASSUME_AUTHORITY_SCHEDULING) { + /* If we don't have a live consensus and we're an authority, + * we should believe our own view of what the schedule ought to be. */ + interval = dirauth_sched_get_configured_interval(); + } else if ((consensus = networkstatus_get_latest_consensus())) { + /* If we're a client, then maybe a latest consensus is good enough? + * It's better than falling back to the non-consensus case. */ interval = (int)(consensus->fresh_until - consensus->valid_after); } else { - /* Same for both a testing and real network. We voluntarily ignore the - * InitialVotingInterval since it complexifies things and it doesn't - * affect the SR protocol. */ - interval = get_options()->V3AuthVotingInterval; + /* We should never be reaching this point, since a client should never + * call this code unless they have some kind of a consensus. All we can + * do is hope that this network is using the default voting interval. */ + tor_assert_nonfatal_unreached_once(); + interval = DEFAULT_NETWORK_VOTING_INTERVAL; } tor_assert(interval > 0); return interval; @@@ -242,28 -241,18 +247,29 @@@ sr_state_get_start_time_of_current_prot int voting_interval = get_voting_interval(); time_t beginning_of_curr_round; - /* This function is not used for voting purposes, so if we have a live - consensus, use its valid-after as the beginning of the current round. - If we have no consensus but we're an authority, use our own - schedule. Otherwise, try using our view of the voting interval - to figure out when the current round _should_ be starting. - */ - networkstatus_t *ns = networkstatus_get_live_consensus(approx_time()); + /* This function is not used for voting purposes, so if we have a reasonably + * live consensus, use its valid-after as the beginning of the current + * round. If we have no consensus but we're an authority, use our own + * schedule. Otherwise, try using our view of the voting interval to figure + * out when the current round _should_ be starting. */ + networkstatus_t *ns = + networkstatus_get_reasonably_live_consensus(approx_time(), + usable_consensus_flavor()); if (ns) { beginning_of_curr_round = ns->valid_after; + } else if (authdir_mode(get_options()) || ASSUME_AUTHORITY_SCHEDULING) { + beginning_of_curr_round = dirauth_sched_get_cur_valid_after_time(); } else { - beginning_of_curr_round = get_start_time_of_current_round(); + /* voting_interval comes from get_voting_interval(), so if we're in + * this case as a client, we already tried to get the voting interval + * from the latest_consensus and gave a bug warning if we couldn't. + * + * We wouldn't want to look at the latest consensus's valid_after time, + * since that would be out of date. */ + beginning_of_curr_round = voting_sched_get_start_of_interval_after( + approx_time() - voting_interval, + voting_interval, + 0); } /* Get current SR protocol round */ diff --cc src/test/test_hs_service.c index 80383baff8,49c8d29d27..8b94bb6cf1 --- a/src/test/test_hs_service.c +++ b/src/test/test_hs_service.c @@@ -89,13 -88,6 +90,14 @@@ mock_networkstatus_get_reasonably_live_ return &mock_ns; } +static networkstatus_t * - mock_networkstatus_get_live_consensus_null(time_t now) ++mock_networkstatus_get_reasonably_live_consensus_null(time_t now, int flavor) +{ + (void) now; ++ (void) flavor; + return NULL; +} + static or_state_t *dummy_state = NULL; /* Mock function to get fake or state (used for rev counters) */ @@@ -2199,490 -2170,6 +2201,490 @@@ test_export_client_circuit_id(void *arg tor_free(cp2); } +static smartlist_t * +mock_node_get_link_specifier_smartlist(const node_t *node, bool direct_conn) +{ + (void) node; + (void) direct_conn; + + smartlist_t *lspecs = smartlist_new(); + link_specifier_t *ls_legacy = link_specifier_new(); + smartlist_add(lspecs, ls_legacy); + + return lspecs; +} + +static node_t *fake_node = NULL; + +static const node_t * +mock_build_state_get_exit_node(cpath_build_state_t *state) +{ + (void) state; + + if (!fake_node) { + curve25519_secret_key_t seckey; + curve25519_secret_key_generate(&seckey, 0); + + fake_node = tor_malloc_zero(sizeof(node_t)); + fake_node->ri = tor_malloc_zero(sizeof(routerinfo_t)); + fake_node->ri->onion_curve25519_pkey = + tor_malloc_zero(sizeof(curve25519_public_key_t)); + curve25519_public_key_generate(fake_node->ri->onion_curve25519_pkey, + &seckey); + } + + return fake_node; +} + +static void +mock_launch_rendezvous_point_circuit(const hs_service_t *service, + const hs_service_intro_point_t *ip, + const hs_cell_introduce2_data_t *data) +{ + (void) service; + (void) ip; + (void) data; + return; +} + +/** + * Test that INTRO2 cells are handled well by onion services in the normal + * case and also when onionbalance is enabled. + */ +static void +test_intro2_handling(void *arg) +{ + (void)arg; + + MOCK(build_state_get_exit_node, mock_build_state_get_exit_node); + MOCK(relay_send_command_from_edge_, mock_relay_send_command_from_edge); + MOCK(node_get_link_specifier_smartlist, + mock_node_get_link_specifier_smartlist); + MOCK(launch_rendezvous_point_circuit, mock_launch_rendezvous_point_circuit); + + memset(relay_payload, 0, sizeof(relay_payload)); + + int retval; + time_t now = 0101010101; + update_approx_time(now); + + /** OK this is the play: + * + * In Act I, we have a standalone onion service X (without onionbalance + * enabled). We test that X can properly handle INTRO2 cells sent by a + * client Alice. + * + * In Act II, we create an onionbalance setup with frontend being Z which + * includes instances X and Y. We then setup onionbalance on X and test that + * Alice who addresses Z can communicate with X through INTRO2 cells. + * + * In Act III, we test that Alice can also communicate with X + * directly even tho onionbalance is enabled. + * + * And finally in Act IV, we check various cases where the INTRO2 cell + * should not go through because the subcredentials don't line up + * (e.g. Alice sends INTRO2 to X using Y's subcredential). + */ + + /** Let's start with some setup! Create the instances and the frontend + service, create Alice, etc: */ + + /* Create instance X */ + hs_service_t x_service; + memset(&x_service, 0, sizeof(hs_service_t)); + /* Disable onionbalance */ + x_service.config.ob_master_pubkeys = NULL; + x_service.state.replay_cache_rend_cookie = replaycache_new(0,0); + + /* Create subcredential for x: */ + ed25519_keypair_t x_identity_keypair; + hs_subcredential_t x_subcred; + ed25519_keypair_generate(&x_identity_keypair, 0); + hs_helper_get_subcred_from_identity_keypair(&x_identity_keypair, + &x_subcred); + + /* Create the x instance's intro point */ + hs_service_intro_point_t *x_ip = NULL; + { + curve25519_secret_key_t seckey; + curve25519_public_key_t pkey; + curve25519_secret_key_generate(&seckey, 0); + curve25519_public_key_generate(&pkey, &seckey); + + node_t intro_node; + memset(&intro_node, 0, sizeof(intro_node)); + routerinfo_t ri; + memset(&ri, 0, sizeof(routerinfo_t)); + ri.onion_curve25519_pkey = &pkey; + intro_node.ri = &ri; + + x_ip = service_intro_point_new(&intro_node); + } + + /* Create z frontend's subcredential */ + ed25519_keypair_t z_identity_keypair; + hs_subcredential_t z_subcred; + ed25519_keypair_generate(&z_identity_keypair, 0); + hs_helper_get_subcred_from_identity_keypair(&z_identity_keypair, + &z_subcred); + + /* Create y instance's subcredential */ + ed25519_keypair_t y_identity_keypair; + hs_subcredential_t y_subcred; + ed25519_keypair_generate(&y_identity_keypair, 0); + hs_helper_get_subcred_from_identity_keypair(&y_identity_keypair, + &y_subcred); + + /* Create Alice's intro point */ + hs_desc_intro_point_t *alice_ip; + ed25519_keypair_t signing_kp; + ed25519_keypair_generate(&signing_kp, 0); + alice_ip = hs_helper_build_intro_point(&signing_kp, now, "1.2.3.4", 0, + &x_ip->auth_key_kp, + &x_ip->enc_key_kp); + + /* Create Alice's intro and rend circuits */ + origin_circuit_t *intro_circ = origin_circuit_new(); + intro_circ->cpath = tor_malloc_zero(sizeof(crypt_path_t)); + intro_circ->cpath->prev = intro_circ->cpath; + intro_circ->hs_ident = tor_malloc_zero(sizeof(*intro_circ->hs_ident)); + origin_circuit_t rend_circ; + rend_circ.hs_ident = tor_malloc_zero(sizeof(*rend_circ.hs_ident)); + curve25519_keypair_generate(&rend_circ.hs_ident->rendezvous_client_kp, 0); + memset(rend_circ.hs_ident->rendezvous_cookie, 'r', HS_REND_COOKIE_LEN); + + /* ************************************************************ */ + + /* Act I: + * + * Where Alice connects to X without onionbalance in the picture */ + + /* Create INTRODUCE1 */ + tt_assert(fast_mem_is_zero(relay_payload, sizeof(relay_payload))); + retval = hs_circ_send_introduce1(intro_circ, &rend_circ, + alice_ip, &x_subcred); + + /* Check that the payload was written successfully */ + tt_int_op(retval, OP_EQ, 0); + tt_assert(!fast_mem_is_zero(relay_payload, sizeof(relay_payload))); + tt_int_op(relay_payload_len, OP_NE, 0); + + /* Handle the cell */ + retval = hs_circ_handle_introduce2(&x_service, + intro_circ, x_ip, + &x_subcred, + (uint8_t*)relay_payload,relay_payload_len); + tt_int_op(retval, OP_EQ, 0); + + /* ************************************************************ */ + + /* Act II: + * + * We now create an onionbalance setup with Z being the frontend and X and Y + * being the backend instances. Make sure that Alice can talk with the + * backend instance X even tho she thinks she is talking to the frontend Z. + */ + + /* Now configure the X instance to do onionbalance with Z as the frontend */ + x_service.config.ob_master_pubkeys = smartlist_new(); + smartlist_add(x_service.config.ob_master_pubkeys, + &z_identity_keypair.pubkey); + + /* Create descriptors for x and load next descriptor with the x's + * subcredential so that it can accept connections for itself. */ + x_service.desc_current = service_descriptor_new(); + memset(x_service.desc_current->desc->subcredential.subcred, 'C',SUBCRED_LEN); + x_service.desc_next = service_descriptor_new(); + memcpy(&x_service.desc_next->desc->subcredential, &x_subcred, SUBCRED_LEN); + + /* Refresh OB keys */ + hs_ob_refresh_keys(&x_service); + + /* Create INTRODUCE1 from Alice to X through Z */ + memset(relay_payload, 0, sizeof(relay_payload)); + retval = hs_circ_send_introduce1(intro_circ, &rend_circ, + alice_ip, &z_subcred); + + /* Check that the payload was written successfully */ + tt_int_op(retval, OP_EQ, 0); + tt_assert(!fast_mem_is_zero(relay_payload, sizeof(relay_payload))); + tt_int_op(relay_payload_len, OP_NE, 0); + + /* Deliver INTRODUCE1 to X even tho it carries Z's subcredential */ + replaycache_free(x_service.state.replay_cache_rend_cookie); + x_service.state.replay_cache_rend_cookie = replaycache_new(0, 0); + + retval = hs_circ_handle_introduce2(&x_service, + intro_circ, x_ip, + &z_subcred, + (uint8_t*)relay_payload, relay_payload_len); + tt_int_op(retval, OP_EQ, 0); + + replaycache_free(x_ip->replay_cache); + x_ip->replay_cache = replaycache_new(0, 0); + + replaycache_free(x_service.state.replay_cache_rend_cookie); + x_service.state.replay_cache_rend_cookie = replaycache_new(0, 0); + + /* ************************************************************ */ + + /* Act III: + * + * Now send a direct INTRODUCE cell from Alice to X using X's subcredential + * and check that it succeeds even with onionbalance enabled. + */ + + /* Refresh OB keys (just to check for memleaks) */ + hs_ob_refresh_keys(&x_service); + + /* Create INTRODUCE1 from Alice to X using X's subcred. */ + memset(relay_payload, 0, sizeof(relay_payload)); + retval = hs_circ_send_introduce1(intro_circ, &rend_circ, + alice_ip, &x_subcred); + + /* Check that the payload was written successfully */ + tt_int_op(retval, OP_EQ, 0); + tt_assert(!fast_mem_is_zero(relay_payload, sizeof(relay_payload))); + tt_int_op(relay_payload_len, OP_NE, 0); + + /* Send INTRODUCE1 to X with X's subcredential (should succeed) */ + replaycache_free(x_service.state.replay_cache_rend_cookie); + x_service.state.replay_cache_rend_cookie = replaycache_new(0, 0); + + retval = hs_circ_handle_introduce2(&x_service, + intro_circ, x_ip, + &x_subcred, + (uint8_t*)relay_payload, relay_payload_len); + tt_int_op(retval, OP_EQ, 0); + + /* ************************************************************ */ + + /* Act IV: + * + * Test cases where the INTRO2 cell should not be able to decode. + */ + + /* Try sending the exact same INTRODUCE2 cell again and see that the intro + * point replay cache triggers: */ + setup_full_capture_of_logs(LOG_WARN); + retval = hs_circ_handle_introduce2(&x_service, + intro_circ, x_ip, + &x_subcred, + (uint8_t*)relay_payload, relay_payload_len); + tt_int_op(retval, OP_EQ, -1); + expect_log_msg_containing("with the same ENCRYPTED section"); + teardown_capture_of_logs(); + + /* Now cleanup the intro point replay cache but not the service replay cache + and see that this one triggers this time. */ + replaycache_free(x_ip->replay_cache); + x_ip->replay_cache = replaycache_new(0, 0); + setup_full_capture_of_logs(LOG_INFO); + retval = hs_circ_handle_introduce2(&x_service, + intro_circ, x_ip, + &x_subcred, + (uint8_t*)relay_payload, relay_payload_len); + tt_int_op(retval, OP_EQ, -1); + expect_log_msg_containing("with same REND_COOKIE"); + teardown_capture_of_logs(); + + /* Now just to make sure cleanup both replay caches and make sure that the + cell gets through */ + replaycache_free(x_ip->replay_cache); + x_ip->replay_cache = replaycache_new(0, 0); + replaycache_free(x_service.state.replay_cache_rend_cookie); + x_service.state.replay_cache_rend_cookie = replaycache_new(0, 0); + retval = hs_circ_handle_introduce2(&x_service, + intro_circ, x_ip, + &x_subcred, + (uint8_t*)relay_payload, relay_payload_len); + tt_int_op(retval, OP_EQ, 0); + + /* As a final thing, create an INTRODUCE1 cell from Alice to X using Y's + * subcred (should fail since Y is just another instance and not the frontend + * service!) */ + memset(relay_payload, 0, sizeof(relay_payload)); + retval = hs_circ_send_introduce1(intro_circ, &rend_circ, + alice_ip, &y_subcred); + tt_int_op(retval, OP_EQ, 0); + + /* Check that the payload was written successfully */ + tt_assert(!fast_mem_is_zero(relay_payload, sizeof(relay_payload))); + tt_int_op(relay_payload_len, OP_NE, 0); + + retval = hs_circ_handle_introduce2(&x_service, + intro_circ, x_ip, + &y_subcred, + (uint8_t*)relay_payload, relay_payload_len); + tt_int_op(retval, OP_EQ, -1); + + done: + /* Start cleaning up X */ + replaycache_free(x_service.state.replay_cache_rend_cookie); + smartlist_free(x_service.config.ob_master_pubkeys); + tor_free(x_service.state.ob_subcreds); + service_descriptor_free(x_service.desc_current); + service_descriptor_free(x_service.desc_next); + service_intro_point_free(x_ip); + + /* Clean up Alice */ + hs_desc_intro_point_free(alice_ip); + tor_free(rend_circ.hs_ident); + + if (fake_node) { + tor_free(fake_node->ri->onion_curve25519_pkey); + tor_free(fake_node->ri); + tor_free(fake_node); + } + + UNMOCK(build_state_get_exit_node); + UNMOCK(relay_send_command_from_edge_); + UNMOCK(node_get_link_specifier_smartlist); + UNMOCK(launch_rendezvous_point_circuit); +} + +static void +test_cannot_upload_descriptors(void *arg) +{ + int ret; + time_t now; + hs_service_t *service; + + (void) arg; + + hs_init(); + MOCK(get_or_state, + get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); ++ MOCK(networkstatus_get_reasonably_live_consensus, ++ mock_networkstatus_get_reasonably_live_consensus); + + dummy_state = or_state_new(); + + ret = parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", + &mock_ns.valid_after); + tt_int_op(ret, OP_EQ, 0); + ret = parse_rfc1123_time("Sat, 26 Oct 1985 14:00:00 UTC", + &mock_ns.fresh_until); + tt_int_op(ret, OP_EQ, 0); + dirauth_sched_recalculate_timing(get_options(), mock_ns.valid_after); + + update_approx_time(mock_ns.valid_after + 1); + now = mock_ns.valid_after + 1; + + /* Create a service with no descriptor. It's added to the global map. */ + service = hs_service_new(get_options()); + tt_assert(service); + service->config.version = HS_VERSION_THREE; + ed25519_secret_key_generate(&service->keys.identity_sk, 0); + ed25519_public_key_generate(&service->keys.identity_pk, + &service->keys.identity_sk); + /* Register service to global map. */ + ret = register_service(get_hs_service_map(), service); + tt_int_op(ret, OP_EQ, 0); + /* But first, build our descriptor. */ + build_all_descriptors(now); + + /* 1. Testing missing intro points reason. */ + { + digest256map_t *cur = service->desc_current->intro_points.map; + digest256map_t *tmp = digest256map_new(); + service->desc_current->intro_points.map = tmp; + service->desc_current->missing_intro_points = 1; + setup_full_capture_of_logs(LOG_INFO); + run_upload_descriptor_event(now); + digest256map_free(tmp, tor_free_); + service->desc_current->intro_points.map = cur; + expect_log_msg_containing( + "Service [scrubbed] can't upload its current descriptor: " + "Missing intro points"); + teardown_capture_of_logs(); + /* Reset. */ + service->desc_current->missing_intro_points = 0; + } + + /* 2. Testing non established intro points. */ + { + setup_full_capture_of_logs(LOG_INFO); + run_upload_descriptor_event(now); + expect_log_msg_containing( + "Service [scrubbed] can't upload its current descriptor: " + "Intro circuits aren't yet all established (0/3)."); + teardown_capture_of_logs(); + } + + /* We need to pass the established circuit tests and thus from now on, we + * MOCK this to return 3 intro points. */ + MOCK(count_desc_circuit_established, mock_count_desc_circuit_established); + num_intro_points = 3; + + /* 3. Testing non established intro points. */ + { + service->desc_current->next_upload_time = now + 1000; + setup_full_capture_of_logs(LOG_INFO); + run_upload_descriptor_event(now); + expect_log_msg_containing( + "Service [scrubbed] can't upload its current descriptor: " + "Next upload time is"); + teardown_capture_of_logs(); + /* Reset. */ + service->desc_current->next_upload_time = 0; + } + + /* 4. Testing missing live consensus. */ + { - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus_null); ++ MOCK(networkstatus_get_reasonably_live_consensus, ++ mock_networkstatus_get_reasonably_live_consensus_null); + setup_full_capture_of_logs(LOG_INFO); + run_upload_descriptor_event(now); + expect_log_msg_containing( + "Service [scrubbed] can't upload its current descriptor: " - "No live consensus"); ++ "No reasonably live consensus"); + teardown_capture_of_logs(); + /* Reset. */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); ++ MOCK(networkstatus_get_reasonably_live_consensus, ++ mock_networkstatus_get_reasonably_live_consensus); + } + + /* 5. Test missing minimum directory information. */ + { + MOCK(router_have_minimum_dir_info, + mock_router_have_minimum_dir_info_false); + setup_full_capture_of_logs(LOG_INFO); + run_upload_descriptor_event(now); + expect_log_msg_containing( + "Service [scrubbed] can't upload its current descriptor: " + "Not enough directory information"); + teardown_capture_of_logs(); + + /* Running it again shouldn't trigger anything due to rate limitation. */ + setup_full_capture_of_logs(LOG_INFO); + run_upload_descriptor_event(now); + expect_no_log_entry(); + teardown_capture_of_logs(); + UNMOCK(router_have_minimum_dir_info); + } + + /* Increase time and redo test (5) in order to test the rate limiting. */ + update_approx_time(mock_ns.valid_after + 61); + { + MOCK(router_have_minimum_dir_info, + mock_router_have_minimum_dir_info_false); + setup_full_capture_of_logs(LOG_INFO); + run_upload_descriptor_event(now); + expect_log_msg_containing( + "Service [scrubbed] can't upload its current descriptor: " + "Not enough directory information"); + teardown_capture_of_logs(); + UNMOCK(router_have_minimum_dir_info); + } + + done: + hs_free_all(); + UNMOCK(count_desc_circuit_established); - UNMOCK(networkstatus_get_live_consensus); ++ UNMOCK(networkstatus_get_reasonably_live_consensus); + UNMOCK(get_or_state); +} + struct testcase_t hs_service_tests[] = { { "e2e_rend_circuit_setup", test_e2e_rend_circuit_setup, TT_FORK, NULL, NULL },

1 0

[tor/maint-0.4.4] tests: Fix unit tests after merge of #40237
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit 0485c7ddba8c3474d1be8972caa9505b09d65644 Author: David Goulet <dgoulet(a)torproject.org> Date: Tue Jan 12 10:50:01 2021 -0500 tests: Fix unit tests after merge of #40237 --- src/test/test_hs_cache.c | 12 ++++++------ src/test/test_hs_client.c | 12 ++++++------ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/src/test/test_hs_cache.c b/src/test/test_hs_cache.c index e3be0e77ec..a1df2b0eea 100644 --- a/src/test/test_hs_cache.c +++ b/src/test/test_hs_cache.c @@ -587,8 +587,8 @@ test_client_cache_decrypt(void *arg) /* Initialize HSDir cache subsystem */ hs_init(); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Set consensus time */ parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", @@ -643,7 +643,7 @@ test_client_cache_decrypt(void *arg) hs_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } static void @@ -657,8 +657,8 @@ test_client_cache_remove(void *arg) hs_init(); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Set consensus time. Lookup will not return the entry if it has expired * and it is checked against the consensus valid_after time. */ @@ -696,7 +696,7 @@ test_client_cache_remove(void *arg) hs_descriptor_free(desc1); hs_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } struct testcase_t hs_cache[] = { diff --git a/src/test/test_hs_client.c b/src/test/test_hs_client.c index 32a79a7f49..e797fdedd0 100644 --- a/src/test/test_hs_client.c +++ b/src/test/test_hs_client.c @@ -1121,8 +1121,8 @@ test_close_intro_circuits_cache_clean(void *arg) /* This is needed because of the client cache expiration timestamp is based * on having a consensus. See cached_client_descriptor_has_expired(). */ - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); /* Set consensus time */ parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", @@ -1187,7 +1187,7 @@ test_close_intro_circuits_cache_clean(void *arg) hs_descriptor_free(desc1); hs_free_all(); rend_cache_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } static void @@ -1204,8 +1204,8 @@ test_socks_hs_errors(void *arg) (void) arg; - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); MOCK(connection_mark_unattached_ap_, mock_connection_mark_unattached_ap_no_close); MOCK(read_file_to_str, mock_read_file_to_str); @@ -1285,7 +1285,7 @@ test_socks_hs_errors(void *arg) hs_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); UNMOCK(connection_mark_unattached_ap_); UNMOCK(read_file_to_str); UNMOCK(tor_listdir);

1 0

[tor/maint-0.4.4] Merge branch 'ticket40237_035_01' into ticket40237_043_01
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit 60da5d62225c975842ef57195b7243baa7033acb Merge: 4b39f46a61 04b0263974 Author: David Goulet <dgoulet(a)torproject.org> Date: Tue Jan 12 10:46:25 2021 -0500 Merge branch 'ticket40237_035_01' into ticket40237_043_01 changes/ticket40237 | 5 ++++ src/core/mainloop/mainloop.c | 3 ++- src/feature/hs/hs_cache.c | 5 +++- src/feature/hs/hs_client.c | 8 ++++--- src/feature/hs/hs_common.c | 12 +++++++--- src/feature/hs/hs_service.c | 7 ++++-- src/feature/hs_common/shared_random_client.c | 23 ++++++++++++------ src/feature/nodelist/nodelist.c | 2 +- src/test/test_hs_cache.c | 7 +++--- src/test/test_hs_client.c | 34 +++++++++++++------------- src/test/test_hs_common.c | 36 ++++++++++++++++++---------- src/test/test_hs_service.c | 21 ++++++++-------- src/test/test_shared_random.c | 22 +++++++++++++---- 13 files changed, 121 insertions(+), 64 deletions(-) diff --cc src/test/test_hs_client.c index 945f631459,53ee3c53d2..32a79a7f49 --- a/src/test/test_hs_client.c +++ b/src/test/test_hs_client.c @@@ -1100,382 -985,9 +1102,382 @@@ test_close_intro_circuits_new_desc(voi hs_descriptor_free(desc1); hs_descriptor_free(desc2); hs_free_all(); - UNMOCK(networkstatus_get_live_consensus); + UNMOCK(networkstatus_get_reasonably_live_consensus); } +static void +test_close_intro_circuits_cache_clean(void *arg) +{ + int ret; + ed25519_keypair_t service_kp; + circuit_t *circ = NULL; + origin_circuit_t *ocirc = NULL; + hs_descriptor_t *desc1 = NULL; + + (void) arg; + + hs_init(); + rend_cache_init(); + + /* This is needed because of the client cache expiration timestamp is based + * on having a consensus. See cached_client_descriptor_has_expired(). */ + MOCK(networkstatus_get_live_consensus, + mock_networkstatus_get_live_consensus); + + /* Set consensus time */ + parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", + &mock_ns.valid_after); + parse_rfc1123_time("Sat, 26 Oct 1985 14:00:00 UTC", + &mock_ns.fresh_until); + parse_rfc1123_time("Sat, 26 Oct 1985 16:00:00 UTC", + &mock_ns.valid_until); + + /* Generate service keypair */ + tt_int_op(0, OP_EQ, ed25519_keypair_generate(&service_kp, 0)); + + /* Create and add to the global list a dummy client introduction circuits. + * We'll then make sure the hs_ident is attached to a dummy descriptor. */ + circ = dummy_origin_circuit_new(0); + tt_assert(circ); + circ->purpose = CIRCUIT_PURPOSE_C_INTRODUCING; + ocirc = TO_ORIGIN_CIRCUIT(circ); + + /* Build the first descriptor and cache it. */ + { + char *encoded; + desc1 = hs_helper_build_hs_desc_with_ip(&service_kp); + tt_assert(desc1); + ret = hs_desc_encode_descriptor(desc1, &service_kp, NULL, &encoded); + tt_int_op(ret, OP_EQ, 0); + tt_assert(encoded); + + /* Store it */ + ret = hs_cache_store_as_client(encoded, &service_kp.pubkey); + tt_int_op(ret, OP_EQ, 0); + tor_free(encoded); + tt_assert(hs_cache_lookup_as_client(&service_kp.pubkey)); + } + + /* We'll pick one introduction point and associate it with the circuit. */ + { + const hs_desc_intro_point_t *ip = + smartlist_get(desc1->encrypted_data.intro_points, 0); + tt_assert(ip); + ocirc->hs_ident = hs_ident_circuit_new(&service_kp.pubkey); + ed25519_pubkey_copy(&ocirc->hs_ident->intro_auth_pk, + &ip->auth_key_cert->signed_key); + } + + /* Before we are about to clean up the intro circuits, make sure it is + * actually there. */ + tt_assert(circuit_get_next_intro_circ(NULL, true)); + + /* Cleanup the client cache. The ns valid after time is what decides if the + * descriptor has expired so put it in the future enough (72h) so we are + * sure to always expire. */ + mock_ns.valid_after = approx_time() + (72 * 24 * 60 * 60); + hs_cache_clean_as_client(0); + + /* Once stored, our intro circuit should be closed because it is related to + * an old introduction point that doesn't exists anymore. */ + tt_assert(!circuit_get_next_intro_circ(NULL, true)); + + done: + circuit_free(circ); + hs_descriptor_free(desc1); + hs_free_all(); + rend_cache_free_all(); + UNMOCK(networkstatus_get_live_consensus); +} + +static void +test_socks_hs_errors(void *arg) +{ + int ret; + char *desc_encoded = NULL; + ed25519_keypair_t service_kp; + ed25519_keypair_t signing_kp; + entry_connection_t *socks_conn = NULL; + dir_connection_t *dir_conn = NULL; + hs_descriptor_t *desc = NULL; + uint8_t descriptor_cookie[HS_DESC_DESCRIPTOR_COOKIE_LEN]; + + (void) arg; + + MOCK(networkstatus_get_live_consensus, + mock_networkstatus_get_live_consensus); + MOCK(connection_mark_unattached_ap_, + mock_connection_mark_unattached_ap_no_close); + MOCK(read_file_to_str, mock_read_file_to_str); + MOCK(tor_listdir, mock_tor_listdir); + MOCK(check_private_dir, mock_check_private_dir); + + /* Set consensus time */ + parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", + &mock_ns.valid_after); + parse_rfc1123_time("Sat, 26 Oct 1985 14:00:00 UTC", + &mock_ns.fresh_until); + parse_rfc1123_time("Sat, 26 Oct 1985 16:00:00 UTC", + &mock_ns.valid_until); + + hs_init(); + + ret = ed25519_keypair_generate(&service_kp, 0); + tt_int_op(ret, OP_EQ, 0); + ret = ed25519_keypair_generate(&signing_kp, 0); + tt_int_op(ret, OP_EQ, 0); + + socks_conn = helper_build_socks_connection(&service_kp.pubkey, + AP_CONN_STATE_RENDDESC_WAIT); + tt_assert(socks_conn); + + /* Create directory connection. */ + dir_conn = dir_connection_new(AF_INET); + dir_conn->hs_ident = tor_malloc_zero(sizeof(hs_ident_dir_conn_t)); + TO_CONN(dir_conn)->purpose = DIR_PURPOSE_FETCH_HSDESC; + ed25519_pubkey_copy(&dir_conn->hs_ident->identity_pk, &service_kp.pubkey); + + /* Encode descriptor so we can decode it. */ + desc = hs_helper_build_hs_desc_with_ip(&service_kp); + tt_assert(desc); + + crypto_rand((char *) descriptor_cookie, sizeof(descriptor_cookie)); + ret = hs_desc_encode_descriptor(desc, &service_kp, descriptor_cookie, + &desc_encoded); + tt_int_op(ret, OP_EQ, 0); + tt_assert(desc_encoded); + + /* Try decoding. Point this to an existing descriptor. The following should + * fail thus the desc_out should be set to NULL. */ + hs_descriptor_t *desc_out = desc; + ret = hs_client_decode_descriptor(desc_encoded, &service_kp.pubkey, + &desc_out); + tt_int_op(ret, OP_EQ, HS_DESC_DECODE_NEED_CLIENT_AUTH); + tt_assert(desc_out == NULL); + + /* The caching will fail to decrypt because the descriptor_cookie used above + * is not known to the HS subsystem. This will lead to a missing client + * auth. */ + hs_client_dir_fetch_done(dir_conn, "Reason", desc_encoded, 200); + + tt_int_op(socks_conn->socks_request->socks_extended_error_code, OP_EQ, + SOCKS5_HS_MISSING_CLIENT_AUTH); + + /* Add in the global client auth list bad creds for this service. */ + helper_add_random_client_auth(&service_kp.pubkey); + + ret = hs_client_decode_descriptor(desc_encoded, &service_kp.pubkey, + &desc_out); + tt_int_op(ret, OP_EQ, HS_DESC_DECODE_BAD_CLIENT_AUTH); + tt_assert(desc_out == NULL); + + /* Simmulate a fetch done again. This should replace the cached descriptor + * and signal a bad client authorization. */ + hs_client_dir_fetch_done(dir_conn, "Reason", desc_encoded, 200); + tt_int_op(socks_conn->socks_request->socks_extended_error_code, OP_EQ, + SOCKS5_HS_BAD_CLIENT_AUTH); + + done: + connection_free_minimal(ENTRY_TO_CONN(socks_conn)); + connection_free_minimal(TO_CONN(dir_conn)); + hs_descriptor_free(desc); + tor_free(desc_encoded); + + hs_free_all(); + + UNMOCK(networkstatus_get_live_consensus); + UNMOCK(connection_mark_unattached_ap_); + UNMOCK(read_file_to_str); + UNMOCK(tor_listdir); + UNMOCK(check_private_dir); +} + +static void +test_close_intro_circuit_failure(void *arg) +{ + char digest[DIGEST_LEN]; + circuit_t *circ = NULL; + ed25519_keypair_t service_kp, intro_kp; + origin_circuit_t *ocirc = NULL; + tor_addr_t addr; + const hs_cache_intro_state_t *entry; + + (void) arg; + + hs_init(); + + /* Generate service keypair */ + tt_int_op(0, OP_EQ, ed25519_keypair_generate(&service_kp, 0)); + tt_int_op(0, OP_EQ, ed25519_keypair_generate(&intro_kp, 0)); + + /* Create and add to the global list a dummy client introduction circuit at + * the ACK WAIT state. */ + circ = dummy_origin_circuit_new(0); + tt_assert(circ); + circ->purpose = CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT; + ocirc = TO_ORIGIN_CIRCUIT(circ); + ocirc->hs_ident = hs_ident_circuit_new(&service_kp.pubkey); + ocirc->build_state = tor_malloc_zero(sizeof(cpath_build_state_t)); + /* Code path will log this exit so build it. */ + ocirc->build_state->chosen_exit = extend_info_new("TestNickname", digest, + NULL, NULL, NULL, &addr, + 4242); + ed25519_pubkey_copy(&ocirc->hs_ident->intro_auth_pk, &intro_kp.pubkey); + + /* We'll make for close the circuit for a timeout failure. It should _NOT_ + * end up in the failure cache just yet. We do that on free() only. */ + circuit_mark_for_close(circ, END_CIRC_REASON_TIMEOUT); + tt_assert(!hs_cache_client_intro_state_find(&service_kp.pubkey, + &intro_kp.pubkey)); + /* Time to free. It should get removed. */ + circuit_free(circ); + entry = hs_cache_client_intro_state_find(&service_kp.pubkey, + &intro_kp.pubkey); + tt_assert(entry); + tt_uint_op(entry->timed_out, OP_EQ, 1); + hs_cache_client_intro_state_purge(); + + /* Again, create and add to the global list a dummy client introduction + * circuit at the INTRODUCING state. */ + circ = dummy_origin_circuit_new(0); + tt_assert(circ); + circ->purpose = CIRCUIT_PURPOSE_C_INTRODUCING; + ocirc = TO_ORIGIN_CIRCUIT(circ); + ocirc->hs_ident = hs_ident_circuit_new(&service_kp.pubkey); + ocirc->build_state = tor_malloc_zero(sizeof(cpath_build_state_t)); + /* Code path will log this exit so build it. */ + ocirc->build_state->chosen_exit = extend_info_new("TestNickname", digest, + NULL, NULL, NULL, &addr, + 4242); + ed25519_pubkey_copy(&ocirc->hs_ident->intro_auth_pk, &intro_kp.pubkey); + + /* On free, we should get an unreachable failure. */ + circuit_free(circ); + entry = hs_cache_client_intro_state_find(&service_kp.pubkey, + &intro_kp.pubkey); + tt_assert(entry); + tt_uint_op(entry->unreachable_count, OP_EQ, 1); + hs_cache_client_intro_state_purge(); + + /* Again, create and add to the global list a dummy client introduction + * circuit at the INTRODUCING state but we'll close it for timeout. It + * should not be noted as a timeout failure. */ + circ = dummy_origin_circuit_new(0); + tt_assert(circ); + circ->purpose = CIRCUIT_PURPOSE_C_INTRODUCING; + ocirc = TO_ORIGIN_CIRCUIT(circ); + ocirc->hs_ident = hs_ident_circuit_new(&service_kp.pubkey); + ocirc->build_state = tor_malloc_zero(sizeof(cpath_build_state_t)); + /* Code path will log this exit so build it. */ + ocirc->build_state->chosen_exit = extend_info_new("TestNickname", digest, + NULL, NULL, NULL, &addr, + 4242); + ed25519_pubkey_copy(&ocirc->hs_ident->intro_auth_pk, &intro_kp.pubkey); + + circuit_mark_for_close(circ, END_CIRC_REASON_TIMEOUT); + circuit_free(circ); + tt_assert(!hs_cache_client_intro_state_find(&service_kp.pubkey, + &intro_kp.pubkey)); + + /* Again, create and add to the global list a dummy client introduction + * circuit at the INTRODUCING state but without a chosen_exit. In theory, it + * can not happen but we'll make sure it doesn't end up in the failure cache + * anyway. */ + circ = dummy_origin_circuit_new(0); + tt_assert(circ); + circ->purpose = CIRCUIT_PURPOSE_C_INTRODUCING; + ocirc = TO_ORIGIN_CIRCUIT(circ); + ocirc->hs_ident = hs_ident_circuit_new(&service_kp.pubkey); + ed25519_pubkey_copy(&ocirc->hs_ident->intro_auth_pk, &intro_kp.pubkey); + + circuit_free(circ); + tt_assert(!hs_cache_client_intro_state_find(&service_kp.pubkey, + &intro_kp.pubkey)); + + done: + circuit_free(circ); + hs_free_all(); +} + +static void +test_purge_ephemeral_client_auth(void *arg) +{ + ed25519_keypair_t service_kp; + hs_client_service_authorization_t *auth = NULL; + hs_client_register_auth_status_t status; + + (void) arg; + + /* We will try to write on disk client credentials. */ + MOCK(check_private_dir, mock_check_private_dir); + MOCK(get_options, mock_get_options); + MOCK(write_str_to_file, mock_write_str_to_file); + + /* Boggus directory so when we try to write the permanent client + * authorization data to disk, we don't fail. See + * store_permanent_client_auth_credentials() for more details. */ + mocked_options.ClientOnionAuthDir = tor_strdup("auth_dir"); + + hs_init(); + + /* Generate service keypair */ + tt_int_op(0, OP_EQ, ed25519_keypair_generate(&service_kp, 0)); + + /* Generate a client authorization object. */ + auth = tor_malloc_zero(sizeof(hs_client_service_authorization_t)); + + /* Set it up. No flags meaning it is ephemeral. */ + curve25519_secret_key_generate(&auth->enc_seckey, 0); + hs_build_address(&service_kp.pubkey, HS_VERSION_THREE, auth->onion_address); + auth->flags = 0; + + /* Confirm that there is nothing in the client auth map. It is unallocated + * until we add the first entry. */ + tt_assert(!get_hs_client_auths_map()); + + /* Add an entry to the client auth list. We loose ownership of the auth + * object so nullify it. */ + status = hs_client_register_auth_credentials(auth); + auth = NULL; + tt_int_op(status, OP_EQ, REGISTER_SUCCESS); + + /* We should have the entry now. */ + digest256map_t *client_auths = get_hs_client_auths_map(); + tt_assert(client_auths); + tt_int_op(digest256map_size(client_auths), OP_EQ, 1); + + /* Purge the cache that should remove all ephemeral values. */ + purge_ephemeral_client_auth(); + tt_int_op(digest256map_size(client_auths), OP_EQ, 0); + + /* Now add a new authorization object but permanent. */ + /* Generate a client authorization object. */ + auth = tor_malloc_zero(sizeof(hs_client_service_authorization_t)); + curve25519_secret_key_generate(&auth->enc_seckey, 0); + hs_build_address(&service_kp.pubkey, HS_VERSION_THREE, auth->onion_address); + auth->flags = CLIENT_AUTH_FLAG_IS_PERMANENT; + + /* Add an entry to the client auth list. We loose ownership of the auth + * object so nullify it. */ + status = hs_client_register_auth_credentials(auth); + auth = NULL; + tt_int_op(status, OP_EQ, REGISTER_SUCCESS); + tt_int_op(digest256map_size(client_auths), OP_EQ, 1); + + /* Purge again, the entry should still be there. */ + purge_ephemeral_client_auth(); + tt_int_op(digest256map_size(client_auths), OP_EQ, 1); + + done: + client_service_authorization_free(auth); + hs_free_all(); + tor_free(mocked_options.ClientOnionAuthDir); + + UNMOCK(check_private_dir); + UNMOCK(get_options); + UNMOCK(write_str_to_file); +} + struct testcase_t hs_client_tests[] = { { "e2e_rend_circuit_setup_legacy", test_e2e_rend_circuit_setup_legacy, TT_FORK, NULL, NULL }, diff --cc src/test/test_hs_service.c index e33d593d94,c60ab6c930..49c8d29d27 --- a/src/test/test_hs_service.c +++ b/src/test/test_hs_service.c @@@ -1454,10 -1409,10 +1455,10 @@@ test_build_update_descriptors(void *arg MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); - dummy_state = tor_malloc_zero(sizeof(or_state_t)); + dummy_state = or_state_new(); ret = parse_rfc1123_time("Sat, 26 Oct 1985 03:00:00 UTC", &mock_ns.valid_after); @@@ -1685,10 -1634,10 +1686,10 @@@ test_build_descriptors(void *arg MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); - dummy_state = tor_malloc_zero(sizeof(or_state_t)); + dummy_state = or_state_new(); ret = parse_rfc1123_time("Sat, 26 Oct 1985 03:00:00 UTC", &mock_ns.valid_after); @@@ -1786,10 -1715,10 +1787,10 @@@ test_upload_descriptors(void *arg hs_init(); MOCK(get_or_state, get_or_state_replacement); - MOCK(networkstatus_get_live_consensus, - mock_networkstatus_get_live_consensus); + MOCK(networkstatus_get_reasonably_live_consensus, + mock_networkstatus_get_reasonably_live_consensus); - dummy_state = tor_malloc_zero(sizeof(or_state_t)); + dummy_state = or_state_new(); ret = parse_rfc1123_time("Sat, 26 Oct 1985 13:00:00 UTC", &mock_ns.valid_after);

1 0

[tor/master] statistics: Properly count all rendezvous cells (avoid undercounting).
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit 248cbc2d2207a0bc3374cd18a17cda9d1b8a8ef1 Author: George Kadianakis <desnacked(a)riseup.net> Date: Mon Sep 7 13:17:41 2020 +0300 statistics: Properly count all rendezvous cells (avoid undercounting). tl;dr We were not counting cells flying from the client to the service, but we were counting cells flying from the service to the client. When a rendezvous cell arrives from the client to the RP, the RP forwards it to the service. For this to happen, the cell first passes through command_process_relay_cell() which normally does the statistics counting. However because the `rend_circ` circuit was not flagged with `circuit_carries_hs_traffic_stats` in rend_mid_rendezvous(), the cell is not counted there. Then the cell goes to circuit_receive_relay_cell() which has a special code block based on `rend_splice` specifically for rendezvous cells, and the cell gets directly passed to `rend_circ` via a direct call to circuit_receive_relay_cell(). The cell never passes through command_process_relay_cell() ever again and hence is never counted by our rephist module. The fix here is to flag the `rend_circ` circuit with `circuit_carries_hs_traffic_stats` so that the cell is counted as soon as it hits command_process_relay_cell(). Furthermore we avoid double-counting cells since the special code block of circuit_receive_relay_cell() makes us count rendezvous cells only as they enter the RP and not as they exit it. Fixes #40117. --- changes/bug40117 | 5 +++++ src/feature/rend/rendmid.c | 6 ++++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/changes/bug40117 b/changes/bug40117 new file mode 100644 index 0000000000..77646edf9c --- /dev/null +++ b/changes/bug40117 @@ -0,0 +1,5 @@ + o Major bugfixes (stats, onion services): + - Fix a bug where we were undercounting the Tor network's total onion + service traffic, by only counting rendezvous traffic originating from + services and ignoring any traffic originating from clients. Fixes bug + 40117; bugfix on 0.2.6.2-alpha. diff --git a/src/feature/rend/rendmid.c b/src/feature/rend/rendmid.c index 3ba48f8858..af02b34e6b 100644 --- a/src/feature/rend/rendmid.c +++ b/src/feature/rend/rendmid.c @@ -333,10 +333,12 @@ rend_mid_rendezvous(or_circuit_t *circ, const uint8_t *request, goto err; } - /* Statistics: Mark this circuit as an RP circuit so that we collect - stats from it. */ + /* Statistics: Mark circuits as RP circuits */ if (options->HiddenServiceStatistics) { + /* `circ` is the RP <-> service circuit */ circ->circuit_carries_hs_traffic_stats = 1; + /* `rend_circ` is the client <-> RP circuit */ + rend_circ->circuit_carries_hs_traffic_stats = 1; } /* Send the RENDEZVOUS2 cell to the client. */

1 0

[tor/master] Merge branch 'maint-0.3.5' into maint-0.4.3
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit 6186288eb639e7600c9d7ab482135d44ed78fe21 Merge: a17be1b5b6 045db909c2 Author: David Goulet <dgoulet(a)torproject.org> Date: Thu Jan 28 12:08:20 2021 -0500 Merge branch 'maint-0.3.5' into maint-0.4.3 changes/bug40117 | 5 +++++ src/feature/rend/rendmid.c | 6 ++++-- 2 files changed, 9 insertions(+), 2 deletions(-)

1 0

[tor/release-0.3.5] Merge remote-tracking branch 'tor-gitlab/mr/140' into maint-0.3.5
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit 045db909c2128b11b3ea02fadd0d1fce974e32e7 Merge: c6fb26695b 248cbc2d22 Author: David Goulet <dgoulet(a)torproject.org> Date: Thu Jan 28 12:08:14 2021 -0500 Merge remote-tracking branch 'tor-gitlab/mr/140' into maint-0.3.5 changes/bug40117 | 5 +++++ src/feature/rend/rendmid.c | 6 ++++-- 2 files changed, 9 insertions(+), 2 deletions(-)

1 0

[tor/master] Merge remote-tracking branch 'tor-gitlab/mr/140' into maint-0.3.5
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit 045db909c2128b11b3ea02fadd0d1fce974e32e7 Merge: c6fb26695b 248cbc2d22 Author: David Goulet <dgoulet(a)torproject.org> Date: Thu Jan 28 12:08:14 2021 -0500 Merge remote-tracking branch 'tor-gitlab/mr/140' into maint-0.3.5 changes/bug40117 | 5 +++++ src/feature/rend/rendmid.c | 6 ++++-- 2 files changed, 9 insertions(+), 2 deletions(-)

1 0

[tor/master] Merge branch 'maint-0.4.3' into maint-0.4.4
by dgoulet＠torproject.org 28 Jan '21

28 Jan '21

commit ed0fcfd2b8804e639fc859b7e63e57d7e52747f5 Merge: 737cd79c42 6186288eb6 Author: David Goulet <dgoulet(a)torproject.org> Date: Thu Jan 28 12:08:21 2021 -0500 Merge branch 'maint-0.4.3' into maint-0.4.4

1 0