[tor] branch main updated (bae04e6a98 -> 4f038d224f)
This is an automated email from the git hooks/post-receive script. dgoulet pushed a change to branch main in repository tor. from bae04e6a98 Merge branch 'tor-gitlab/mr/555' new 8345b3bd92 Stop using published_on to decide whether to republish. new db7d067ab1 Retain all routerinfos listed in the consensus. new 08d452b38c Stop using published_on in rs to decide whether to download a routerdesc. new 73639fc3c1 Change a log not to use published_on. new a7fb5563bc Stop checking published_on in routerstatus_has_visibly_changed() new 1d2c918dfd Move published_on from routerstatus_t to vote_routerstatus_t. new 96f1e69f24 Implement proposal 275: don't put "published" times in md consensus new 4f038d224f Merge branch 'tor-gitlab/mr/489' The 8 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: changes/prop275 | 12 ++++++++++ src/feature/dirauth/dirvote.c | 36 +++++++++++++++++++++-------- src/feature/dirauth/dirvote.h | 8 ++++++- src/feature/dirparse/ns_parse.c | 5 +++- src/feature/nodelist/fmt_routerstatus.c | 16 +++++++++++-- src/feature/nodelist/fmt_routerstatus.h | 3 ++- src/feature/nodelist/networkstatus.c | 9 ++------ src/feature/nodelist/routerlist.c | 19 +++++++-------- src/feature/nodelist/routerstatus_st.h | 1 - src/feature/nodelist/vote_routerstatus_st.h | 1 + src/feature/relay/router.c | 2 -- src/test/test_dir.c | 23 +++++++++--------- src/test/test_dir_common.c | 8 +++---- src/test/test_nodelist.c | 8 ++----- src/test/test_router.c | 14 +---------- src/test/test_voting_flags.c | 4 ---- 16 files changed, 96 insertions(+), 73 deletions(-) create mode 100644 changes/prop275 -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. commit 8345b3bd92fb126a65ee096914f84f512d20614a Author: Nick Mathewson <nickm@torproject.org> AuthorDate: Tue Nov 9 08:46:36 2021 -0500 Stop using published_on to decide whether to republish. Thanks to the StaleDesc flag, this is not something we need to look at any longer. --- src/feature/relay/router.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/feature/relay/router.c b/src/feature/relay/router.c index a2ca472307..3daa8688ba 100644 --- a/src/feature/relay/router.c +++ b/src/feature/relay/router.c @@ -2554,8 +2554,6 @@ mark_my_descriptor_dirty_if_too_old(time_t now) rs = networkstatus_vote_find_entry(ns, server_identitykey_digest); if (rs == NULL) retry_fast_reason = "not listed in consensus"; - else if (rs->published_on < slow_cutoff) - retry_fast_reason = "version listed in consensus is quite old"; else if (rs->is_staledesc && ns->valid_after > desc_clean_since) retry_fast_reason = "listed as stale in consensus"; } -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. commit db7d067ab178e90163bd013c3892ca707a846433 Author: Nick Mathewson <nickm@torproject.org> AuthorDate: Tue Nov 9 08:49:07 2021 -0500 Retain all routerinfos listed in the consensus. Previously we'd look at the routerstatus published_on field when deciding what to dump, which really has no point. If something's in the consensus with an ancient published date, then we do want to keep it. --- src/feature/nodelist/routerlist.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/feature/nodelist/routerlist.c b/src/feature/nodelist/routerlist.c index c00f7ffb26..2d6ae13711 100644 --- a/src/feature/nodelist/routerlist.c +++ b/src/feature/nodelist/routerlist.c @@ -1924,11 +1924,9 @@ routerlist_remove_old_routers(void) retain = digestset_new(n_max_retain); } - cutoff = now - OLD_ROUTER_DESC_MAX_AGE; /* Retain anything listed in the consensus. */ if (consensus) { SMARTLIST_FOREACH(consensus->routerstatus_list, routerstatus_t *, rs, - if (rs->published_on >= cutoff) digestset_add(retain, rs->descriptor_digest)); } -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. commit 08d452b38c84c6522b9ec4b0ebae29c5bba6c83d Author: Nick Mathewson <nickm@torproject.org> AuthorDate: Tue Nov 9 08:54:11 2021 -0500 Stop using published_on in rs to decide whether to download a routerdesc. The consensus voters shouldn't actually include such old routers in the consensus anyway, so this logic shouldn't come up... but if a client _does_ download something it wouldn't use, it won't retry infinitely: see checks for WRA_NEVER_DOWNLOADABLE. --- src/feature/nodelist/networkstatus.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/src/feature/nodelist/networkstatus.c b/src/feature/nodelist/networkstatus.c index d57db4c415..d4a37c4db9 100644 --- a/src/feature/nodelist/networkstatus.c +++ b/src/feature/nodelist/networkstatus.c @@ -2615,15 +2615,12 @@ networkstatus_parse_flavor_name(const char *flavname) int client_would_use_router(const routerstatus_t *rs, time_t now) { + (void) now; if (!rs->is_flagged_running) { /* If we had this router descriptor, we wouldn't even bother using it. * (Fetching and storing depends on by we_want_to_fetch_flavor().) */ return 0; } - if (rs->published_on + OLD_ROUTER_DESC_MAX_AGE < now) { - /* We'd drop it immediately for being too old. */ - return 0; - } if (!routerstatus_version_supports_extend2_cells(rs, 1)) { /* We'd ignore it because it doesn't support EXTEND2 cells. * If we don't know the version, download the descriptor so we can -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. commit 73639fc3c1696e3073f51875c71355e5bd0bf656 Author: Nick Mathewson <nickm@torproject.org> AuthorDate: Tue Nov 9 09:07:04 2021 -0500 Change a log not to use published_on. It used to describe when the old and new routerinfos were published when we'd decide to download a routerinfo. Now it describes what their descriptor digests are. --- src/feature/nodelist/routerlist.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/feature/nodelist/routerlist.c b/src/feature/nodelist/routerlist.c index 2d6ae13711..8bcc42bc3f 100644 --- a/src/feature/nodelist/routerlist.c +++ b/src/feature/nodelist/routerlist.c @@ -2719,17 +2719,20 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, continue; /* We would never use it ourself. */ } if (is_vote && source) { - char time_bufnew[ISO_TIME_LEN+1]; - char time_bufold[ISO_TIME_LEN+1]; + char old_digest_buf[HEX_DIGEST_LEN+1]; + const char *old_digest = "none"; const routerinfo_t *oldrouter; oldrouter = router_get_by_id_digest(rs->identity_digest); - format_iso_time(time_bufnew, rs->published_on); - if (oldrouter) - format_iso_time(time_bufold, oldrouter->cache_info.published_on); + if (oldrouter) { + base16_encode(old_digest_buf, sizeof(old_digest_buf), + oldrouter->cache_info.signed_descriptor_digest, + DIGEST_LEN); + old_digest = old_digest_buf; + } log_info(LD_DIR, "Learned about %s (%s vs %s) from %s's vote (%s)", routerstatus_describe(rs), - time_bufnew, - oldrouter ? time_bufold : "none", + hex_str(rs->descriptor_digest, DIGEST_LEN), + old_digest, source->nickname, oldrouter ? "known" : "unknown"); } smartlist_add(downloadable, rs->descriptor_digest); -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. commit a7fb5563bc4902950f57203fd9fff82ec28179c2 Author: Nick Mathewson <nickm@torproject.org> AuthorDate: Tue Nov 9 09:08:48 2021 -0500 Stop checking published_on in routerstatus_has_visibly_changed() This function is only used for the controller; and any time that the published_on time has changed, the digest should also change. --- src/feature/nodelist/networkstatus.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/feature/nodelist/networkstatus.c b/src/feature/nodelist/networkstatus.c index d4a37c4db9..c006ee015e 100644 --- a/src/feature/nodelist/networkstatus.c +++ b/src/feature/nodelist/networkstatus.c @@ -1613,7 +1613,6 @@ routerstatus_has_visibly_changed(const routerstatus_t *a, a->is_hs_dir != b->is_hs_dir || a->is_staledesc != b->is_staledesc || a->has_bandwidth != b->has_bandwidth || - a->published_on != b->published_on || a->ipv6_orport != b->ipv6_orport || a->is_v2_dir != b->is_v2_dir || a->bandwidth_kb != b->bandwidth_kb || -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. commit 1d2c918dfdfc1356890775bc57358b3523830224 Author: Nick Mathewson <nickm@torproject.org> AuthorDate: Tue Nov 9 13:29:36 2021 -0500 Move published_on from routerstatus_t to vote_routerstatus_t. Nothing breaks here, since all non-voting users of routerstatus_t.published_on have been adjusted or removed in previous commits. We have to expand the API of routerstatus_format_entry() a bit, though, so that it can always get a published time as argument, since it can't get it from the routerstatus any more. This should have no effect on voter behavior. --- src/feature/dirauth/dirvote.c | 22 ++++++++++++---------- src/feature/dirparse/ns_parse.c | 5 ++++- src/feature/nodelist/fmt_routerstatus.c | 16 ++++++++++++++-- src/feature/nodelist/fmt_routerstatus.h | 3 ++- src/feature/nodelist/networkstatus.c | 3 +-- src/feature/nodelist/routerstatus_st.h | 1 - src/feature/nodelist/vote_routerstatus_st.h | 1 + src/test/test_dir.c | 23 +++++++++++------------ src/test/test_dir_common.c | 8 ++++---- src/test/test_nodelist.c | 8 ++------ src/test/test_router.c | 14 +------------- src/test/test_voting_flags.c | 4 ---- 12 files changed, 52 insertions(+), 56 deletions(-) diff --git a/src/feature/dirauth/dirvote.c b/src/feature/dirauth/dirvote.c index cdd2c132ef..71f059dbc8 100644 --- a/src/feature/dirauth/dirvote.c +++ b/src/feature/dirauth/dirvote.c @@ -390,7 +390,8 @@ format_networkstatus_vote(crypto_pk_t *private_signing_key, rsf = routerstatus_format_entry(&vrs->status, vrs->version, vrs->protocols, NS_V3_VOTE, - vrs); + vrs, + -1); if (rsf) smartlist_add(chunks, rsf); @@ -618,8 +619,8 @@ compare_vote_rs(const vote_routerstatus_t *a, const vote_routerstatus_t *b) * the descriptor digests matched, so somebody is making SHA1 collisions. */ #define CMP_FIELD(utype, itype, field) do { \ - utype aval = (utype) (itype) a->status.field; \ - utype bval = (utype) (itype) b->status.field; \ + utype aval = (utype) (itype) a->field; \ + utype bval = (utype) (itype) b->field; \ utype u = bval - aval; \ itype r2 = (itype) u; \ if (r2 < 0) { \ @@ -638,8 +639,8 @@ compare_vote_rs(const vote_routerstatus_t *a, const vote_routerstatus_t *b) CMP_EXACT))) { return r; } - CMP_FIELD(unsigned, int, ipv4_orport); - CMP_FIELD(unsigned, int, ipv4_dirport); + CMP_FIELD(unsigned, int, status.ipv4_orport); + CMP_FIELD(unsigned, int, status.ipv4_dirport); return 0; } @@ -692,10 +693,10 @@ compute_routerstatus_consensus(smartlist_t *votes, int consensus_method, } else { if (cur && (cur_n > most_n || (cur_n == most_n && - cur->status.published_on > most_published))) { + cur->published_on > most_published))) { most = cur; most_n = cur_n; - most_published = cur->status.published_on; + most_published = cur->published_on; } cur_n = 1; cur = rs; @@ -703,7 +704,7 @@ compute_routerstatus_consensus(smartlist_t *votes, int consensus_method, } SMARTLIST_FOREACH_END(rs); if (cur_n > most_n || - (cur && cur_n == most_n && cur->status.published_on > most_published)) { + (cur && cur_n == most_n && cur->published_on > most_published)) { most = cur; // most_n = cur_n; // unused after this point. // most_published = cur->status.published_on; // unused after this point. @@ -2047,7 +2048,7 @@ networkstatus_compute_consensus(smartlist_t *votes, memcpy(rs_out.descriptor_digest, rs->status.descriptor_digest, DIGEST_LEN); tor_addr_copy(&rs_out.ipv4_addr, &rs->status.ipv4_addr); - rs_out.published_on = rs->status.published_on; + time_t published_on = rs->published_on; rs_out.ipv4_dirport = rs->status.ipv4_dirport; rs_out.ipv4_orport = rs->status.ipv4_orport; tor_addr_copy(&rs_out.ipv6_addr, &alt_orport.addr); @@ -2276,7 +2277,7 @@ networkstatus_compute_consensus(smartlist_t *votes, /* Okay!! Now we can write the descriptor... */ /* First line goes into "buf". */ buf = routerstatus_format_entry(&rs_out, NULL, NULL, - rs_format, NULL); + rs_format, NULL, published_on); if (buf) smartlist_add(chunks, buf); } @@ -4745,6 +4746,7 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_t *private_key, dirauth_set_routerstatus_from_routerinfo(rs, node, ri, now, list_bad_exits, list_middle_only); + vrs->published_on = ri->cache_info.published_on; if (ri->cache_info.signing_key_cert) { memcpy(vrs->ed25519_id, diff --git a/src/feature/dirparse/ns_parse.c b/src/feature/dirparse/ns_parse.c index cd3e2731be..3e1f9a3bd3 100644 --- a/src/feature/dirparse/ns_parse.c +++ b/src/feature/dirparse/ns_parse.c @@ -371,14 +371,17 @@ routerstatus_parse_entry_from_string(memarea_t *area, } } + time_t published_on; if (tor_snprintf(timebuf, sizeof(timebuf), "%s %s", tok->args[3+offset], tok->args[4+offset]) < 0 || - parse_iso_time(timebuf, &rs->published_on)<0) { + parse_iso_time(timebuf, &published_on)<0) { log_warn(LD_DIR, "Error parsing time '%s %s' [%d %d]", tok->args[3+offset], tok->args[4+offset], offset, (int)flav); goto err; } + if (vote_rs) + vote_rs->published_on = published_on; if (tor_inet_aton(tok->args[5+offset], &in) == 0) { log_warn(LD_DIR, "Error parsing router address in network-status %s", diff --git a/src/feature/nodelist/fmt_routerstatus.c b/src/feature/nodelist/fmt_routerstatus.c index 95379a7721..07b4e63472 100644 --- a/src/feature/nodelist/fmt_routerstatus.c +++ b/src/feature/nodelist/fmt_routerstatus.c @@ -26,6 +26,9 @@ /** Helper: write the router-status information in <b>rs</b> into a newly * allocated character buffer. Use the same format as in network-status * documents. If <b>version</b> is non-NULL, add a "v" line for the platform. + * If <b>declared_publish_time</b> is nonnegative, we declare it as the + * publication time. Otherwise we look for a publication time in <b>vrs</b>, + * and fall back to a default (not useful) publication time. * * Return 0 on success, -1 on failure. * @@ -38,12 +41,14 @@ * NS_V3_VOTE - Output a complete V3 NS vote. If <b>vrs</b> is present, * it contains additional information for the vote. * NS_CONTROL_PORT - Output a NS document for the control port. + * */ char * routerstatus_format_entry(const routerstatus_t *rs, const char *version, const char *protocols, routerstatus_format_type_t format, - const vote_routerstatus_t *vrs) + const vote_routerstatus_t *vrs, + time_t declared_publish_time) { char *summary; char *result = NULL; @@ -53,11 +58,18 @@ routerstatus_format_entry(const routerstatus_t *rs, const char *version, char digest64[BASE64_DIGEST_LEN+1]; smartlist_t *chunks = smartlist_new(); + if (declared_publish_time != -1) { + format_iso_time(published, declared_publish_time); + } else if (vrs) { + format_iso_time(published, vrs->published_on); + } else { + strlcpy(published, "2038-01-01 00:00:00", sizeof(published)); + } + const char *ip_str = fmt_addr(&rs->ipv4_addr); if (ip_str[0] == '\0') goto err; - format_iso_time(published, rs->published_on); digest_to_base64(identity64, rs->identity_digest); digest_to_base64(digest64, rs->descriptor_digest); diff --git a/src/feature/nodelist/fmt_routerstatus.h b/src/feature/nodelist/fmt_routerstatus.h index 7482f373e1..740ea51dd9 100644 --- a/src/feature/nodelist/fmt_routerstatus.h +++ b/src/feature/nodelist/fmt_routerstatus.h @@ -35,6 +35,7 @@ char *routerstatus_format_entry( const char *version, const char *protocols, routerstatus_format_type_t format, - const vote_routerstatus_t *vrs); + const vote_routerstatus_t *vrs, + time_t declared_publish_time); #endif /* !defined(TOR_FMT_ROUTERSTATUS_H) */ diff --git a/src/feature/nodelist/networkstatus.c b/src/feature/nodelist/networkstatus.c index c006ee015e..f7b3933b84 100644 --- a/src/feature/nodelist/networkstatus.c +++ b/src/feature/nodelist/networkstatus.c @@ -2364,7 +2364,7 @@ char * networkstatus_getinfo_helper_single(const routerstatus_t *rs) { return routerstatus_format_entry(rs, NULL, NULL, NS_CONTROL_PORT, - NULL); + NULL, -1); } /** @@ -2396,7 +2396,6 @@ set_routerstatus_from_routerinfo(routerstatus_t *rs, rs->is_hs_dir = node->is_hs_dir; rs->is_named = rs->is_unnamed = 0; - rs->published_on = ri->cache_info.published_on; memcpy(rs->identity_digest, node->identity, DIGEST_LEN); memcpy(rs->descriptor_digest, ri->cache_info.signed_descriptor_digest, DIGEST_LEN); diff --git a/src/feature/nodelist/routerstatus_st.h b/src/feature/nodelist/routerstatus_st.h index 55b76de581..a36c80917c 100644 --- a/src/feature/nodelist/routerstatus_st.h +++ b/src/feature/nodelist/routerstatus_st.h @@ -21,7 +21,6 @@ struct routerstatus_t { * routerstatus_has_visibly_changed and the printing function * routerstatus_format_entry in NS_CONTROL_PORT mode. */ - time_t published_on; /**< When was this router published? */ char nickname[MAX_NICKNAME_LEN+1]; /**< The nickname this router says it * has. */ char identity_digest[DIGEST_LEN]; /**< Digest of the router's identity diff --git a/src/feature/nodelist/vote_routerstatus_st.h b/src/feature/nodelist/vote_routerstatus_st.h index 6b2f7b92a9..41d465db8f 100644 --- a/src/feature/nodelist/vote_routerstatus_st.h +++ b/src/feature/nodelist/vote_routerstatus_st.h @@ -18,6 +18,7 @@ struct vote_routerstatus_t { routerstatus_t status; /**< Underlying 'status' object for this router. * Flags are redundant. */ + time_t published_on; /**< When was this router published? */ /** How many known-flags are allowed in a vote? This is the width of * the flags field of vote_routerstatus_t */ #define MAX_KNOWN_FLAGS_IN_VOTE 64 diff --git a/src/test/test_dir.c b/src/test/test_dir.c index 186e09f236..248fd8ab5d 100644 --- a/src/test/test_dir.c +++ b/src/test/test_dir.c @@ -2971,7 +2971,7 @@ test_vrs_for_v3ns(vote_routerstatus_t *vrs, int voter, time_t now) (voter == 1)) { /* Check the first routerstatus. */ tt_str_op(vrs->version,OP_EQ, "0.1.2.14"); - tt_int_op(rs->published_on,OP_EQ, now-1500); + tt_int_op(vrs->published_on,OP_EQ, now-1500); tt_str_op(rs->nickname,OP_EQ, "router2"); tt_mem_op(rs->identity_digest,OP_EQ, "\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3" @@ -2996,7 +2996,7 @@ test_vrs_for_v3ns(vote_routerstatus_t *vrs, int voter, time_t now) if (voter == 1) { /* Check the second routerstatus. */ tt_str_op(vrs->version,OP_EQ, "0.2.0.5"); - tt_int_op(rs->published_on,OP_EQ, now-1000); + tt_int_op(vrs->published_on,OP_EQ, now-1000); tt_str_op(rs->nickname,OP_EQ, "router1"); } tt_mem_op(rs->descriptor_digest,OP_EQ, "MMMMMMMMMMMMMMMMMMMM", DIGEST_LEN); @@ -3057,6 +3057,7 @@ test_consensus_for_v3ns(networkstatus_t *con, time_t now) static void test_routerstatus_for_v3ns(routerstatus_t *rs, time_t now) { + (void)now; tor_addr_t addr_ipv6; tt_assert(rs); @@ -3093,7 +3094,6 @@ test_routerstatus_for_v3ns(routerstatus_t *rs, time_t now) DIGEST_LEN); tt_str_op(rs->nickname,OP_EQ, "router1"); tt_mem_op(rs->descriptor_digest,OP_EQ, "MMMMMMMMMMMMMMMMMMMM", DIGEST_LEN); - tt_int_op(rs->published_on,OP_EQ, now-1000); tt_assert(tor_addr_eq_ipv4h(&rs->ipv4_addr, 0x99009901)); tt_int_op(rs->ipv4_orport,OP_EQ, 443); tt_int_op(rs->ipv4_dirport,OP_EQ, 0); @@ -3968,7 +3968,7 @@ gen_routerstatus_for_umbw(int idx, time_t now) vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; vrs->version = tor_strdup("0.1.2.14"); - rs->published_on = now-1500; + vrs->published_on = now-1500; strlcpy(rs->nickname, "router2", sizeof(rs->nickname)); memset(rs->identity_digest, 3, DIGEST_LEN); memset(rs->descriptor_digest, 78, DIGEST_LEN); @@ -3993,7 +3993,7 @@ gen_routerstatus_for_umbw(int idx, time_t now) vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; vrs->version = tor_strdup("0.2.0.5"); - rs->published_on = now-1000; + vrs->published_on = now-1000; strlcpy(rs->nickname, "router1", sizeof(rs->nickname)); memset(rs->identity_digest, 5, DIGEST_LEN); memset(rs->descriptor_digest, 77, DIGEST_LEN); @@ -4020,7 +4020,7 @@ gen_routerstatus_for_umbw(int idx, time_t now) vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; vrs->version = tor_strdup("0.1.0.3"); - rs->published_on = now-1000; + vrs->published_on = now-1000; strlcpy(rs->nickname, "router3", sizeof(rs->nickname)); memset(rs->identity_digest, 0x33, DIGEST_LEN); memset(rs->descriptor_digest, 79, DIGEST_LEN); @@ -4046,7 +4046,7 @@ gen_routerstatus_for_umbw(int idx, time_t now) vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; vrs->version = tor_strdup("0.1.6.3"); - rs->published_on = now-1000; + vrs->published_on = now-1000; strlcpy(rs->nickname, "router4", sizeof(rs->nickname)); memset(rs->identity_digest, 0x34, DIGEST_LEN); memset(rs->descriptor_digest, 47, DIGEST_LEN); @@ -4146,7 +4146,7 @@ test_vrs_for_umbw(vote_routerstatus_t *vrs, int voter, time_t now) * cutoff. */ tt_str_op(vrs->version,OP_EQ, "0.1.2.14"); - tt_int_op(rs->published_on,OP_EQ, now-1500); + tt_int_op(vrs->published_on,OP_EQ, now-1500); tt_str_op(rs->nickname,OP_EQ, "router2"); tt_mem_op(rs->identity_digest,OP_EQ, "\x3\x3\x3\x3\x3\x3\x3\x3\x3\x3" @@ -4170,7 +4170,7 @@ test_vrs_for_umbw(vote_routerstatus_t *vrs, int voter, time_t now) * cutoff. */ tt_str_op(vrs->version,OP_EQ, "0.2.0.5"); - tt_int_op(rs->published_on,OP_EQ, now-1000); + tt_int_op(vrs->published_on,OP_EQ, now-1000); tt_str_op(rs->nickname,OP_EQ, "router1"); tt_mem_op(rs->identity_digest,OP_EQ, "\x5\x5\x5\x5\x5\x5\x5\x5\x5\x5" @@ -4245,6 +4245,7 @@ test_consensus_for_umbw(networkstatus_t *con, time_t now) static void test_routerstatus_for_umbw(routerstatus_t *rs, time_t now) { + (void)now; tor_addr_t addr_ipv6; uint32_t max_unmeasured_bw_kb = (alternate_clip_bw > 0) ? alternate_clip_bw : DEFAULT_MAX_UNMEASURED_BW_KB; @@ -4285,7 +4286,6 @@ test_routerstatus_for_umbw(routerstatus_t *rs, time_t now) DIGEST_LEN); tt_str_op(rs->nickname,OP_EQ, "router1"); tt_mem_op(rs->descriptor_digest,OP_EQ, "MMMMMMMMMMMMMMMMMMMM", DIGEST_LEN); - tt_int_op(rs->published_on,OP_EQ, now-1000); tt_assert(tor_addr_eq_ipv4h(&rs->ipv4_addr, 0x99009901)); tt_int_op(rs->ipv4_orport,OP_EQ, 443); tt_int_op(rs->ipv4_dirport,OP_EQ, 0); @@ -4385,7 +4385,6 @@ test_dir_fmt_control_ns(void *arg) (void)arg; memset(&rs, 0, sizeof(rs)); - rs.published_on = 1364925198; strlcpy(rs.nickname, "TetsuoMilk", sizeof(rs.nickname)); memcpy(rs.identity_digest, "Stately, plump Buck ", DIGEST_LEN); memcpy(rs.descriptor_digest, "Mulligan came up fro", DIGEST_LEN); @@ -4403,7 +4402,7 @@ test_dir_fmt_control_ns(void *arg) tt_assert(s); tt_str_op(s, OP_EQ, "r TetsuoMilk U3RhdGVseSwgcGx1bXAgQnVjayA " - "TXVsbGlnYW4gY2FtZSB1cCBmcm8 2013-04-02 17:53:18 " + "TXVsbGlnYW4gY2FtZSB1cCBmcm8 2038-01-01 00:00:00 " "32.48.64.80 9001 9002\n" "s Exit Fast Running V2Dir\n" "w Bandwidth=1000\n"); diff --git a/src/test/test_dir_common.c b/src/test/test_dir_common.c index 201ea900ff..50ba32b562 100644 --- a/src/test/test_dir_common.c +++ b/src/test/test_dir_common.c @@ -93,7 +93,7 @@ dir_common_gen_routerstatus_for_v3ns(int idx, time_t now) vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; vrs->version = tor_strdup("0.1.2.14"); - rs->published_on = now-1500; + vrs->published_on = now-1500; strlcpy(rs->nickname, "router2", sizeof(rs->nickname)); memset(rs->identity_digest, TEST_DIR_ROUTER_ID_1, DIGEST_LEN); memset(rs->descriptor_digest, TEST_DIR_ROUTER_DD_1, DIGEST_LEN); @@ -111,7 +111,7 @@ dir_common_gen_routerstatus_for_v3ns(int idx, time_t now) vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; vrs->version = tor_strdup("0.2.0.5"); - rs->published_on = now-1000; + vrs->published_on = now-1000; strlcpy(rs->nickname, "router1", sizeof(rs->nickname)); memset(rs->identity_digest, TEST_DIR_ROUTER_ID_2, DIGEST_LEN); memset(rs->descriptor_digest, TEST_DIR_ROUTER_DD_2, DIGEST_LEN); @@ -130,7 +130,7 @@ dir_common_gen_routerstatus_for_v3ns(int idx, time_t now) vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; vrs->version = tor_strdup("0.1.0.3"); - rs->published_on = now-1000; + vrs->published_on = now-1000; strlcpy(rs->nickname, "router3", sizeof(rs->nickname)); memset(rs->identity_digest, TEST_DIR_ROUTER_ID_3, DIGEST_LEN); memset(rs->descriptor_digest, TEST_DIR_ROUTER_DD_3, DIGEST_LEN); @@ -147,7 +147,7 @@ dir_common_gen_routerstatus_for_v3ns(int idx, time_t now) vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); rs = &vrs->status; vrs->version = tor_strdup("0.1.6.3"); - rs->published_on = now-1000; + vrs->published_on = now-1000; strlcpy(rs->nickname, "router4", sizeof(rs->nickname)); memset(rs->identity_digest, TEST_DIR_ROUTER_ID_4, DIGEST_LEN); memset(rs->descriptor_digest, TEST_DIR_ROUTER_DD_4, DIGEST_LEN); diff --git a/src/test/test_nodelist.c b/src/test/test_nodelist.c index 250db9a964..ecd29f5464 100644 --- a/src/test/test_nodelist.c +++ b/src/test/test_nodelist.c @@ -1273,7 +1273,6 @@ test_nodelist_routerstatus_has_visibly_changed(void *arg) memcpy(rs_orig.descriptor_digest, "abcdefghijklmnopqrst", 20); tor_addr_from_ipv4h(&rs_orig.ipv4_addr, 0x7f000001); rs_orig.ipv4_orport = 3; - rs_orig.published_on = time(NULL); rs_orig.has_bandwidth = 1; rs_orig.bandwidth_kb = 20; @@ -1284,9 +1283,9 @@ test_nodelist_routerstatus_has_visibly_changed(void *arg) tor_free(fmt); \ fmt_orig = routerstatus_format_entry(&rs_orig, NULL, NULL, \ NS_CONTROL_PORT, \ - NULL); \ + NULL, -1); \ fmt = routerstatus_format_entry(&rs, NULL, NULL, NS_CONTROL_PORT, \ - NULL); \ + NULL, -1); \ tt_assert(fmt_orig); \ tt_assert(fmt); \ STMT_END @@ -1322,9 +1321,6 @@ test_nodelist_routerstatus_has_visibly_changed(void *arg) strlcpy(rs.nickname, "fr1end1y", sizeof(rs.nickname)); ASSERT_CHANGED(); - rs.published_on += 3600; - ASSERT_CHANGED(); - rs.ipv4_orport = 55; ASSERT_CHANGED(); diff --git a/src/test/test_router.c b/src/test/test_router.c index 15cc93fbfc..47084bba01 100644 --- a/src/test/test_router.c +++ b/src/test/test_router.c @@ -282,7 +282,6 @@ test_router_mark_if_too_old(void *arg) mock_ns = &ns; mock_ns->valid_after = now-3600; mock_rs = &rs; - mock_rs->published_on = now - 10; // no reason to mark this time. desc_clean_since = now-10; @@ -302,25 +301,14 @@ test_router_mark_if_too_old(void *arg) tt_i64_op(desc_clean_since, OP_EQ, 0); tt_str_op(desc_dirty_reason, OP_EQ, "time for new descriptor"); - // Version in consensus published a long time ago? We won't mark it - // if it's been clean for only a short time. desc_clean_since = now - 10; desc_dirty_reason = NULL; - mock_rs->published_on = now - 3600 * 96; mark_my_descriptor_dirty_if_too_old(now); tt_i64_op(desc_clean_since, OP_EQ, now - 10); - // ... but if it's been clean a while, we mark. - desc_clean_since = now - 2 * 3600; - mark_my_descriptor_dirty_if_too_old(now); - tt_i64_op(desc_clean_since, OP_EQ, 0); - tt_str_op(desc_dirty_reason, OP_EQ, - "version listed in consensus is quite old"); - - // same deal if we're marked stale. + // Version in consensus marked as stale? We'll mark it. desc_clean_since = now - 2 * 3600; desc_dirty_reason = NULL; - mock_rs->published_on = now - 10; mock_rs->is_staledesc = 1; mark_my_descriptor_dirty_if_too_old(now); tt_i64_op(desc_clean_since, OP_EQ, 0); diff --git a/src/test/test_voting_flags.c b/src/test/test_voting_flags.c index 457b0fa796..a5b1248cc1 100644 --- a/src/test/test_voting_flags.c +++ b/src/test/test_voting_flags.c @@ -40,7 +40,6 @@ setup_cfg(flag_vote_test_cfg_t *c) memset(c->ri.cache_info.signed_descriptor_digest, 0xee, DIGEST_LEN); c->ri.cache_info.published_on = c->now - 100; - c->expected.published_on = c->now - 100; tor_addr_from_ipv4h(&c->ri.ipv4_addr, 0x7f010105); tor_addr_from_ipv4h(&c->expected.ipv4_addr, 0x7f010105); @@ -65,7 +64,6 @@ check_result(flag_vote_test_cfg_t *c) dirauth_set_routerstatus_from_routerinfo(&rs, &c->node, &c->ri, c->now, 0, 0); - tt_i64_op(rs.published_on, OP_EQ, c->expected.published_on); tt_str_op(rs.nickname, OP_EQ, c->expected.nickname); // identity_digest and descriptor_digest are not set here. @@ -144,13 +142,11 @@ test_voting_flags_staledesc(void *arg) time_t now = cfg->now; cfg->ri.cache_info.published_on = now - DESC_IS_STALE_INTERVAL + 10; - cfg->expected.published_on = now - DESC_IS_STALE_INTERVAL + 10; // no change in expectations for is_staledesc if (!check_result(cfg)) goto done; cfg->ri.cache_info.published_on = now - DESC_IS_STALE_INTERVAL - 10; - cfg->expected.published_on = now - DESC_IS_STALE_INTERVAL - 10; cfg->expected.is_staledesc = 1; if (!check_result(cfg)) goto done; -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. commit 96f1e69f24ec5c056964b44cb8538004649c504d Author: Nick Mathewson <nickm@torproject.org> AuthorDate: Tue Nov 9 13:43:48 2021 -0500 Implement proposal 275: don't put "published" times in md consensus When a new consensus method is negotiated, these values will all get replaced with "2038-01-01 00:00:00". This change should be safe because: * As of 0.2.9.11 / 0.3.0.7 / 0.3.1.1-alpha, Tor takes no action about published_on times in the future. * The only remaining parties relying on published_on values are (we believe) relays running 0.3.5.x, which rely on the values in a NS consensus to see whether their descriptors are out of date. But this patch only changes microdesc consensuses. * The latest Tor no longer looks at this field in consensuses. Why make this change? In experiments, replacing these values with a fixed value made the size of compressed consensus diffs much much smaller. (Like, by over 50%!) Implements proposal 275; Implements #40130. --- changes/prop275 | 12 ++++++++++++ src/feature/dirauth/dirvote.c | 16 +++++++++++++++- src/feature/dirauth/dirvote.h | 8 +++++++- src/feature/nodelist/fmt_routerstatus.c | 2 +- 4 files changed, 35 insertions(+), 3 deletions(-) diff --git a/changes/prop275 b/changes/prop275 new file mode 100644 index 0000000000..bbbf38d959 --- /dev/null +++ b/changes/prop275 @@ -0,0 +1,12 @@ + o Minor features (directory authority): + - Add a new consensus method in which the "published" times on router + entries in a microdesc consensus are all set to a meaningless fixed + date. Doing this will make the download size for compressed microdesc + consensus diffs much smaller. + Part of ticket 40130; implements proposal 275. + + o Minor features (network documents): + - Clients and relays no longer track the "published on" time declared + for relays in any consensus documents. When reporting this time on + the control port, they instead report a fixed date in the future. + Part of ticket 40130. diff --git a/src/feature/dirauth/dirvote.c b/src/feature/dirauth/dirvote.c index 71f059dbc8..b4a9f83f19 100644 --- a/src/feature/dirauth/dirvote.c +++ b/src/feature/dirauth/dirvote.c @@ -2048,7 +2048,6 @@ networkstatus_compute_consensus(smartlist_t *votes, memcpy(rs_out.descriptor_digest, rs->status.descriptor_digest, DIGEST_LEN); tor_addr_copy(&rs_out.ipv4_addr, &rs->status.ipv4_addr); - time_t published_on = rs->published_on; rs_out.ipv4_dirport = rs->status.ipv4_dirport; rs_out.ipv4_orport = rs->status.ipv4_orport; tor_addr_copy(&rs_out.ipv6_addr, &alt_orport.addr); @@ -2056,6 +2055,21 @@ networkstatus_compute_consensus(smartlist_t *votes, rs_out.has_bandwidth = 0; rs_out.has_exitsummary = 0; + time_t published_on = rs->published_on; + + /* Starting with this consensus method, we no longer include a + meaningful published_on time for microdescriptor consensuses. This + makes their diffs smaller and more compressible. + + We need to keep including a meaningful published_on time for NS + consensuses, however, until 035 relays are all obsolete. (They use + it for a purpose similar to the current StaleDesc flag.) + */ + if (consensus_method >= MIN_METHOD_TO_SUPPRESS_MD_PUBLISHED && + flavor == FLAV_MICRODESC) { + published_on = -1; + } + if (chosen_name && !naming_conflict) { strlcpy(rs_out.nickname, chosen_name, sizeof(rs_out.nickname)); } else { diff --git a/src/feature/dirauth/dirvote.h b/src/feature/dirauth/dirvote.h index 64aaec116e..ae8d43a6f0 100644 --- a/src/feature/dirauth/dirvote.h +++ b/src/feature/dirauth/dirvote.h @@ -53,7 +53,7 @@ #define MIN_SUPPORTED_CONSENSUS_METHOD 28 /** The highest consensus method that we currently support. */ -#define MAX_SUPPORTED_CONSENSUS_METHOD 32 +#define MAX_SUPPORTED_CONSENSUS_METHOD 33 /** * Lowest consensus method where microdescriptor lines are put in canonical @@ -74,6 +74,12 @@ */ #define MIN_METHOD_FOR_MIDDLEONLY 32 +/** + * Lowest consensus method for which we suppress the published time in + * microdescriptor consensuses. + */ +#define MIN_METHOD_TO_SUPPRESS_MD_PUBLISHED 33 + /** Default bandwidth to clip unmeasured bandwidths to using method >= * MIN_METHOD_TO_CLIP_UNMEASURED_BW. (This is not a consensus method; do not * get confused with the above macros.) */ diff --git a/src/feature/nodelist/fmt_routerstatus.c b/src/feature/nodelist/fmt_routerstatus.c index 07b4e63472..e068c87c9b 100644 --- a/src/feature/nodelist/fmt_routerstatus.c +++ b/src/feature/nodelist/fmt_routerstatus.c @@ -58,7 +58,7 @@ routerstatus_format_entry(const routerstatus_t *rs, const char *version, char digest64[BASE64_DIGEST_LEN+1]; smartlist_t *chunks = smartlist_new(); - if (declared_publish_time != -1) { + if (declared_publish_time >= 0) { format_iso_time(published, declared_publish_time); } else if (vrs) { format_iso_time(published, vrs->published_on); -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. dgoulet pushed a commit to branch main in repository tor. commit 4f038d224f5a3df5fb396eff0173bba0eec89127 Merge: bae04e6a98 96f1e69f24 Author: David Goulet <dgoulet@torproject.org> AuthorDate: Mon May 16 08:51:00 2022 -0400 Merge branch 'tor-gitlab/mr/489' changes/prop275 | 12 ++++++++++ src/feature/dirauth/dirvote.c | 36 +++++++++++++++++++++-------- src/feature/dirauth/dirvote.h | 8 ++++++- src/feature/dirparse/ns_parse.c | 5 +++- src/feature/nodelist/fmt_routerstatus.c | 16 +++++++++++-- src/feature/nodelist/fmt_routerstatus.h | 3 ++- src/feature/nodelist/networkstatus.c | 9 ++------ src/feature/nodelist/routerlist.c | 19 +++++++-------- src/feature/nodelist/routerstatus_st.h | 1 - src/feature/nodelist/vote_routerstatus_st.h | 1 + src/feature/relay/router.c | 2 -- src/test/test_dir.c | 23 +++++++++--------- src/test/test_dir_common.c | 8 +++---- src/test/test_nodelist.c | 8 ++----- src/test/test_router.c | 14 +---------- src/test/test_voting_flags.c | 4 ---- 16 files changed, 96 insertions(+), 73 deletions(-) -- To stop receiving notification emails like this one, please contact the administrator of this repository.
participants (1)
-
gitolite role