June 2020 - tor-commits - lists.torproject.org

[tor/master] circuitbuild: Do node checks when choosing exits
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit 766fc86df46ff88364610b18e28904c3b45f98f1 Author: teor <teor(a)riseup.net> Date: Wed May 13 13:17:41 2020 +1000 circuitbuild: Do node checks when choosing exits And check that the correct flags are passed when choosing exits. Adds the following checks for exits: * must support EXTEND2 cells, * must have an ntor circuit crypto key, * can't require the guard flag, * can't be a direct connection. All these checks are already implied by other code. Part of 34200. --- src/core/or/circuitbuild.c | 48 ++++++++++++++++++---------------------------- 1 file changed, 19 insertions(+), 29 deletions(-) diff --git a/src/core/or/circuitbuild.c b/src/core/or/circuitbuild.c index b840ca7a7..bac82eb9f 100644 --- a/src/core/or/circuitbuild.c +++ b/src/core/or/circuitbuild.c @@ -1563,7 +1563,23 @@ choose_good_exit_server_general(router_crn_flags_t flags) const node_t *selected_node=NULL; const int need_uptime = (flags & CRN_NEED_UPTIME) != 0; const int need_capacity = (flags & CRN_NEED_CAPACITY) != 0; - const int direct_conn = (flags & CRN_DIRECT_CONN) != 0; + + /* We should not require guard flags on exits. */ + IF_BUG_ONCE(flags & CRN_NEED_GUARD) + return NULL; + + /* We reject single-hop exits for all node positions. */ + IF_BUG_ONCE(flags & CRN_DIRECT_CONN) + return NULL; + + /* This isn't the function for picking rendezvous nodes. */ + IF_BUG_ONCE(flags & CRN_RENDEZVOUS_V3) + return NULL; + + /* We only want exits to extend if we cannibalize the circuit. + * But we don't require IPv6 extends yet. */ + IF_BUG_ONCE(flags & CRN_INITIATE_IPV6_EXTEND) + return NULL; connections = get_connection_array(); @@ -1596,19 +1612,14 @@ choose_good_exit_server_general(router_crn_flags_t flags) */ continue; } - if (!node_has_preferred_descriptor(node, direct_conn)) { + if (!router_can_choose_node(node, flags)) { n_supported[i] = -1; continue; } - if (!node->is_running || node->is_bad_exit) { + if (node->is_bad_exit) { n_supported[i] = -1; continue; /* skip routers that are known to be down or bad exits */ } - if (node_get_purpose(node) != ROUTER_PURPOSE_GENERAL) { - /* never pick a non-general node as a random exit. */ - n_supported[i] = -1; - continue; - } if (routerset_contains_node(options->ExcludeExitNodesUnion_, node)) { n_supported[i] = -1; continue; /* user asked us not to use it, no matter what */ @@ -1618,27 +1629,6 @@ choose_good_exit_server_general(router_crn_flags_t flags) n_supported[i] = -1; continue; /* not one of our chosen exit nodes */ } - - if (node_is_unreliable(node, need_uptime, need_capacity, 0)) { - n_supported[i] = -1; - continue; /* skip routers that are not suitable. Don't worry if - * this makes us reject all the possible routers: if so, - * we'll retry later in this function with need_update and - * need_capacity set to 0. */ - } - if (!(node->is_valid)) { - /* if it's invalid and we don't want it */ - n_supported[i] = -1; -// log_fn(LOG_DEBUG,"Skipping node %s (index %d) -- invalid router.", -// router->nickname, i); - continue; /* skip invalid routers */ - } - /* We do not allow relays that allow single hop exits by default. Option - * was deprecated in 0.2.9.2-alpha and removed in 0.3.1.0-alpha. */ - if (node_allows_single_hop_exits(node)) { - n_supported[i] = -1; - continue; - } if (node_exit_policy_rejects_all(node)) { n_supported[i] = -1; // log_fn(LOG_DEBUG,"Skipping node %s (index %d) -- it rejects all.",

1 0

[tor/master] practracker: Accept extra file lines, enforce a smaller function
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit 8ec4d9cc3c90a56bf66cacd128e1b54124868f9f Author: teor <teor(a)riseup.net> Date: Mon May 11 18:12:31 2020 +1000 practracker: Accept extra file lines, enforce a smaller function Accept extra lines in nodelist and routerlist due to extra features, and due to refactors that simplify some functions. Most of the refactor eliminated duplicate code in smaller functions, so there's only one large function that got smaller. Part of 34200. --- scripts/maint/practracker/exceptions.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/maint/practracker/exceptions.txt b/scripts/maint/practracker/exceptions.txt index ca06d94fb..8a62c260b 100644 --- a/scripts/maint/practracker/exceptions.txt +++ b/scripts/maint/practracker/exceptions.txt @@ -96,7 +96,7 @@ problem function-size /src/core/or/channeltls.c:channel_tls_process_authenticate problem dependency-violation /src/core/or/channeltls.c 11 problem include-count /src/core/or/circuitbuild.c 53 problem function-size /src/core/or/circuitbuild.c:get_unique_circ_id_by_chan() 128 -problem function-size /src/core/or/circuitbuild.c:choose_good_exit_server_general() 206 +problem function-size /src/core/or/circuitbuild.c:choose_good_exit_server_general() 196 problem dependency-violation /src/core/or/circuitbuild.c 25 problem include-count /src/core/or/circuitlist.c 55 problem function-size /src/core/or/circuitlist.c:HT_PROTOTYPE() 109 @@ -259,7 +259,7 @@ problem function-size /src/feature/nodelist/node_select.c:router_pick_directory_ problem function-size /src/feature/nodelist/node_select.c:compute_weighted_bandwidths() 204 problem function-size /src/feature/nodelist/node_select.c:router_pick_trusteddirserver_impl() 116 problem function-size /src/feature/nodelist/nodelist.c:compute_frac_paths_available() 190 -problem file-size /src/feature/nodelist/routerlist.c 3300 +problem file-size /src/feature/nodelist/routerlist.c 3350 problem function-size /src/feature/nodelist/routerlist.c:router_rebuild_store() 148 problem function-size /src/feature/nodelist/routerlist.c:router_add_to_routerlist() 168 problem function-size /src/feature/nodelist/routerlist.c:routerlist_remove_old_routers() 121

1 0

[tor/master] circuitbuild: Do node checks when counting nodes
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit e46c3d95f4257fd1292c721a5d737af0f74f2f83 Author: teor <teor(a)riseup.net> Date: Wed May 13 13:23:39 2020 +1000 circuitbuild: Do node checks when counting nodes Use the node check function to check that there are enough nodes to select a circuit path. Adds these checks, which are implied by other code: * supports EXTEND2 cells, * does not allow single-hop exits, Adds these extra checks: * has a general-purpose routerinfo, * if it is a direct connection, check reachable addresses. These checks reduce the node count, but they will never under-count nodes. Bridge nodes aren't handled correctly, we'll fix that in the next commit. Part of 34200. --- src/core/or/circuitbuild.c | 29 ++++++++++++----------------- 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/src/core/or/circuitbuild.c b/src/core/or/circuitbuild.c index bac82eb9f..233ea9bd0 100644 --- a/src/core/or/circuitbuild.c +++ b/src/core/or/circuitbuild.c @@ -2103,32 +2103,27 @@ circuit_extend_to_new_exit(origin_circuit_t *circ, extend_info_t *exit_ei) return 0; } -/** Return the number of routers in routers that are currently up - * and available for building circuits through. +/** Return the number of routers in nodes that are currently up and + * available for building circuits through. * - * (Note that this function may overcount or undercount, if we have - * descriptors that are not the type we would prefer to use for some - * particular router. See bug #25885.) + * If direct is true, only count nodes that are suitable for direct + * connections. Counts nodes regardless of whether their addresses are + * preferred. */ MOCK_IMPL(STATIC int, count_acceptable_nodes, (const smartlist_t *nodes, int direct)) { int num=0; + int flags = CRN_NEED_DESC; + + if (direct) + flags |= CRN_DIRECT_CONN; SMARTLIST_FOREACH_BEGIN(nodes, const node_t *, node) { // log_debug(LD_CIRC, -// "Contemplating whether router %d (%s) is a new option.", -// i, r->nickname); - if (! node->is_running) -// log_debug(LD_CIRC,"Nope, the directory says %d is not running.",i); - continue; - if (! node->is_valid) -// log_debug(LD_CIRC,"Nope, the directory says %d is not valid.",i); - continue; - if (! node_has_preferred_descriptor(node, direct)) - continue; - /* The node has a descriptor, so we can just check the ntor key directly */ - if (!node_has_curve25519_onion_key(node)) + // "Contemplating whether router %d (%s) is a new option.", + // i, r->nickname); + if (!router_can_choose_node(node, flags)) continue; ++num; } SMARTLIST_FOREACH_END(node);

1 0

[tor/master] node: Clean up some outdated comments
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit 8fbcc055e5e7217404039e84e23cf35c5435dbe5 Author: teor <teor(a)riseup.net> Date: Wed May 13 15:34:37 2020 +1000 node: Clean up some outdated comments Part of 34200. --- src/feature/nodelist/node_st.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/feature/nodelist/node_st.h b/src/feature/nodelist/node_st.h index b1ec4db20..3769f9dc8 100644 --- a/src/feature/nodelist/node_st.h +++ b/src/feature/nodelist/node_st.h @@ -84,12 +84,11 @@ struct node_t { /* Local info: derived. */ - /** True if the IPv6 OR port is preferred over the IPv4 OR port. - * XX/teor - can this become out of date if the torrc changes? */ + /** True if the IPv6 OR port is preferred over the IPv4 OR port. */ unsigned int ipv6_preferred:1; /** According to the geoip db what country is this router in? */ - /* XXXprop186 what is this suppose to mean with multiple OR ports? */ + /* IPv6: what is this supposed to mean with multiple OR ports? */ country_t country; /* The below items are used only by authdirservers for

1 0

[tor/master] routerlist: Choose bridges for direct bridge connections
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit 73ace125a972c421287503738e9599f411193e1a Author: teor <teor(a)riseup.net> Date: Wed May 13 13:47:52 2020 +1000 routerlist: Choose bridges for direct bridge connections When counting and choosing nodes on a client that uses bridges, only choose bridges for direct connections. Part of 34200. --- src/feature/nodelist/routerlist.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/src/feature/nodelist/routerlist.c b/src/feature/nodelist/routerlist.c index f606e3c18..ece3379fa 100644 --- a/src/feature/nodelist/routerlist.c +++ b/src/feature/nodelist/routerlist.c @@ -516,11 +516,19 @@ routers_have_same_or_addrs(const routerinfo_t *r1, const routerinfo_t *r2) * The following conditions are applied to all nodes: * - is running; * - is valid; - * - has a general-purpose routerinfo; * - supports EXTEND2 cells; * - has an ntor circuit crypto key; and * - does not allow single-hop exits. * + * If the node has a routerinfo, we're checking for a direct connection, and + * we're using bridges, the following condition is applied: + * - has a bridge-purpose routerinfo; + * and for all other nodes: + * - has a general-purpose routerinfo (or no routerinfo). + * + * Nodes that don't have a routerinfo must be general-purpose nodes, because + * routerstatuses and microdescriptors only come via consensuses. + * * The flags chech that node: * - CRN_NEED_UPTIME: has more than a minimum uptime; * - CRN_NEED_CAPACITY: has more than a minimum capacity; @@ -550,16 +558,21 @@ router_can_choose_node(const node_t *node, int flags) const bool rendezvous_v3 = (flags & CRN_RENDEZVOUS_V3) != 0; const bool initiate_ipv6_extend = (flags & CRN_INITIATE_IPV6_EXTEND) != 0; + const or_options_t *options = get_options(); const bool check_reach = - !router_or_conn_should_skip_reachable_address_check(get_options(), - pref_addr); + !router_or_conn_should_skip_reachable_address_check(options, pref_addr); + const bool direct_bridge = direct_conn && options->UseBridges; if (!node->is_running || !node->is_valid) return false; if (need_desc && !node_has_preferred_descriptor(node, direct_conn)) return false; - if (node->ri && node->ri->purpose != ROUTER_PURPOSE_GENERAL) - return false; + if (node->ri) { + if (direct_bridge && node->ri->purpose != ROUTER_PURPOSE_BRIDGE) + return false; + else if (node->ri->purpose != ROUTER_PURPOSE_GENERAL) + return false; + } if (node_is_unreliable(node, need_uptime, need_capacity, need_guard)) return false; /* Don't choose nodes if we are certain they can't do EXTEND2 cells */

1 0

[tor/master] Merge remote-tracking branch 'tor-github/pr/1888/head'
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit 354f085e5f14c2bb4918b015e36cdc72748e0ea6 Merge: eaae5625c 1df451aba Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jun 9 15:44:58 2020 -0400 Merge remote-tracking branch 'tor-github/pr/1888/head' changes/bug34248 | 4 + changes/bug34251 | 4 + changes/ticket33222 | 8 + changes/ticket34200 | 3 + doc/tor.1.txt | 6 +- scripts/maint/practracker/exceptions.txt | 14 +- src/core/or/circuitbuild.c | 190 +++++++++------- src/core/or/circuitbuild.h | 4 + src/core/or/circuitlist.c | 2 +- src/core/or/circuituse.c | 22 +- src/core/or/circuituse.h | 16 +- src/core/or/cpath_build_state_st.h | 2 + src/core/or/or.h | 21 +- src/core/or/protover.c | 11 +- src/core/or/protover.h | 24 +- src/core/or/versions.c | 55 ++++- src/feature/control/control_getinfo.c | 17 +- src/feature/nodelist/node_select.c | 152 ++++++------- src/feature/nodelist/node_select.h | 28 ++- src/feature/nodelist/node_st.h | 5 +- src/feature/nodelist/nodelist.c | 76 +++++-- src/feature/nodelist/nodelist.h | 16 +- src/feature/nodelist/routerinfo.c | 35 ++- src/feature/nodelist/routerinfo.h | 5 +- src/feature/nodelist/routerlist.c | 150 +++++++++--- src/feature/nodelist/routerlist.h | 14 +- src/feature/relay/relay_periodic.c | 4 +- src/feature/relay/router.c | 21 +- src/feature/relay/selftest.c | 231 +++++++++++++------ src/feature/relay/selftest.h | 10 +- src/feature/stats/predict_ports.c | 4 +- src/lib/net/address.c | 36 ++- src/lib/net/address.h | 2 + src/rust/protover/ffi.rs | 10 +- src/rust/protover/protover.rs | 8 +- src/test/test_circuitbuild.c | 377 ++++++++++++++++++++++++++++++- src/test/test_circuitstats.c | 95 ++++---- src/test/test_helpers.c | 62 ++++- src/test/test_helpers.h | 5 + src/test/test_protover.c | 331 ++++++++++++++++++++++++--- 40 files changed, 1595 insertions(+), 485 deletions(-) diff --cc src/feature/relay/relay_periodic.c index 08ad110cf,2b6be1ce1..6a92f49d2 --- a/src/feature/relay/relay_periodic.c +++ b/src/feature/relay/relay_periodic.c @@@ -201,36 -201,31 +201,36 @@@ reachability_warnings_callback(time_t n have_completed_a_circuit()) { /* every 20 minutes, check and complain if necessary */ const routerinfo_t *me = router_get_my_routerinfo(); - if (me && !check_whether_orport_reachable(options)) { + if (me && !router_should_skip_orport_reachability_check(options)) { char *address = tor_dup_ip(me->addr); - log_warn(LD_CONFIG,"Your server (%s:%d) has not managed to confirm that " - "its ORPort is reachable. Relays do not publish descriptors " - "until their ORPort and DirPort are reachable. Please check " - "your firewalls, ports, address, /etc/hosts file, etc.", - address, me->or_port); - control_event_server_status(LOG_WARN, - "REACHABILITY_FAILED ORADDRESS=%s:%d", - address, me->or_port); - tor_free(address); + if (address) { + log_warn(LD_CONFIG, + "Your server (%s:%d) has not managed to confirm that " + "its ORPort is reachable. Relays do not publish descriptors " + "until their ORPort and DirPort are reachable. Please check " + "your firewalls, ports, address, /etc/hosts file, etc.", + address, me->or_port); + control_event_server_status(LOG_WARN, + "REACHABILITY_FAILED ORADDRESS=%s:%d", + address, me->or_port); + tor_free(address); + } } - if (me && !check_whether_dirport_reachable(options)) { + if (me && !router_should_skip_dirport_reachability_check(options)) { char *address = tor_dup_ip(me->addr); - log_warn(LD_CONFIG, - "Your server (%s:%d) has not managed to confirm that its " - "DirPort is reachable. Relays do not publish descriptors " - "until their ORPort and DirPort are reachable. Please check " - "your firewalls, ports, address, /etc/hosts file, etc.", - address, me->dir_port); - control_event_server_status(LOG_WARN, - "REACHABILITY_FAILED DIRADDRESS=%s:%d", - address, me->dir_port); - tor_free(address); + if (address) { + log_warn(LD_CONFIG, + "Your server (%s:%d) has not managed to confirm that its " + "DirPort is reachable. Relays do not publish descriptors " + "until their ORPort and DirPort are reachable. Please check " + "your firewalls, ports, address, /etc/hosts file, etc.", + address, me->dir_port); + control_event_server_status(LOG_WARN, + "REACHABILITY_FAILED DIRADDRESS=%s:%d", + address, me->dir_port); + tor_free(address); + } } } diff --cc src/feature/relay/selftest.c index 18fe25b98,ae5633bfb..2b0fc951b --- a/src/feature/relay/selftest.c +++ b/src/feature/relay/selftest.c @@@ -224,30 -303,40 +303,41 @@@ inform_testing_reachability(void const routerinfo_t *me = router_get_my_routerinfo(); if (!me) return 0; + - address = tor_dup_ip(me->addr); - if (!address) - return 0; - + /* IPv4 ORPort */ + strlcpy(ipv4_or_buf, fmt_addr32_port(me->addr, me->or_port), + sizeof(ipv4_or_buf)); control_event_server_status(LOG_NOTICE, - "CHECKING_REACHABILITY ORADDRESS=%s:%d", - address, me->or_port); + "CHECKING_REACHABILITY ORADDRESS=%s", + ipv4_or_buf); + /* IPv6 ORPort */ + const bool has_ipv6 = tor_addr_port_is_valid(&me->ipv6_addr, + me->ipv6_orport, 0); + if (has_ipv6) { + strlcpy(ipv6_or_buf, fmt_addrport(&me->ipv6_addr, me->ipv6_orport), + sizeof(ipv6_or_buf)); + /* We'll add an IPv6 control event in #34068. */ + } + /* IPv4 DirPort (there are no advertised IPv6 DirPorts) */ if (me->dir_port) { - tor_snprintf(dirbuf, sizeof(dirbuf), " and DirPort %s:%d", - address, me->dir_port); + strlcpy(ipv4_dir_buf, fmt_addr32_port(me->addr, me->dir_port), + sizeof(ipv4_dir_buf)); control_event_server_status(LOG_NOTICE, - "CHECKING_REACHABILITY DIRADDRESS=%s:%d", - address, me->dir_port); + "CHECKING_REACHABILITY DIRADDRESS=%s", + ipv4_dir_buf); } - log_notice(LD_OR, "Now checking whether ORPort %s:%d%s %s reachable... " - "(this may take up to %d minutes -- look for log " - "messages indicating success)", - address, me->or_port, - me->dir_port ? dirbuf : "", - me->dir_port ? "are" : "is", - TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT/60); - - tor_free(address); + log_notice(LD_OR, "Now checking whether ORPort%s %s%s%s%s%s %s reachable... " + "(this may take up to %d minutes -- look for log " + "messages indicating success)", + has_ipv6 ? "s" : "", + ipv4_or_buf, + has_ipv6 ? " and " : "", + has_ipv6 ? ipv6_or_buf : "", + me->dir_port ? " and DirPort " : "", + me->dir_port ? ipv4_dir_buf : "", + has_ipv6 || me->dir_port ? "are" : "is", + TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT/60); + return 1; }

1 0

[tor/master] changes: file for 34200
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit 1df451aba116f77a5a24e4e54cf343ec46d55f9a Author: teor <teor(a)riseup.net> Date: Mon May 18 22:12:26 2020 +1000 changes: file for 34200 --- changes/ticket34200 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/changes/ticket34200 b/changes/ticket34200 new file mode 100644 index 000000000..b984bd83b --- /dev/null +++ b/changes/ticket34200 @@ -0,0 +1,3 @@ + o Code simplification and refactoring: + - Refactor some common node selection code into a single function. + Closes ticket 34200.

1 0

[tor/master] Merge branch 'ticket34375'
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit ca8b04f194e827162a5f054e33d108b4bbc1372d Merge: d2f136aab 811853c67 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jun 9 14:16:21 2020 -0400 Merge branch 'ticket34375' scripts/git/git-list-tor-branches.sh | 3 --- 1 file changed, 3 deletions(-)

1 0

[tor/master] Remove 0.4.1 from list of tor branches used by git scripts.
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit 811853c67719bf2e8817d2e4256f715016a03345 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jun 3 14:54:21 2020 -0400 Remove 0.4.1 from list of tor branches used by git scripts. --- scripts/git/git-list-tor-branches.sh | 3 --- 1 file changed, 3 deletions(-) diff --git a/scripts/git/git-list-tor-branches.sh b/scripts/git/git-list-tor-branches.sh index d6b30f064..b50365590 100755 --- a/scripts/git/git-list-tor-branches.sh +++ b/scripts/git/git-list-tor-branches.sh @@ -139,9 +139,6 @@ finish() { branch maint-0.3.5 branch release-0.3.5 -branch maint-0.4.1 -branch release-0.4.1 - branch maint-0.4.2 branch release-0.4.2

1 0

[tor/master] Add 0.4.4 to list of tor branches used by git scripts.
by nickm＠torproject.org 09 Jun '20

09 Jun '20

commit eaae5625cb1c5fdd26c766157742725aa4b7ad9e Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Jun 9 14:17:15 2020 -0400 Add 0.4.4 to list of tor branches used by git scripts. --- scripts/git/git-list-tor-branches.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/git/git-list-tor-branches.sh b/scripts/git/git-list-tor-branches.sh index b50365590..b0c30d2e8 100755 --- a/scripts/git/git-list-tor-branches.sh +++ b/scripts/git/git-list-tor-branches.sh @@ -145,6 +145,9 @@ branch release-0.4.2 branch maint-0.4.3 branch release-0.4.3 +branch maint-0.4.4 +branch release-0.4.4 + branch master finish

1 0