April 2019 - tor-commits - lists.torproject.org

[tor/maint-0.3.5] Merge remote-tracking branch 'tor-github/pr/710' into maint-0.3.5
by teor＠torproject.org 19 Apr '19

19 Apr '19

commit 742b5b32d502712c7ad6d75adbf25fca6986c898 Merge: cb084de5e 78220aae1 Author: teor <teor(a)torproject.org> Date: Fri Apr 19 11:52:48 2019 +1000 Merge remote-tracking branch 'tor-github/pr/710' into maint-0.3.5 changes/bug28698 | 3 +++ src/core/or/circuituse.c | 30 ++++++++++++++++++------------ 2 files changed, 21 insertions(+), 12 deletions(-)

1 0

[tor/maint-0.3.5] stats: Make PaddingStatistics depend on ExtraInfoStatistics
by teor＠torproject.org 19 Apr '19

19 Apr '19

commit 85c598cbc2d4ca1a7c729864002b5b222980ce19 Author: teor <teor(a)torproject.org> Date: Tue Jan 8 18:21:10 2019 +1000 stats: Make PaddingStatistics depend on ExtraInfoStatistics When ExtraInfoStatistics is 0, stop including PaddingStatistics in relay and bridge extra-info documents. Fixes bug 29017; bugfix on 0.3.1.1-alpha. --- changes/bug29017 | 4 ++++ doc/tor.1.txt | 2 +- src/or/router.c | 11 +++++------ 3 files changed, 10 insertions(+), 7 … [View More]

1 0

[tor/maint-0.3.5] Fix some error-checking logic and a misleading error message
by teor＠torproject.org 19 Apr '19

19 Apr '19

commit 1b9e77349ffbe813c7b16af67c069d967327f63b Author: Kris Katterjohn <katterjohn(a)gmail.com> Date: Fri Jan 25 15:52:40 2019 -0600 Fix some error-checking logic and a misleading error message When IPv4Only (IPv6Only) was used but the address could not be interpreted as a IPv4 (IPv6) address, the error message referred to the wrong IP version. This also fixes up the error-checking logic so it's more precise about what's being checked. Fixes … [View More]

1 0

[tor/maint-0.3.5] Merge branch 'maint-0.2.9' into maint-0.3.4
by teor＠torproject.org 19 Apr '19

19 Apr '19

commit cbce8dedd3703424111928d68dfe8920900ce054 Merge: 116970dda 2460b4461 Author: teor <teor(a)torproject.org> Date: Fri Apr 19 11:59:18 2019 +1000 Merge branch 'maint-0.2.9' into maint-0.3.4 "ours" merge, to avoid taking any changes from PR 792 in 0.3.4. (We already merged PR 791 for 29665 into 0.3.4.)

1 0

[tor/maint-0.3.5] Merge branch 'maint-0.3.4' into maint-0.3.5
by teor＠torproject.org 19 Apr '19

19 Apr '19

commit 231036a110c1062e39b214b4b88fdc2a1eb46dc8 Merge: 742b5b32d cbce8dedd Author: teor <teor(a)torproject.org> Date: Fri Apr 19 12:00:41 2019 +1000 Merge branch 'maint-0.3.4' into maint-0.3.5 changes/bug29017 | 4 ++++ changes/bug29665 | 7 +++++++ doc/tor.1.txt | 2 +- src/core/or/circuituse.c | 4 ++++ src/feature/relay/router.c | 11 +++++------ 5 files changed, 21 insertions(+), 7 deletions(-) diff --cc src/core/or/circuituse.c index … [View More]e306307c4,000000000..02bfa15fb mode 100644,000000..100644 --- a/src/core/or/circuituse.c +++ b/src/core/or/circuituse.c @@@ -1,3132 -1,0 +1,3136 @@@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2019, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +/** + * \file circuituse.c + * \brief Launch the right sort of circuits and attach the right streams to + * them. + * + * As distinct from circuitlist.c, which manages lookups to find circuits, and + * circuitbuild.c, which handles the logistics of circuit construction, this + * module keeps track of which streams can be attached to which circuits (in + * circuit_get_best()), and attaches streams to circuits (with + * circuit_try_attaching_streams(), connection_ap_handshake_attach_circuit(), + * and connection_ap_handshake_attach_chosen_circuit() ). + * + * This module also makes sure that we are building circuits for all of the + * predicted ports, using circuit_remove_handled_ports(), + * circuit_stream_is_being_handled(), and circuit_build_needed_cirs(). It + * handles launching circuits for specific targets using + * circuit_launch_by_extend_info(). + * + * This is also where we handle expiring circuits that have been around for + * too long without actually completing, along with the circuit_build_timeout + * logic in circuitstats.c. + **/ + +#include "core/or/or.h" +#include "app/config/config.h" +#include "core/mainloop/connection.h" +#include "core/or/channel.h" +#include "core/or/circuitbuild.h" +#include "core/or/circuitlist.h" +#include "core/or/circuitstats.h" +#include "core/or/circuituse.h" +#include "core/or/connection_edge.h" +#include "core/or/policies.h" +#include "feature/client/addressmap.h" +#include "feature/client/bridges.h" +#include "feature/client/circpathbias.h" +#include "feature/client/entrynodes.h" +#include "feature/control/control.h" +#include "feature/dircommon/directory.h" +#include "feature/hs/hs_circuit.h" +#include "feature/hs/hs_client.h" +#include "feature/hs/hs_common.h" +#include "feature/hs/hs_ident.h" +#include "feature/hs/hs_stats.h" +#include "feature/nodelist/describe.h" +#include "feature/nodelist/networkstatus.h" +#include "feature/nodelist/nodelist.h" +#include "feature/nodelist/routerlist.h" +#include "feature/relay/routermode.h" +#include "feature/relay/selftest.h" +#include "feature/rend/rendclient.h" +#include "feature/rend/rendcommon.h" +#include "feature/rend/rendservice.h" +#include "feature/stats/predict_ports.h" +#include "lib/math/fp.h" +#include "lib/time/tvdiff.h" + +#include "core/or/cpath_build_state_st.h" +#include "feature/dircommon/dir_connection_st.h" +#include "core/or/entry_connection_st.h" +#include "core/or/extend_info_st.h" +#include "core/or/or_circuit_st.h" +#include "core/or/origin_circuit_st.h" +#include "core/or/socks_request_st.h" + +static void circuit_expire_old_circuits_clientside(void); +static void circuit_increment_failure_count(void); + +/** Check whether the hidden service destination of the stream at + * edge_conn is the same as the destination of the circuit at + * origin_circ. */ +static int +circuit_matches_with_rend_stream(const edge_connection_t *edge_conn, + const origin_circuit_t *origin_circ) +{ + /* Check if this is a v2 rendezvous circ/stream */ + if ((edge_conn->rend_data && !origin_circ->rend_data) || + (!edge_conn->rend_data && origin_circ->rend_data) || + (edge_conn->rend_data && origin_circ->rend_data && + rend_cmp_service_ids(rend_data_get_address(edge_conn->rend_data), + rend_data_get_address(origin_circ->rend_data)))) { + /* this circ is not for this conn */ + return 0; + } + + /* Check if this is a v3 rendezvous circ/stream */ + if ((edge_conn->hs_ident && !origin_circ->hs_ident) || + (!edge_conn->hs_ident && origin_circ->hs_ident) || + (edge_conn->hs_ident && origin_circ->hs_ident && + !ed25519_pubkey_eq(&edge_conn->hs_ident->identity_pk, + &origin_circ->hs_ident->identity_pk))) { + /* this circ is not for this conn */ + return 0; + } + + return 1; +} + +/** Return 1 if circ could be returned by circuit_get_best(). + * Else return 0. + */ +static int +circuit_is_acceptable(const origin_circuit_t *origin_circ, + const entry_connection_t *conn, + int must_be_open, uint8_t purpose, + int need_uptime, int need_internal, + time_t now) +{ + const circuit_t *circ = TO_CIRCUIT(origin_circ); + const node_t *exitnode; + cpath_build_state_t *build_state; + tor_assert(circ); + tor_assert(conn); + tor_assert(conn->socks_request); + + if (must_be_open && (circ->state != CIRCUIT_STATE_OPEN || !circ->n_chan)) + return 0; /* ignore non-open circs */ + if (circ->marked_for_close) + return 0; + + /* if this circ isn't our purpose, skip. */ + if (purpose == CIRCUIT_PURPOSE_C_REND_JOINED && !must_be_open) { + if (circ->purpose != CIRCUIT_PURPOSE_C_ESTABLISH_REND && + circ->purpose != CIRCUIT_PURPOSE_C_REND_READY && + circ->purpose != CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED && + circ->purpose != CIRCUIT_PURPOSE_C_REND_JOINED) + return 0; + } else if (purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT && + !must_be_open) { + if (circ->purpose != CIRCUIT_PURPOSE_C_INTRODUCING && + circ->purpose != CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) + return 0; + } else { + if (purpose != circ->purpose) + return 0; + } + + /* If this is a timed-out hidden service circuit, skip it. */ + if (origin_circ->hs_circ_has_timed_out) { + return 0; + } + + if (purpose == CIRCUIT_PURPOSE_C_GENERAL || + purpose == CIRCUIT_PURPOSE_C_HSDIR_GET || + purpose == CIRCUIT_PURPOSE_S_HSDIR_POST || + purpose == CIRCUIT_PURPOSE_HS_VANGUARDS || + purpose == CIRCUIT_PURPOSE_C_REND_JOINED) { + if (circ->timestamp_dirty && + circ->timestamp_dirty+get_options()->MaxCircuitDirtiness <= now) + return 0; + } + + if (origin_circ->unusable_for_new_conns) + return 0; + + /* decide if this circ is suitable for this conn */ + + /* for rend circs, circ->cpath->prev is not the last router in the + * circuit, it's the magical extra service hop. so just check the nickname + * of the one we meant to finish at. + */ + build_state = origin_circ->build_state; + exitnode = build_state_get_exit_node(build_state); + + if (need_uptime && !build_state->need_uptime) + return 0; + if (need_internal != build_state->is_internal) + return 0; + + if (purpose == CIRCUIT_PURPOSE_C_GENERAL || + purpose == CIRCUIT_PURPOSE_S_HSDIR_POST || + purpose == CIRCUIT_PURPOSE_C_HSDIR_GET) { + tor_addr_t addr; + const int family = tor_addr_parse(&addr, conn->socks_request->address); + if (!exitnode && !build_state->onehop_tunnel) { + log_debug(LD_CIRC,"Not considering circuit with unknown router."); + return 0; /* this circuit is screwed and doesn't know it yet, + * or is a rendezvous circuit. */ + } + if (build_state->onehop_tunnel) { + if (!conn->want_onehop) { + log_debug(LD_CIRC,"Skipping one-hop circuit."); + return 0; + } + tor_assert(conn->chosen_exit_name); + if (build_state->chosen_exit) { + char digest[DIGEST_LEN]; + if (hexdigest_to_digest(conn->chosen_exit_name, digest) < 0) + return 0; /* broken digest, we don't want it */ + if (tor_memneq(digest, build_state->chosen_exit->identity_digest, + DIGEST_LEN)) + return 0; /* this is a circuit to somewhere else */ + if (tor_digest_is_zero(digest)) { + /* we don't know the digest; have to compare addr:port */ + if (family < 0 || + !tor_addr_eq(&build_state->chosen_exit->addr, &addr) || + build_state->chosen_exit->port != conn->socks_request->port) + return 0; + } + } + } else { + if (conn->want_onehop) { + /* don't use three-hop circuits -- that could hurt our anonymity. */ + return 0; + } + } + if (origin_circ->prepend_policy && family != -1) { + int r = compare_tor_addr_to_addr_policy(&addr, + conn->socks_request->port, + origin_circ->prepend_policy); + if (r == ADDR_POLICY_REJECTED) + return 0; + } + if (exitnode && !connection_ap_can_use_exit(conn, exitnode)) { + /* can't exit from this router */ + return 0; + } + } else { /* not general: this might be a rend circuit */ + const edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(conn); + if (!circuit_matches_with_rend_stream(edge_conn, origin_circ)) { + return 0; + } + } + + if (!connection_edge_compatible_with_circuit(conn, origin_circ)) { + /* conn needs to be isolated from other conns that have already used + * origin_circ */ + return 0; + } + + return 1; +} + +/** Return 1 if circuit a is better than circuit b for + * conn, and return 0 otherwise. Used by circuit_get_best. + */ +static int +circuit_is_better(const origin_circuit_t *oa, const origin_circuit_t *ob, + const entry_connection_t *conn) +{ + const circuit_t *a = TO_CIRCUIT(oa); + const circuit_t *b = TO_CIRCUIT(ob); + const uint8_t purpose = ENTRY_TO_CONN(conn)->purpose; + int a_bits, b_bits; + + /* If one of the circuits was allowed to live due to relaxing its timeout, + * it is definitely worse (it's probably a much slower path). */ + if (oa->relaxed_timeout && !ob->relaxed_timeout) + return 0; /* ob is better. It's not relaxed. */ + if (!oa->relaxed_timeout && ob->relaxed_timeout) + return 1; /* oa is better. It's not relaxed. */ + + switch (purpose) { + case CIRCUIT_PURPOSE_S_HSDIR_POST: + case CIRCUIT_PURPOSE_C_HSDIR_GET: + case CIRCUIT_PURPOSE_C_GENERAL: + /* if it's used but less dirty it's best; + * else if it's more recently created it's best + */ + if (b->timestamp_dirty) { + if (a->timestamp_dirty && + a->timestamp_dirty > b->timestamp_dirty) + return 1; + } else { + if (a->timestamp_dirty || + timercmp(&a->timestamp_began, &b->timestamp_began, OP_GT)) + return 1; + if (ob->build_state->is_internal) + /* XXXX++ what the heck is this internal thing doing here. I + * think we can get rid of it. circuit_is_acceptable() already + * makes sure that is_internal is exactly what we need it to + * be. -RD */ + return 1; + } + break; + case CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT: + /* the closer it is to ack_wait the better it is */ + if (a->purpose > b->purpose) + return 1; + break; + case CIRCUIT_PURPOSE_C_REND_JOINED: + /* the closer it is to rend_joined the better it is */ + if (a->purpose > b->purpose) + return 1; + break; + } + + /* XXXX Maybe this check should get a higher priority to avoid + * using up circuits too rapidly. */ + + a_bits = connection_edge_update_circuit_isolation(conn, + (origin_circuit_t*)oa, 1); + b_bits = connection_edge_update_circuit_isolation(conn, + (origin_circuit_t*)ob, 1); + /* if x_bits < 0, then we have not used x for anything; better not to dirty + * a connection if we can help it. */ + if (a_bits < 0) { + return 0; + } else if (b_bits < 0) { + return 1; + } + a_bits &= ~ oa->isolation_flags_mixed; + a_bits &= ~ ob->isolation_flags_mixed; + if (n_bits_set_u8(a_bits) < n_bits_set_u8(b_bits)) { + /* The fewer new restrictions we need to make on a circuit for stream + * isolation, the better. */ + return 1; + } + + return 0; +} + +/** Find the best circ that conn can use, preferably one which is + * dirty. Circ must not be too old. + * + * Conn must be defined. + * + * If must_be_open, ignore circs not in CIRCUIT_STATE_OPEN. + * + * circ_purpose specifies what sort of circuit we must have. + * It can be C_GENERAL, C_INTRODUCE_ACK_WAIT, or C_REND_JOINED. + * + * If it's REND_JOINED and must_be_open==0, then return the closest + * rendezvous-purposed circuit that you can find. + * + * If it's INTRODUCE_ACK_WAIT and must_be_open==0, then return the + * closest introduce-purposed circuit that you can find. + */ +static origin_circuit_t * +circuit_get_best(const entry_connection_t *conn, + int must_be_open, uint8_t purpose, + int need_uptime, int need_internal) +{ + origin_circuit_t *best=NULL; + struct timeval now; + int intro_going_on_but_too_old = 0; + + tor_assert(conn); + + tor_assert(purpose == CIRCUIT_PURPOSE_C_GENERAL || + purpose == CIRCUIT_PURPOSE_HS_VANGUARDS || + purpose == CIRCUIT_PURPOSE_C_HSDIR_GET || + purpose == CIRCUIT_PURPOSE_S_HSDIR_POST || + purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT || + purpose == CIRCUIT_PURPOSE_C_REND_JOINED); + + tor_gettimeofday(&now); + + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *, circ) { + origin_circuit_t *origin_circ; + if (!CIRCUIT_IS_ORIGIN(circ)) + continue; + origin_circ = TO_ORIGIN_CIRCUIT(circ); + + /* Log an info message if we're going to launch a new intro circ in + * parallel */ + if (purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT && + !must_be_open && origin_circ->hs_circ_has_timed_out && + !circ->marked_for_close) { + intro_going_on_but_too_old = 1; + continue; + } + + if (!circuit_is_acceptable(origin_circ,conn,must_be_open,purpose, + need_uptime,need_internal, (time_t)now.tv_sec)) + continue; + + /* now this is an acceptable circ to hand back. but that doesn't + * mean it's the *best* circ to hand back. try to decide. + */ + if (!best || circuit_is_better(origin_circ,best,conn)) + best = origin_circ; + } + SMARTLIST_FOREACH_END(circ); + + if (!best && intro_going_on_but_too_old) + log_info(LD_REND|LD_CIRC, "There is an intro circuit being created " + "right now, but it has already taken quite a while. Starting " + "one in parallel."); + + return best; +} + +/** Return the number of not-yet-open general-purpose origin circuits. */ +static int +count_pending_general_client_circuits(void) +{ + int count = 0; + + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *, circ) { + if (circ->marked_for_close || + circ->state == CIRCUIT_STATE_OPEN || + !CIRCUIT_PURPOSE_COUNTS_TOWARDS_MAXPENDING(circ->purpose) || + !CIRCUIT_IS_ORIGIN(circ)) + continue; + + ++count; + } + SMARTLIST_FOREACH_END(circ); + + return count; +} + +#if 0 +/** Check whether, according to the policies in options, the + * circuit circ makes sense. */ +/* XXXX currently only checks Exclude{Exit}Nodes; it should check more. + * Also, it doesn't have the right definition of an exit circuit. Also, + * it's never called. */ +int +circuit_conforms_to_options(const origin_circuit_t *circ, + const or_options_t *options) +{ + const crypt_path_t *cpath, *cpath_next = NULL; + + /* first check if it includes any excluded nodes */ + for (cpath = circ->cpath; cpath_next != circ->cpath; cpath = cpath_next) { + cpath_next = cpath->next; + if (routerset_contains_extendinfo(options->ExcludeNodes, + cpath->extend_info)) + return 0; + } + + /* then consider the final hop */ + if (routerset_contains_extendinfo(options->ExcludeExitNodes, + circ->cpath->prev->extend_info)) + return 0; + + return 1; +} +#endif /* 0 */ + +/** + * Close all circuits that start at us, aren't open, and were born + * at least CircuitBuildTimeout seconds ago. + * + * TODO: This function is now partially redundant to + * circuit_build_times_handle_completed_hop(), but that function only + * covers circuits up to and including 3 hops that are still actually + * completing hops. However, circuit_expire_building() also handles longer + * circuits, as well as circuits that are completely stalled. + * In the future (after prop247/other path selection revamping), we probably + * want to eliminate this rats nest in favor of a simpler approach. + */ +void +circuit_expire_building(void) +{ + /* circ_times.timeout_ms and circ_times.close_ms are from + * circuit_build_times_get_initial_timeout() if we haven't computed + * custom timeouts yet */ + struct timeval general_cutoff, begindir_cutoff, fourhop_cutoff, + close_cutoff, extremely_old_cutoff, hs_extremely_old_cutoff, + cannibalized_cutoff, c_intro_cutoff, s_intro_cutoff, stream_cutoff; + const or_options_t *options = get_options(); + struct timeval now; + cpath_build_state_t *build_state; + int any_opened_circs = 0; + + tor_gettimeofday(&now); + + /* Check to see if we have any opened circuits. If we don't, + * we want to be more lenient with timeouts, in case the + * user has relocated and/or changed network connections. + * See bug #3443. */ + any_opened_circs = circuit_any_opened_circuits(); + +#define SET_CUTOFF(target, msec) do { \ + long ms = tor_lround(msec); \ + struct timeval diff; \ + diff.tv_sec = ms / 1000; \ + diff.tv_usec = (int)((ms % 1000) * 1000); \ + timersub(&now, &diff, &target); \ + } while (0) + + /** + * Because circuit build timeout is calculated only based on 3 hop + * general purpose circuit construction, we need to scale the timeout + * to make it properly apply to longer circuits, and circuits of + * certain usage types. The following diagram illustrates how we + * derive the scaling below. In short, we calculate the number + * of times our telescoping-based circuit construction causes cells + * to traverse each link for the circuit purpose types in question, + * and then assume each link is equivalent. + * + * OP --a--> A --b--> B --c--> C + * OP --a--> A --b--> B --c--> C --d--> D + * + * Let h = a = b = c = d + * + * Three hops (general_cutoff) + * RTTs = 3a + 2b + c + * RTTs = 6h + * Cannibalized: + * RTTs = a+b+c+d + * RTTs = 4h + * Four hops: + * RTTs = 4a + 3b + 2c + d + * RTTs = 10h + * Client INTRODUCE1+ACK: // XXX: correct? + * RTTs = 5a + 4b + 3c + 2d + * RTTs = 14h + * Server intro: + * RTTs = 4a + 3b + 2c + * RTTs = 9h + */ + SET_CUTOFF(general_cutoff, get_circuit_build_timeout_ms()); + SET_CUTOFF(begindir_cutoff, get_circuit_build_timeout_ms()); + + // TODO: We should probably use route_len_for_purpose() here instead, + // except that does not count the extra round trip for things like server + // intros and rends. + + /* > 3hop circs seem to have a 1.0 second delay on their cannibalized + * 4th hop. */ + SET_CUTOFF(fourhop_cutoff, get_circuit_build_timeout_ms() * (10/6.0) + 1000); + + /* CIRCUIT_PURPOSE_C_ESTABLISH_REND behaves more like a RELAY cell. + * Use the stream cutoff (more or less). */ + SET_CUTOFF(stream_cutoff, MAX(options->CircuitStreamTimeout,15)*1000 + 1000); + + /* Be lenient with cannibalized circs. They already survived the official + * CBT, and they're usually not performance-critical. */ + SET_CUTOFF(cannibalized_cutoff, + MAX(get_circuit_build_close_time_ms()*(4/6.0), + options->CircuitStreamTimeout * 1000) + 1000); + + /* Intro circs have an extra round trip (and are also 4 hops long) */ + SET_CUTOFF(c_intro_cutoff, get_circuit_build_timeout_ms() * (14/6.0) + 1000); + + /* Server intro circs have an extra round trip */ + SET_CUTOFF(s_intro_cutoff, get_circuit_build_timeout_ms() * (9/6.0) + 1000); + + SET_CUTOFF(close_cutoff, get_circuit_build_close_time_ms()); + SET_CUTOFF(extremely_old_cutoff, get_circuit_build_close_time_ms()*2 + 1000); + + SET_CUTOFF(hs_extremely_old_cutoff, + MAX(get_circuit_build_close_time_ms()*2 + 1000, + options->SocksTimeout * 1000)); + + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *,victim) { + struct timeval cutoff; + bool fixed_time = circuit_build_times_disabled(get_options()); + + if (!CIRCUIT_IS_ORIGIN(victim) || /* didn't originate here */ + victim->marked_for_close) /* don't mess with marked circs */ + continue; + + /* If we haven't yet started the first hop, it means we don't have + * any orconns available, and thus have not started counting time yet + * for this circuit. See circuit_deliver_create_cell() and uses of + * timestamp_began. + * + * Continue to wait in this case. The ORConn should timeout + * independently and kill us then. + */ + if (TO_ORIGIN_CIRCUIT(victim)->cpath->state == CPATH_STATE_CLOSED) { + continue; + } + + build_state = TO_ORIGIN_CIRCUIT(victim)->build_state; + if (build_state && build_state->onehop_tunnel) + cutoff = begindir_cutoff; + else if (victim->purpose == CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) + cutoff = close_cutoff; + else if (victim->purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) + cutoff = c_intro_cutoff; + else if (victim->purpose == CIRCUIT_PURPOSE_S_ESTABLISH_INTRO) + cutoff = s_intro_cutoff; + else if (victim->purpose == CIRCUIT_PURPOSE_C_ESTABLISH_REND) + cutoff = stream_cutoff; + else if (victim->purpose == CIRCUIT_PURPOSE_PATH_BIAS_TESTING) + cutoff = close_cutoff; + else if (TO_ORIGIN_CIRCUIT(victim)->has_opened && + victim->state != CIRCUIT_STATE_OPEN) + cutoff = cannibalized_cutoff; + else if (build_state && build_state->desired_path_len >= 4) + cutoff = fourhop_cutoff; + else + cutoff = general_cutoff; + + if (TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out) + cutoff = hs_extremely_old_cutoff; + + if (timercmp(&victim->timestamp_began, &cutoff, OP_GT)) + continue; /* it's still young, leave it alone */ + + /* We need to double-check the opened state here because + * we don't want to consider opened 1-hop dircon circuits for + * deciding when to relax the timeout, but we *do* want to relax + * those circuits too if nothing else is opened *and* they still + * aren't either. */ + if (!any_opened_circs && victim->state != CIRCUIT_STATE_OPEN) { + /* It's still young enough that we wouldn't close it, right? */ + if (timercmp(&victim->timestamp_began, &close_cutoff, OP_GT)) { + if (!TO_ORIGIN_CIRCUIT(victim)->relaxed_timeout) { + int first_hop_succeeded = TO_ORIGIN_CIRCUIT(victim)->cpath->state + == CPATH_STATE_OPEN; + if (!fixed_time) { + log_info(LD_CIRC, + "No circuits are opened. Relaxing timeout for circuit %d " + "(a %s %d-hop circuit in state %s with channel state %s).", + TO_ORIGIN_CIRCUIT(victim)->global_identifier, + circuit_purpose_to_string(victim->purpose), + TO_ORIGIN_CIRCUIT(victim)->build_state ? + TO_ORIGIN_CIRCUIT(victim)->build_state->desired_path_len : + -1, + circuit_state_to_string(victim->state), + victim->n_chan ? + channel_state_to_string(victim->n_chan->state) : "none"); + } + + /* We count the timeout here for CBT, because technically this + * was a timeout, and the timeout value needs to reset if we + * see enough of them. Note this means we also need to avoid + * double-counting below, too. */ + circuit_build_times_count_timeout(get_circuit_build_times_mutable(), + first_hop_succeeded); + TO_ORIGIN_CIRCUIT(victim)->relaxed_timeout = 1; + } + continue; + } else { + static ratelim_t relax_timeout_limit = RATELIM_INIT(3600); + const double build_close_ms = get_circuit_build_close_time_ms(); + if (!fixed_time) { + log_fn_ratelim(&relax_timeout_limit, LOG_NOTICE, LD_CIRC, + "No circuits are opened. Relaxed timeout for circuit %d " + "(a %s %d-hop circuit in state %s with channel state %s) to " + "%ldms. However, it appears the circuit has timed out " + "anyway.", + TO_ORIGIN_CIRCUIT(victim)->global_identifier, + circuit_purpose_to_string(victim->purpose), + TO_ORIGIN_CIRCUIT(victim)->build_state ? + TO_ORIGIN_CIRCUIT(victim)->build_state->desired_path_len : + -1, + circuit_state_to_string(victim->state), + victim->n_chan ? + channel_state_to_string(victim->n_chan->state) : "none", + (long)build_close_ms); + } + } + } + +#if 0 + /* some debug logs, to help track bugs */ + if (victim->purpose >= CIRCUIT_PURPOSE_C_INTRODUCING && + victim->purpose <= CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED) { + if (!victim->timestamp_dirty) + log_fn(LOG_DEBUG,"Considering %sopen purpose %d to %s (circid %d)." + "(clean).", + victim->state == CIRCUIT_STATE_OPEN ? "" : "non", + victim->purpose, victim->build_state->chosen_exit_name, + victim->n_circ_id); + else + log_fn(LOG_DEBUG,"Considering %sopen purpose %d to %s (circid %d). " + "%d secs since dirty.", + victim->state == CIRCUIT_STATE_OPEN ? "" : "non", + victim->purpose, victim->build_state->chosen_exit_name, + victim->n_circ_id, + (int)(now - victim->timestamp_dirty)); + } +#endif /* 0 */ + + /* if circ is !open, or if it's open but purpose is a non-finished + * intro or rend, then mark it for close */ + if (victim->state == CIRCUIT_STATE_OPEN) { + switch (victim->purpose) { + default: /* most open circuits can be left alone. */ + continue; /* yes, continue inside a switch refers to the nearest + * enclosing loop. C is smart. */ + case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: + break; /* too old, need to die */ + case CIRCUIT_PURPOSE_C_REND_READY: + /* it's a rend_ready circ -- has it already picked a query? */ + /* c_rend_ready circs measure age since timestamp_dirty, + * because that's set when they switch purposes + */ + if (TO_ORIGIN_CIRCUIT(victim)->rend_data || + TO_ORIGIN_CIRCUIT(victim)->hs_ident || + victim->timestamp_dirty > cutoff.tv_sec) + continue; + break; + case CIRCUIT_PURPOSE_PATH_BIAS_TESTING: + /* Open path bias testing circuits are given a long + * time to complete the test, but not forever */ + TO_ORIGIN_CIRCUIT(victim)->path_state = PATH_STATE_USE_FAILED; + break; + case CIRCUIT_PURPOSE_C_INTRODUCING: + /* That purpose means that the intro point circuit has been opened + * successfully but the INTRODUCE1 cell hasn't been sent yet because + * the client is waiting for the rendezvous point circuit to open. + * Keep this circuit open while waiting for the rendezvous circuit. + * We let the circuit idle timeout take care of cleaning this + * circuit if it never used. */ + continue; + case CIRCUIT_PURPOSE_C_ESTABLISH_REND: + case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED: + case CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT: + /* rend and intro circs become dirty each time they + * make an introduction attempt. so timestamp_dirty + * will reflect the time since the last attempt. + */ + if (victim->timestamp_dirty > cutoff.tv_sec) + continue; + break; + } + } else { /* circuit not open, consider recording failure as timeout */ + int first_hop_succeeded = TO_ORIGIN_CIRCUIT(victim)->cpath && + TO_ORIGIN_CIRCUIT(victim)->cpath->state == CPATH_STATE_OPEN; + + if (TO_ORIGIN_CIRCUIT(victim)->p_streams != NULL) { + log_warn(LD_BUG, "Circuit %d (purpose %d, %s) has timed out, " + "yet has attached streams!", + TO_ORIGIN_CIRCUIT(victim)->global_identifier, + victim->purpose, + circuit_purpose_to_string(victim->purpose)); + tor_fragile_assert(); + continue; + } + + if (circuit_timeout_want_to_count_circ(TO_ORIGIN_CIRCUIT(victim)) && + circuit_build_times_enough_to_compute(get_circuit_build_times())) { + + log_info(LD_CIRC, + "Deciding to count the timeout for circuit %"PRIu32"\n", + TO_ORIGIN_CIRCUIT(victim)->global_identifier); + + /* Circuits are allowed to last longer for measurement. + * Switch their purpose and wait. */ + if (victim->purpose != CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) { + circuit_build_times_mark_circ_as_measurement_only(TO_ORIGIN_CIRCUIT( + victim)); + continue; + } + + /* + * If the circuit build time is much greater than we would have cut + * it off at, we probably had a suspend event along this codepath, + * and we should discard the value. + */ + if (timercmp(&victim->timestamp_began, &extremely_old_cutoff, OP_LT)) { + log_notice(LD_CIRC, + "Extremely large value for circuit build timeout: %lds. " + "Assuming clock jump. Purpose %d (%s)", + (long)(now.tv_sec - victim->timestamp_began.tv_sec), + victim->purpose, + circuit_purpose_to_string(victim->purpose)); + } else if (circuit_build_times_count_close( + get_circuit_build_times_mutable(), + first_hop_succeeded, + (time_t)victim->timestamp_created.tv_sec)) { + circuit_build_times_set_timeout(get_circuit_build_times_mutable()); + } + } + } + + /* If this is a hidden service client circuit which is far enough along in + * connecting to its destination, and we haven't already flagged it as + * 'timed out', flag it so we'll launch another intro or rend circ, but + * don't mark it for close yet. + * + * (Circs flagged as 'timed out' are given a much longer timeout + * period above, so we won't close them in the next call to + * circuit_expire_building.) */ + if (!(TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out)) { + switch (victim->purpose) { + case CIRCUIT_PURPOSE_C_REND_READY: + /* We only want to spare a rend circ if it has been specified in + * an INTRODUCE1 cell sent to a hidden service. A circ's + * pending_final_cpath field is non-NULL iff it is a rend circ + * and we have tried to send an INTRODUCE1 cell specifying it. + * Thus, if the pending_final_cpath field *is* NULL, then we + * want to not spare it. */ + if (TO_ORIGIN_CIRCUIT(victim)->build_state && + TO_ORIGIN_CIRCUIT(victim)->build_state->pending_final_cpath == + NULL) + break; + /* fallthrough! */ + case CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT: + case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED: + /* If we have reached this line, we want to spare the circ for now. */ + log_info(LD_CIRC,"Marking circ %u (state %d:%s, purpose %d) " + "as timed-out HS circ", + (unsigned)victim->n_circ_id, + victim->state, circuit_state_to_string(victim->state), + victim->purpose); + TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out = 1; + continue; + default: + break; + } + } + + /* If this is a service-side rendezvous circuit which is far + * enough along in connecting to its destination, consider sparing + * it. */ + if (!(TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out) && + victim->purpose == CIRCUIT_PURPOSE_S_CONNECT_REND) { + log_info(LD_CIRC,"Marking circ %u (state %d:%s, purpose %d) " + "as timed-out HS circ; relaunching rendezvous attempt.", + (unsigned)victim->n_circ_id, + victim->state, circuit_state_to_string(victim->state), + victim->purpose); + TO_ORIGIN_CIRCUIT(victim)->hs_circ_has_timed_out = 1; + hs_circ_retry_service_rendezvous_point(TO_ORIGIN_CIRCUIT(victim)); + continue; + } + + if (victim->n_chan) + log_info(LD_CIRC, + "Abandoning circ %u %s:%u (state %d,%d:%s, purpose %d, " + "len %d)", TO_ORIGIN_CIRCUIT(victim)->global_identifier, + channel_get_canonical_remote_descr(victim->n_chan), + (unsigned)victim->n_circ_id, + TO_ORIGIN_CIRCUIT(victim)->has_opened, + victim->state, circuit_state_to_string(victim->state), + victim->purpose, + TO_ORIGIN_CIRCUIT(victim)->build_state ? + TO_ORIGIN_CIRCUIT(victim)->build_state->desired_path_len : + -1); + else + log_info(LD_CIRC, + "Abandoning circ %u %u (state %d,%d:%s, purpose %d, len %d)", + TO_ORIGIN_CIRCUIT(victim)->global_identifier, + (unsigned)victim->n_circ_id, + TO_ORIGIN_CIRCUIT(victim)->has_opened, + victim->state, + circuit_state_to_string(victim->state), victim->purpose, + TO_ORIGIN_CIRCUIT(victim)->build_state ? + TO_ORIGIN_CIRCUIT(victim)->build_state->desired_path_len : + -1); + + circuit_log_path(LOG_INFO,LD_CIRC,TO_ORIGIN_CIRCUIT(victim)); + if (victim->purpose == CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT) + circuit_mark_for_close(victim, END_CIRC_REASON_MEASUREMENT_EXPIRED); + else + circuit_mark_for_close(victim, END_CIRC_REASON_TIMEOUT); + + pathbias_count_timeout(TO_ORIGIN_CIRCUIT(victim)); + } SMARTLIST_FOREACH_END(victim); +} + +/** + * Mark for close all circuits that start here, that were built through a + * guard we weren't sure if we wanted to use, and that have been waiting + * around for way too long. + */ +void +circuit_expire_waiting_for_better_guard(void) +{ + SMARTLIST_FOREACH_BEGIN(circuit_get_global_origin_circuit_list(), + origin_circuit_t *, circ) { + if (TO_CIRCUIT(circ)->marked_for_close) + continue; + if (circ->guard_state == NULL) + continue; + if (entry_guard_state_should_expire(circ->guard_state)) + circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_REASON_NONE); + } SMARTLIST_FOREACH_END(circ); +} + +/** For debugging #8387: track when we last called + * circuit_expire_old_circuits_clientside. */ +static time_t last_expired_clientside_circuits = 0; + +/** + * As a diagnostic for bug 8387, log information about how many one-hop + * circuits we have around that have been there for at least age + * seconds. Log a few of them. Ignores Single Onion Service intro, it is + * expected to be long-term one-hop circuits. + */ +void +circuit_log_ancient_one_hop_circuits(int age) +{ +#define MAX_ANCIENT_ONEHOP_CIRCUITS_TO_LOG 10 + time_t now = time(NULL); + time_t cutoff = now - age; + int n_found = 0; + smartlist_t *log_these = smartlist_new(); + const or_options_t *options = get_options(); + + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *, circ) { + const origin_circuit_t *ocirc; + if (! CIRCUIT_IS_ORIGIN(circ)) + continue; + if (circ->timestamp_created.tv_sec >= cutoff) + continue; + /* Single Onion Services deliberately make long term one-hop intro + * and rendezvous connections. Don't log the established ones. */ + if (rend_service_allow_non_anonymous_connection(options) && + (circ->purpose == CIRCUIT_PURPOSE_S_INTRO || + circ->purpose == CIRCUIT_PURPOSE_S_REND_JOINED)) + continue; + ocirc = CONST_TO_ORIGIN_CIRCUIT(circ); + + if (ocirc->build_state && ocirc->build_state->onehop_tunnel) { + ++n_found; + + if (smartlist_len(log_these) < MAX_ANCIENT_ONEHOP_CIRCUITS_TO_LOG) + smartlist_add(log_these, (origin_circuit_t*) ocirc); + } + } + SMARTLIST_FOREACH_END(circ); + + if (n_found == 0) + goto done; + + log_notice(LD_HEARTBEAT, + "Diagnostic for issue 8387: Found %d one-hop circuits more " + "than %d seconds old! Logging %d...", + n_found, age, smartlist_len(log_these)); + + SMARTLIST_FOREACH_BEGIN(log_these, const origin_circuit_t *, ocirc) { + char created[ISO_TIME_LEN+1]; + int stream_num; + const edge_connection_t *conn; + char *dirty = NULL; + const circuit_t *circ = TO_CIRCUIT(ocirc); + + format_local_iso_time(created, + (time_t)circ->timestamp_created.tv_sec); + + if (circ->timestamp_dirty) { + char dirty_since[ISO_TIME_LEN+1]; + format_local_iso_time(dirty_since, circ->timestamp_dirty); + + tor_asprintf(&dirty, "Dirty since %s (%ld seconds vs %ld-second cutoff)", + dirty_since, (long)(now - circ->timestamp_dirty), + (long) options->MaxCircuitDirtiness); + } else { + dirty = tor_strdup("Not marked dirty"); + } + + log_notice(LD_HEARTBEAT, " #%d created at %s. %s, %s. %s for close. " + "Package window: %d. " + "%s for new conns. %s.", + ocirc_sl_idx, + created, + circuit_state_to_string(circ->state), + circuit_purpose_to_string(circ->purpose), + circ->marked_for_close ? "Marked" : "Not marked", + circ->package_window, + ocirc->unusable_for_new_conns ? "Not usable" : "usable", + dirty); + tor_free(dirty); + + stream_num = 0; + for (conn = ocirc->p_streams; conn; conn = conn->next_stream) { + const connection_t *c = TO_CONN(conn); + char stream_created[ISO_TIME_LEN+1]; + if (++stream_num >= 5) + break; + + format_local_iso_time(stream_created, c->timestamp_created); + + log_notice(LD_HEARTBEAT, " Stream#%d created at %s. " + "%s conn in state %s. " + "It is %slinked and %sreading from a linked connection %p. " + "Package window %d. " + "%s for close (%s:%d). Hold-open is %sset. " + "Has %ssent RELAY_END. %s on circuit.", + stream_num, + stream_created, + conn_type_to_string(c->type), + conn_state_to_string(c->type, c->state), + c->linked ? "" : "not ", + c->reading_from_linked_conn ? "": "not", + c->linked_conn, + conn->package_window, + c->marked_for_close ? "Marked" : "Not marked", + c->marked_for_close_file ? c->marked_for_close_file : "--", + c->marked_for_close, + c->hold_open_until_flushed ? "" : "not ", + conn->edge_has_sent_end ? "" : "not ", + conn->edge_blocked_on_circ ? "Blocked" : "Not blocked"); + if (! c->linked_conn) + continue; + + c = c->linked_conn; + + log_notice(LD_HEARTBEAT, " Linked to %s connection in state %s " + "(Purpose %d). %s for close (%s:%d). Hold-open is %sset. ", + conn_type_to_string(c->type), + conn_state_to_string(c->type, c->state), + c->purpose, + c->marked_for_close ? "Marked" : "Not marked", + c->marked_for_close_file ? c->marked_for_close_file : "--", + c->marked_for_close, + c->hold_open_until_flushed ? "" : "not "); + } + } SMARTLIST_FOREACH_END(ocirc); + + log_notice(LD_HEARTBEAT, "It has been %ld seconds since I last called " + "circuit_expire_old_circuits_clientside().", + (long)(now - last_expired_clientside_circuits)); + + done: + smartlist_free(log_these); +} + +/** Remove any elements in needed_ports that are handled by an + * open or in-progress circuit. + */ +void +circuit_remove_handled_ports(smartlist_t *needed_ports) +{ + int i; + uint16_t *port; + + for (i = 0; i < smartlist_len(needed_ports); ++i) { + port = smartlist_get(needed_ports, i); + tor_assert(*port); + if (circuit_stream_is_being_handled(NULL, *port, + MIN_CIRCUITS_HANDLING_STREAM)) { + log_debug(LD_CIRC,"Port %d is already being handled; removing.", *port); + smartlist_del(needed_ports, i--); + tor_free(port); + } else { + log_debug(LD_CIRC,"Port %d is not handled.", *port); + } + } +} + +/** Return 1 if at least min general-purpose non-internal circuits + * will have an acceptable exit node for exit stream conn if it + * is defined, else for "*:port". + * Else return 0. + */ +int +circuit_stream_is_being_handled(entry_connection_t *conn, + uint16_t port, int min) +{ + const node_t *exitnode; + int num=0; + time_t now = time(NULL); + int need_uptime = smartlist_contains_int_as_string( + get_options()->LongLivedPorts, + conn ? conn->socks_request->port : port); + + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *, circ) { + if (CIRCUIT_IS_ORIGIN(circ) && + !circ->marked_for_close && + circ->purpose == CIRCUIT_PURPOSE_C_GENERAL && + (!circ->timestamp_dirty || + circ->timestamp_dirty + get_options()->MaxCircuitDirtiness > now)) { + origin_circuit_t *origin_circ = TO_ORIGIN_CIRCUIT(circ); + cpath_build_state_t *build_state = origin_circ->build_state; + if (build_state->is_internal || build_state->onehop_tunnel) + continue; + if (origin_circ->unusable_for_new_conns) + continue; + if (origin_circ->isolation_values_set && + (conn == NULL || + !connection_edge_compatible_with_circuit(conn, origin_circ))) + continue; + + exitnode = build_state_get_exit_node(build_state); + if (exitnode && (!need_uptime || build_state->need_uptime)) { + int ok; + if (conn) { + ok = connection_ap_can_use_exit(conn, exitnode); + } else { + addr_policy_result_t r; + r = compare_tor_addr_to_node_policy(NULL, port, exitnode); + ok = r != ADDR_POLICY_REJECTED && r != ADDR_POLICY_PROBABLY_REJECTED; + } + if (ok) { + if (++num >= min) + return 1; + } + } + } + } + SMARTLIST_FOREACH_END(circ); + return 0; +} + +/** Don't keep more than this many unused open circuits around. */ +#define MAX_UNUSED_OPEN_CIRCUITS 14 + +/* Return true if a circuit is available for use, meaning that it is open, + * clean, usable for new multi-hop connections, and a general purpose origin + * circuit. + * Accept any kind of circuit, return false if the above conditions are not + * met. */ +STATIC int +circuit_is_available_for_use(const circuit_t *circ) +{ + const origin_circuit_t *origin_circ; + cpath_build_state_t *build_state; + + if (!CIRCUIT_IS_ORIGIN(circ)) + return 0; /* We first filter out only origin circuits before doing the + following checks. */ + if (circ->marked_for_close) + return 0; /* Don't mess with marked circs */ + if (circ->timestamp_dirty) + return 0; /* Only count clean circs */ + if (circ->purpose != CIRCUIT_PURPOSE_C_GENERAL && + circ->purpose != CIRCUIT_PURPOSE_HS_VANGUARDS) + return 0; /* We only pay attention to general purpose circuits. + General purpose circuits are always origin circuits. */ + + origin_circ = CONST_TO_ORIGIN_CIRCUIT(circ); + if (origin_circ->unusable_for_new_conns) + return 0; + + build_state = origin_circ->build_state; + if (build_state->onehop_tunnel) + return 0; + + return 1; +} + +/* Return true if we need any more exit circuits. + * needs_uptime and needs_capacity are set only if we need more exit circuits. + * Check if we know of a port that's been requested recently and no circuit + * is currently available that can handle it. */ +STATIC int +needs_exit_circuits(time_t now, int *needs_uptime, int *needs_capacity) +{ + return (!circuit_all_predicted_ports_handled(now, needs_uptime, + needs_capacity) && + router_have_consensus_path() == CONSENSUS_PATH_EXIT); +} + +/* Hidden services need at least this many internal circuits */ +#define SUFFICIENT_UPTIME_INTERNAL_HS_SERVERS 3 + +/* Return true if we need any more hidden service server circuits. + * HS servers only need an internal circuit. */ +STATIC int +needs_hs_server_circuits(time_t now, int num_uptime_internal) +{ + if (!rend_num_services() && !hs_service_get_num_services()) { + /* No services, we don't need anything. */ + goto no_need; + } + + if (num_uptime_internal >= SUFFICIENT_UPTIME_INTERNAL_HS_SERVERS) { + /* We have sufficient amount of internal circuit. */ + goto no_need; + } + + if (router_have_consensus_path() == CONSENSUS_PATH_UNKNOWN) { + /* Consensus hasn't been checked or might be invalid so requesting + * internal circuits is not wise. */ + goto no_need; + } + + /* At this point, we need a certain amount of circuits and we will most + * likely use them for rendezvous so we note down the use of internal + * circuit for our prediction for circuit needing uptime and capacity. */ + rep_hist_note_used_internal(now, 1, 1); + + return 1; + no_need: + return 0; +} + +/* We need at least this many internal circuits for hidden service clients */ +#define SUFFICIENT_INTERNAL_HS_CLIENTS 3 + +/* We need at least this much uptime for internal circuits for hidden service + * clients */ +#define SUFFICIENT_UPTIME_INTERNAL_HS_CLIENTS 2 + +/* Return true if we need any more hidden service client circuits. + * HS clients only need an internal circuit. */ +STATIC int +needs_hs_client_circuits(time_t now, int *needs_uptime, int *needs_capacity, + int num_internal, int num_uptime_internal) +{ + int used_internal_recently = rep_hist_get_predicted_internal(now, + needs_uptime, + needs_capacity); + int requires_uptime = num_uptime_internal < + SUFFICIENT_UPTIME_INTERNAL_HS_CLIENTS && + needs_uptime; + + return (used_internal_recently && + (requires_uptime || num_internal < SUFFICIENT_INTERNAL_HS_CLIENTS) && + router_have_consensus_path() != CONSENSUS_PATH_UNKNOWN); +} + +/* This is how many circuits can be opened concurrently during the cbt learning + * phase. This number cannot exceed the tor-wide MAX_UNUSED_OPEN_CIRCUITS. */ +#define DFLT_CBT_UNUSED_OPEN_CIRCS (10) +#define MIN_CBT_UNUSED_OPEN_CIRCS 0 +#define MAX_CBT_UNUSED_OPEN_CIRCS MAX_UNUSED_OPEN_CIRCUITS + +/* Return true if we need more circuits for a good build timeout. + * XXXX make the assumption that build timeout streams should be + * created whenever we can build internal circuits. */ +STATIC int +needs_circuits_for_build(int num) +{ + if (router_have_consensus_path() != CONSENSUS_PATH_UNKNOWN) { + if (num < networkstatus_get_param(NULL, "cbtmaxopencircs", + DFLT_CBT_UNUSED_OPEN_CIRCS, + MIN_CBT_UNUSED_OPEN_CIRCS, + MAX_CBT_UNUSED_OPEN_CIRCS) && + !circuit_build_times_disabled(get_options()) && + circuit_build_times_needs_circuits_now(get_circuit_build_times())) { + return 1; + } + } + return 0; +} + +/** + * Launch the appropriate type of predicted circuit for hidden + * services, depending on our options. + */ +static void +circuit_launch_predicted_hs_circ(int flags) +{ + /* K.I.S.S. implementation of bug #23101: If we are using + * vanguards or pinned middles, pre-build a specific purpose + * for HS circs. */ + if (circuit_should_use_vanguards(CIRCUIT_PURPOSE_HS_VANGUARDS)) { + circuit_launch(CIRCUIT_PURPOSE_HS_VANGUARDS, flags); + } else { + /* If no vanguards, then no HS-specific prebuilt circuits are needed. + * Normal GENERAL circs are fine */ + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, flags); + } +} + +/** Determine how many circuits we have open that are clean, + * Make sure it's enough for all the upcoming behaviors we predict we'll have. + * But put an upper bound on the total number of circuits. + */ +static void +circuit_predict_and_launch_new(void) +{ + int num=0, num_internal=0, num_uptime_internal=0; + int hidserv_needs_uptime=0, hidserv_needs_capacity=1; + int port_needs_uptime=0, port_needs_capacity=1; + time_t now = time(NULL); + int flags = 0; + + /* Count how many of each type of circuit we currently have. */ + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *, circ) { + if (!circuit_is_available_for_use(circ)) + continue; + + num++; + + cpath_build_state_t *build_state = TO_ORIGIN_CIRCUIT(circ)->build_state; + if (build_state->is_internal) + num_internal++; + if (build_state->need_uptime && build_state->is_internal) + num_uptime_internal++; + } + SMARTLIST_FOREACH_END(circ); + + /* If that's enough, then stop now. */ + if (num >= MAX_UNUSED_OPEN_CIRCUITS) + return; + + if (needs_exit_circuits(now, &port_needs_uptime, &port_needs_capacity)) { + if (port_needs_uptime) + flags |= CIRCLAUNCH_NEED_UPTIME; + if (port_needs_capacity) + flags |= CIRCLAUNCH_NEED_CAPACITY; + + log_info(LD_CIRC, + "Have %d clean circs (%d internal), need another exit circ.", + num, num_internal); + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, flags); + return; + } + + if (needs_hs_server_circuits(now, num_uptime_internal)) { + flags = (CIRCLAUNCH_NEED_CAPACITY | CIRCLAUNCH_NEED_UPTIME | + CIRCLAUNCH_IS_INTERNAL); + + log_info(LD_CIRC, + "Have %d clean circs (%d internal), need another internal " + "circ for my hidden service.", + num, num_internal); + circuit_launch_predicted_hs_circ(flags); + return; + } + + if (needs_hs_client_circuits(now, &hidserv_needs_uptime, + &hidserv_needs_capacity, + num_internal, num_uptime_internal)) + { + if (hidserv_needs_uptime) + flags |= CIRCLAUNCH_NEED_UPTIME; + if (hidserv_needs_capacity) + flags |= CIRCLAUNCH_NEED_CAPACITY; + flags |= CIRCLAUNCH_IS_INTERNAL; + + log_info(LD_CIRC, + "Have %d clean circs (%d uptime-internal, %d internal), need" + " another hidden service circ.", + num, num_uptime_internal, num_internal); + + circuit_launch_predicted_hs_circ(flags); + return; + } + + if (needs_circuits_for_build(num)) { + flags = CIRCLAUNCH_NEED_CAPACITY; + /* if there are no exits in the consensus, make timeout + * circuits internal */ + if (router_have_consensus_path() == CONSENSUS_PATH_INTERNAL) + flags |= CIRCLAUNCH_IS_INTERNAL; + + log_info(LD_CIRC, + "Have %d clean circs need another buildtime test circ.", num); + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, flags); + return; + } +} + +/** Build a new test circuit every 5 minutes */ +#define TESTING_CIRCUIT_INTERVAL 300 + +/** This function is called once a second, if router_have_minimum_dir_info() + * is true. Its job is to make sure all services we offer have enough circuits + * available. Some services just want enough circuits for current tasks, + * whereas others want a minimum set of idle circuits hanging around. + */ +void +circuit_build_needed_circs(time_t now) +{ + const or_options_t *options = get_options(); + + /* launch a new circ for any pending streams that need one + * XXXX make the assumption that (some) AP streams (i.e. HS clients) + * don't require an exit circuit, review in #13814. + * This allows HSs to function in a consensus without exits. */ + if (router_have_consensus_path() != CONSENSUS_PATH_UNKNOWN) + connection_ap_rescan_and_attach_pending(); + + circuit_expire_old_circs_as_needed(now); + + if (!options->DisablePredictedCircuits) + circuit_predict_and_launch_new(); +} + +/** + * Called once a second either directly or from + * circuit_build_needed_circs(). As appropriate (once per NewCircuitPeriod) + * resets failure counts and expires old circuits. + */ +void +circuit_expire_old_circs_as_needed(time_t now) +{ + static time_t time_to_expire_and_reset = 0; + + if (time_to_expire_and_reset < now) { + circuit_reset_failure_count(1); + time_to_expire_and_reset = now + get_options()->NewCircuitPeriod; + if (proxy_mode(get_options())) + addressmap_clean(now); + circuit_expire_old_circuits_clientside(); + +#if 0 /* disable for now, until predict-and-launch-new can cull leftovers */ + + /* If we ever re-enable, this has to move into + * circuit_build_needed_circs */ + + circ = circuit_get_youngest_clean_open(CIRCUIT_PURPOSE_C_GENERAL); + if (get_options()->RunTesting && + circ && + circ->timestamp_began.tv_sec + TESTING_CIRCUIT_INTERVAL < now) { + log_fn(LOG_INFO,"Creating a new testing circuit."); + circuit_launch(CIRCUIT_PURPOSE_C_GENERAL, 0); + } +#endif /* 0 */ + } +} + +/** If the stream conn is a member of any of the linked + * lists of circ, then remove it from the list. + */ +void +circuit_detach_stream(circuit_t *circ, edge_connection_t *conn) +{ + edge_connection_t *prevconn; + + tor_assert(circ); + tor_assert(conn); + + if (conn->base_.type == CONN_TYPE_AP) { + entry_connection_t *entry_conn = EDGE_TO_ENTRY_CONN(conn); + entry_conn->may_use_optimistic_data = 0; + } + conn->cpath_layer = NULL; /* don't keep a stale pointer */ + conn->on_circuit = NULL; + + if (CIRCUIT_IS_ORIGIN(circ)) { + origin_circuit_t *origin_circ = TO_ORIGIN_CIRCUIT(circ); + int removed = 0; + if (conn == origin_circ->p_streams) { + origin_circ->p_streams = conn->next_stream; + removed = 1; + } else { + for (prevconn = origin_circ->p_streams; + prevconn && prevconn->next_stream && prevconn->next_stream != conn; + prevconn = prevconn->next_stream) + ; + if (prevconn && prevconn->next_stream) { + prevconn->next_stream = conn->next_stream; + removed = 1; + } + } + if (removed) { + log_debug(LD_APP, "Removing stream %d from circ %u", + conn->stream_id, (unsigned)circ->n_circ_id); + + /* If the stream was removed, and it was a rend stream, decrement the + * number of streams on the circuit associated with the rend service. + */ + if (circ->purpose == CIRCUIT_PURPOSE_S_REND_JOINED) { + hs_dec_rdv_stream_counter(origin_circ); + } + return; + } + } else { + or_circuit_t *or_circ = TO_OR_CIRCUIT(circ); + if (conn == or_circ->n_streams) { + or_circ->n_streams = conn->next_stream; + return; + } + if (conn == or_circ->resolving_streams) { + or_circ->resolving_streams = conn->next_stream; + return; + } + + for (prevconn = or_circ->n_streams; + prevconn && prevconn->next_stream && prevconn->next_stream != conn; + prevconn = prevconn->next_stream) + ; + if (prevconn && prevconn->next_stream) { + prevconn->next_stream = conn->next_stream; + return; + } + + for (prevconn = or_circ->resolving_streams; + prevconn && prevconn->next_stream && prevconn->next_stream != conn; + prevconn = prevconn->next_stream) + ; + if (prevconn && prevconn->next_stream) { + prevconn->next_stream = conn->next_stream; + return; + } + } + + log_warn(LD_BUG,"Edge connection not in circuit's list."); + /* Don't give an error here; it's harmless. */ + tor_fragile_assert(); +} + +/** Find each circuit that has been unused for too long, or dirty + * for too long and has no streams on it: mark it for close. + */ +static void +circuit_expire_old_circuits_clientside(void) +{ + struct timeval cutoff, now; + + tor_gettimeofday(&now); + last_expired_clientside_circuits = now.tv_sec; + + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *, circ) { + if (circ->marked_for_close || !CIRCUIT_IS_ORIGIN(circ)) + continue; + + cutoff = now; + cutoff.tv_sec -= TO_ORIGIN_CIRCUIT(circ)->circuit_idle_timeout; + + /* If the circuit has been dirty for too long, and there are no streams + * on it, mark it for close. + */ + if (circ->timestamp_dirty && + circ->timestamp_dirty + get_options()->MaxCircuitDirtiness < + now.tv_sec && + !TO_ORIGIN_CIRCUIT(circ)->p_streams /* nothing attached */ ) { + log_debug(LD_CIRC, "Closing n_circ_id %u (dirty %ld sec ago, " + "purpose %d)", + (unsigned)circ->n_circ_id, + (long)(now.tv_sec - circ->timestamp_dirty), + circ->purpose); + /* Don't do this magic for testing circuits. Their death is governed + * by circuit_expire_building */ + if (circ->purpose != CIRCUIT_PURPOSE_PATH_BIAS_TESTING) + circuit_mark_for_close(circ, END_CIRC_REASON_FINISHED); + } else if (!circ->timestamp_dirty && circ->state == CIRCUIT_STATE_OPEN) { + if (timercmp(&circ->timestamp_began, &cutoff, OP_LT)) { + if (circ->purpose == CIRCUIT_PURPOSE_C_GENERAL || + circ->purpose == CIRCUIT_PURPOSE_C_HSDIR_GET || + circ->purpose == CIRCUIT_PURPOSE_S_HSDIR_POST || + circ->purpose == CIRCUIT_PURPOSE_HS_VANGUARDS || + circ->purpose == CIRCUIT_PURPOSE_C_MEASURE_TIMEOUT || + circ->purpose == CIRCUIT_PURPOSE_S_ESTABLISH_INTRO || + circ->purpose == CIRCUIT_PURPOSE_TESTING || + (circ->purpose >= CIRCUIT_PURPOSE_C_INTRODUCING && + circ->purpose <= CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED) || + circ->purpose == CIRCUIT_PURPOSE_S_CONNECT_REND) { + log_info(LD_CIRC, + "Closing circuit %"PRIu32 + " that has been unused for %ld msec.", + TO_ORIGIN_CIRCUIT(circ)->global_identifier, + tv_mdiff(&circ->timestamp_began, &now)); + circuit_mark_for_close(circ, END_CIRC_REASON_FINISHED); + } else if (!TO_ORIGIN_CIRCUIT(circ)->is_ancient) { + /* Server-side rend joined circuits can end up really old, because + * they are reused by clients for longer than normal. The client + * controls their lifespan. (They never become dirty, because + * connection_exit_begin_conn() never marks anything as dirty.) + * Similarly, server-side intro circuits last a long time. */ + if (circ->purpose != CIRCUIT_PURPOSE_S_REND_JOINED && + circ->purpose != CIRCUIT_PURPOSE_S_INTRO) { + log_notice(LD_CIRC, + "Ancient non-dirty circuit %d is still around after " + "%ld milliseconds. Purpose: %d (%s)", + TO_ORIGIN_CIRCUIT(circ)->global_identifier, + tv_mdiff(&circ->timestamp_began, &now), + circ->purpose, + circuit_purpose_to_string(circ->purpose)); + TO_ORIGIN_CIRCUIT(circ)->is_ancient = 1; + } + } + } + } + } SMARTLIST_FOREACH_END(circ); +} + +/** How long do we wait before killing circuits with the properties + * described below? + * + * Probably we could choose a number here as low as 5 to 10 seconds, + * since these circs are used for begindir, and a) generally you either + * ask another begindir question right after or you don't for a long time, + * b) clients at least through 0.2.1.x choose from the whole set of + * directory mirrors at each choice, and c) re-establishing a one-hop + * circuit via create-fast is a light operation assuming the TLS conn is + * still there. + * + * I expect "b" to go away one day when we move to using directory + * guards, but I think "a" and "c" are good enough reasons that a low + * number is safe even then. + */ +#define IDLE_ONE_HOP_CIRC_TIMEOUT 60 + +/** Find each non-origin circuit that has been unused for too long, + * has no streams on it, came from a client, and ends here: mark it + * for close. + */ +void +circuit_expire_old_circuits_serverside(time_t now) +{ + or_circuit_t *or_circ; + time_t cutoff = now - IDLE_ONE_HOP_CIRC_TIMEOUT; + + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *, circ) { + if (circ->marked_for_close || CIRCUIT_IS_ORIGIN(circ)) + continue; + or_circ = TO_OR_CIRCUIT(circ); + /* If the circuit has been idle for too long, and there are no streams + * on it, and it ends here, and it used a create_fast, mark it for close. ++ * ++ * Also if there is a rend_splice on it, it's a single onion service ++ * circuit and we should not close it. + */ + if (or_circ->p_chan && channel_is_client(or_circ->p_chan) && + !circ->n_chan && + !or_circ->n_streams && !or_circ->resolving_streams && ++ !or_circ->rend_splice && + channel_when_last_xmit(or_circ->p_chan) <= cutoff) { + log_info(LD_CIRC, "Closing circ_id %u (empty %d secs ago)", + (unsigned)or_circ->p_circ_id, + (int)(now - channel_when_last_xmit(or_circ->p_chan))); + circuit_mark_for_close(circ, END_CIRC_REASON_FINISHED); + } + } + SMARTLIST_FOREACH_END(circ); +} + +/** Number of testing circuits we want open before testing our bandwidth. */ +#define NUM_PARALLEL_TESTING_CIRCS 4 + +/** True iff we've ever had enough testing circuits open to test our + * bandwidth. */ +static int have_performed_bandwidth_test = 0; + +/** Reset have_performed_bandwidth_test, so we'll start building + * testing circuits again so we can exercise our bandwidth. */ +void +reset_bandwidth_test(void) +{ + have_performed_bandwidth_test = 0; +} + +/** Return 1 if we've already exercised our bandwidth, or if we + * have fewer than NUM_PARALLEL_TESTING_CIRCS testing circuits + * established or on the way. Else return 0. + */ +int +circuit_enough_testing_circs(void) +{ + int num = 0; + + if (have_performed_bandwidth_test) + return 1; + + SMARTLIST_FOREACH_BEGIN(circuit_get_global_list(), circuit_t *, circ) { + if (!circ->marked_for_close && CIRCUIT_IS_ORIGIN(circ) && + circ->purpose == CIRCUIT_PURPOSE_TESTING && + circ->state == CIRCUIT_STATE_OPEN) + num++; + } + SMARTLIST_FOREACH_END(circ); + return num >= NUM_PARALLEL_TESTING_CIRCS; +} + +/** A testing circuit has completed. Take whatever stats we want. + * Noticing reachability is taken care of in onionskin_answer(), + * so there's no need to record anything here. But if we still want + * to do the bandwidth test, and we now have enough testing circuits + * open, do it. + */ +static void +circuit_testing_opened(origin_circuit_t *circ) +{ + if (have_performed_bandwidth_test || + !check_whether_orport_reachable(get_options())) { + /* either we've already done everything we want with testing circuits, + * or this testing circuit became open due to a fluke, e.g. we picked + * a last hop where we already had the connection open due to an + * outgoing local circuit. */ + circuit_mark_for_close(TO_CIRCUIT(circ), END_CIRC_AT_ORIGIN); + } else if (circuit_enough_testing_circs()) { + router_perform_bandwidth_test(NUM_PARALLEL_TESTING_CIRCS, time(NULL)); + have_performed_bandwidth_test = 1; + } else + router_do_reachability_checks(1, 0); +} + +/** A testing circuit has failed to build. Take whatever stats we want. */ +static void +circuit_testing_failed(origin_circuit_t *circ, int at_last_hop) +{ + const or_options_t *options = get_options(); + if (server_mode(options) && check_whether_orport_reachable(options)) + return; + + log_info(LD_GENERAL, + "Our testing circuit (to see if your ORPort is reachable) " + "has failed. I'll try again later."); + + /* These aren't used yet. */ + (void)circ; + (void)at_last_hop; +} + +/** The circuit circ has just become open. Take the next + * step: for rendezvous circuits, we pass circ to the appropriate + * function in rendclient or rendservice. For general circuits, we + * call connection_ap_attach_pending, which looks for pending streams + * that could use circ. + */ +void +circuit_has_opened(origin_circuit_t *circ) +{ + control_event_circuit_status(circ, CIRC_EVENT_BUILT, 0); + + /* Remember that this circuit has finished building. Now if we start + * it building again later (e.g. by extending it), we will know not + * to consider its build time. */ + circ->has_opened = 1; + + switch (TO_CIRCUIT(circ)->purpose) { + case CIRCUIT_PURPOSE_C_ESTABLISH_REND: + hs_client_circuit_has_opened(circ); + /* Start building an intro circ if we don't have one yet. */ + connection_ap_attach_pending(1); + /* This isn't a call to circuit_try_attaching_streams because a + * circuit in _C_ESTABLISH_REND state isn't connected to its + * hidden service yet, thus we can't attach streams to it yet, + * thus circuit_try_attaching_streams would always clear the + * circuit's isolation state. circuit_try_attaching_streams is + * called later, when the rend circ enters _C_REND_JOINED + * state. */ + break; + case CIRCUIT_PURPOSE_C_INTRODUCING: + hs_client_circuit_has_opened(circ); + break; + case CIRCUIT_PURPOSE_C_GENERAL: + case CIRCUIT_PURPOSE_C_HSDIR_GET: + case CIRCUIT_PURPOSE_S_HSDIR_POST: + /* Tell any AP connections that have been waiting for a new + * circuit that one is ready. */ + circuit_try_attaching_streams(circ); + break; + case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: + /* at the service, waiting for introductions */ + hs_service_circuit_has_opened(circ); + break; + case CIRCUIT_PURPOSE_S_CONNECT_REND: + /* at the service, connecting to rend point */ + hs_service_circuit_has_opened(circ); + break; + case CIRCUIT_PURPOSE_TESTING: + circuit_testing_opened(circ); + break; + /* default: + * This won't happen in normal operation, but might happen if the + * controller did it. Just let it slide. */ + } +} + +/** If the stream-isolation state of circ can be cleared, clear + * it. Return non-zero iff circ's isolation state was cleared. */ +static int +circuit_try_clearing_isolation_state(origin_circuit_t *circ) +{ + if (/* The circuit may have become non-open if it was cannibalized.*/ + circ->base_.state == CIRCUIT_STATE_OPEN && + /* If !isolation_values_set, there is nothing to clear. */ + circ->isolation_values_set && + /* It's not legal to clear a circuit's isolation info if it's ever had + * streams attached */ + !circ->isolation_any_streams_attached) { + /* If we have any isolation information set on this circuit, and + * we didn't manage to attach any streams to it, then we can + * and should clear it and try again. */ + circuit_clear_isolation(circ); + return 1; + } else { + return 0; + } +} + +/** Called when a circuit becomes ready for streams to be attached to + * it. */ +void +circuit_try_attaching_streams(origin_circuit_t *circ) +{ + /* Attach streams to this circuit if we can. */ + connection_ap_attach_pending(1); + + /* The call to circuit_try_clearing_isolation_state here will do + * nothing and return 0 if we didn't attach any streams to circ + * above. */ + if (circuit_try_clearing_isolation_state(circ)) { + /* Maybe *now* we can attach some streams to this circuit. */ + connection_ap_attach_pending(1); + } +} + +/** Called whenever a circuit could not be successfully built. + */ +void +circuit_build_failed(origin_circuit_t *circ) +{ + channel_t *n_chan = NULL; + /* we should examine circ and see if it failed because of + * the last hop or an earlier hop. then use this info below. + */ + int failed_at_last_hop = 0; + + /* First, check to see if this was a path failure, rather than build + * failure. + * + * Note that we deliberately use circuit_get_cpath_len() (and not + * circuit_get_cpath_opened_len()) because we only want to ensure + * that a full path is *chosen*. This is different than a full path + * being *built*. We only want to count *build* failures below. + * + * Path selection failures can happen spuriously for a number + * of reasons (such as aggressive/invalid user-specified path + * restrictions in the torrc, insufficient microdescriptors, and + * non-user reasons like exitpolicy issues), and so should not be + * counted as failures below. + */ + if (circuit_get_cpath_len(circ) < circ->build_state->desired_path_len) { + static ratelim_t pathfail_limit = RATELIM_INIT(3600); + log_fn_ratelim(&pathfail_limit, LOG_NOTICE, LD_CIRC, + "Our circuit %u (id: %" PRIu32 ") died due to an invalid " + "selected path, purpose %s. This may be a torrc " + "configuration issue, or a bug.", + TO_CIRCUIT(circ)->n_circ_id, circ->global_identifier, + circuit_purpose_to_string(TO_CIRCUIT(circ)->purpose)); + + /* If the path failed on an RP, retry it. */ + if (TO_CIRCUIT(circ)->purpose == CIRCUIT_PURPOSE_S_CONNECT_REND) + hs_circ_retry_service_rendezvous_point(circ); + + /* In all other cases, just bail. The rest is just failure accounting + * that we don't want to do */ + return; + } + + /* If the last hop isn't open, and the second-to-last is, we failed + * at the last hop. */ + if (circ->cpath && + circ->cpath->prev->state != CPATH_STATE_OPEN && + circ->cpath->prev->prev->state == CPATH_STATE_OPEN) { + failed_at_last_hop = 1; + } + + /* Check if we failed at first hop */ + if (circ->cpath && + circ->cpath->state != CPATH_STATE_OPEN && + ! circ->base_.received_destroy) { + /* We failed at the first hop for some reason other than a DESTROY cell. + * If there's an OR connection to blame, blame it. Also, avoid this relay + * for a while, and fail any one-hop directory fetches destined for it. */ + const char *n_chan_ident = circ->cpath->extend_info->identity_digest; + tor_assert(n_chan_ident); + int already_marked = 0; + if (circ->base_.n_chan) { + n_chan = circ->base_.n_chan; + + if (n_chan->is_bad_for_new_circs) { + /* We only want to blame this router when a fresh healthy + * connection fails. So don't mark this router as newly failed, + * since maybe this was just an old circuit attempt that's + * finally timing out now. Also, there's no need to blow away + * circuits/streams/etc, since the failure of an unhealthy conn + * doesn't tell us much about whether a healthy conn would + * succeed. */ + already_marked = 1; + } + log_info(LD_OR, + "Our circuit %u (id: %" PRIu32 ") failed to get a response " + "from the first hop (%s). I'm going to try to rotate to a " + "better connection.", + TO_CIRCUIT(circ)->n_circ_id, circ->global_identifier, + channel_get_canonical_remote_descr(n_chan)); + n_chan->is_bad_for_new_circs = 1; + } else { + log_info(LD_OR, + "Our circuit %u (id: %" PRIu32 ") died before the first hop " + "with no connection", + TO_CIRCUIT(circ)->n_circ_id, circ->global_identifier); + } + if (!already_marked) { + /* + * If we have guard state (new guard API) and our path selection + * code actually chose a full path, then blame the failure of this + * circuit on the guard. + */ + if (circ->guard_state) + entry_guard_failed(&circ->guard_state); + /* if there are any one-hop streams waiting on this circuit, fail + * them now so they can retry elsewhere. */ + connection_ap_fail_onehop(n_chan_ident, circ->build_state); + } + } + + switch (circ->base_.purpose) { + case CIRCUIT_PURPOSE_C_HSDIR_GET: + case CIRCUIT_PURPOSE_S_HSDIR_POST: + case CIRCUIT_PURPOSE_C_GENERAL: + /* If we never built the circuit, note it as a failure. */ + circuit_increment_failure_count(); + if (failed_at_last_hop) { + /* Make sure any streams that demand our last hop as their exit + * know that it's unlikely to happen. */ + circuit_discard_optional_exit_enclaves(circ->cpath->prev->extend_info); + } + break; + case CIRCUIT_PURPOSE_TESTING: + circuit_testing_failed(circ, failed_at_last_hop); + break; + case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: + /* at the service, waiting for introductions */ + if (circ->base_.state != CIRCUIT_STATE_OPEN) { + circuit_increment_failure_count(); + } + /* no need to care here, because the service will rebuild intro + * points periodically. */ + break; + case CIRCUIT_PURPOSE_C_INTRODUCING: + /* at the client, connecting to intro point */ + /* Don't increment failure count, since the service may have picked + * the introduction point maliciously */ + /* The client will pick a new intro point when this one dies, if + * the stream in question still cares. No need to act here. */ + break; + case CIRCUIT_PURPOSE_C_ESTABLISH_REND: + /* at the client, waiting for the service */ + circuit_increment_failure_count(); + /* the client will pick a new rend point when this one dies, if + * the stream in question still cares. No need to act here. */ + break; + case CIRCUIT_PURPOSE_S_CONNECT_REND: + /* at the service, connecting to rend point */ + /* Don't increment failure count, since the client may have picked + * the rendezvous point maliciously */ + log_info(LD_REND, + "Couldn't connect to the client's chosen rend point %s " + "(%s hop failed).", + escaped(build_state_get_exit_nickname(circ->build_state)), + failed_at_last_hop?"last":"non-last"); + hs_circ_retry_service_rendezvous_point(circ); + break; + /* default: + * This won't happen in normal operation, but might happen if the + * controller did it. Just let it slide. */ + } +} + +/** Number of consecutive failures so far; should only be touched by + * circuit_launch_new and circuit_*_failure_count. + */ +static int n_circuit_failures = 0; +/** Before the last time we called circuit_reset_failure_count(), were + * there a lot of failures? */ +static int did_circs_fail_last_period = 0; + +/** Don't retry launching a new circuit if we try this many times with no + * success. */ +#define MAX_CIRCUIT_FAILURES 5 + +/** Launch a new circuit; see circuit_launch_by_extend_info() for + * details on arguments. */ +origin_circuit_t * +circuit_launch(uint8_t purpose, int flags) +{ + return circuit_launch_by_extend_info(purpose, NULL, flags); +} + +/* Do we have enough descriptors to build paths? + * If need_exit is true, return 1 if we can build exit paths. + * (We need at least one Exit in the consensus to build exit paths.) + * If need_exit is false, return 1 if we can build internal paths. + */ +static int +have_enough_path_info(int need_exit) +{ + if (need_exit) + return router_have_consensus_path() == CONSENSUS_PATH_EXIT; + else + return router_have_consensus_path() != CONSENSUS_PATH_UNKNOWN; +} + +/** + * Tell us if a circuit is a hidden service circuit. + */ +int +circuit_purpose_is_hidden_service(uint8_t purpose) +{ + if (purpose == CIRCUIT_PURPOSE_HS_VANGUARDS) { + return 1; + } + + /* Client-side purpose */ + if (purpose >= CIRCUIT_PURPOSE_C_HS_MIN_ && + purpose <= CIRCUIT_PURPOSE_C_HS_MAX_) { + return 1; + } + + /* Service-side purpose */ + if (purpose >= CIRCUIT_PURPOSE_S_HS_MIN_ && + purpose <= CIRCUIT_PURPOSE_S_HS_MAX_) { + return 1; + } + + return 0; +} + +/** + * Return true if this circuit purpose should use vanguards + * or pinned Layer2 or Layer3 guards. + * + * This function takes both the circuit purpose and the + * torrc options for pinned middles/vanguards into account + * (ie: the circuit must be a hidden service circuit and + * vanguards/pinned middles must be enabled for it to return + * true). + */ +int +circuit_should_use_vanguards(uint8_t purpose) +{ + const or_options_t *options = get_options(); + + /* Only hidden service circuits use vanguards */ + if (!circuit_purpose_is_hidden_service(purpose)) + return 0; + + /* Pinned middles are effectively vanguards */ + if (options->HSLayer2Nodes || options->HSLayer3Nodes) + return 1; + + return 0; +} + +/** + * Return true for the set of conditions for which it is OK to use + * a cannibalized circuit. + * + * Don't cannibalize for onehops, or certain purposes. + */ +static int +circuit_should_cannibalize_to_build(uint8_t purpose_to_build, + int has_extend_info, + int onehop_tunnel) +{ + + /* Do not try to cannibalize if this is a one hop circuit. */ + if (onehop_tunnel) { + return 0; + } + + /* Don't try to cannibalize for general purpose circuits that do not + * specify a custom exit. */ + if (purpose_to_build == CIRCUIT_PURPOSE_C_GENERAL && !has_extend_info) { + return 0; + } + + /* Don't cannibalize for testing circuits. We want to see if they + * complete normally. Also don't cannibalize for vanguard-purpose + * circuits, since those are specially pre-built for later + * cannibalization by the actual specific circuit types that need + * vanguards. + */ + if (purpose_to_build == CIRCUIT_PURPOSE_TESTING || + purpose_to_build == CIRCUIT_PURPOSE_HS_VANGUARDS) { + return 0; + } + + /* For vanguards, the server-side intro circ is not cannibalized + * because we pre-build 4 hop HS circuits, and it only needs a 3 hop + * circuit. It is also long-lived, so it is more important that + * it have lower latency than get built fast. + */ + if (circuit_should_use_vanguards(purpose_to_build) && + purpose_to_build == CIRCUIT_PURPOSE_S_ESTABLISH_INTRO) { + return 0; + } + + return 1; +} + +/** Launch a new circuit with purpose purpose and exit node + * extend_info (or NULL to select a random exit node). If flags + * contains CIRCLAUNCH_NEED_UPTIME, choose among routers with high uptime. If + * CIRCLAUNCH_NEED_CAPACITY is set, choose among routers with high bandwidth. + * If CIRCLAUNCH_IS_INTERNAL is true, the last hop need not be an exit node. + * If CIRCLAUNCH_ONEHOP_TUNNEL is set, the circuit will have only one hop. + * Return the newly allocated circuit on success, or NULL on failure. */ +origin_circuit_t * +circuit_launch_by_extend_info(uint8_t purpose, + extend_info_t *extend_info, + int flags) +{ + origin_circuit_t *circ; + int onehop_tunnel = (flags & CIRCLAUNCH_ONEHOP_TUNNEL) != 0; + int have_path = have_enough_path_info(! (flags & CIRCLAUNCH_IS_INTERNAL) ); + + /* Keep some stats about our attempts to launch HS rendezvous circuits */ + if (purpose == CIRCUIT_PURPOSE_S_CONNECT_REND) { + hs_stats_note_service_rendezvous_launch(); + } + + if (!onehop_tunnel && (!router_have_minimum_dir_info() || !have_path)) { + log_debug(LD_CIRC,"Haven't %s yet; canceling " + "circuit launch.", + !router_have_minimum_dir_info() ? + "fetched enough directory info" : + "received a consensus with exits"); + return NULL; + } + + /* If we can/should cannibalize another circuit to build this one, + * then do so. */ + if (circuit_should_cannibalize_to_build(purpose, + extend_info != NULL, + onehop_tunnel)) { + /* see if there are appropriate circs available to cannibalize. */ + /* XXX if we're planning to add a hop, perhaps we want to look for + * internal circs rather than exit circs? -RD */ + circ = circuit_find_to_cannibalize(purpose, extend_info, flags); + if (circ) { + uint8_t old_purpose = circ->base_.purpose; + struct timeval old_timestamp_began = circ->base_.timestamp_began; + + log_info(LD_CIRC, "Cannibalizing circ %u (id: %" PRIu32 ") for " + "purpose %d (%s)", + TO_CIRCUIT(circ)->n_circ_id, circ->global_identifier, purpose, + circuit_purpose_to_string(purpose)); + + if ((purpose == CIRCUIT_PURPOSE_S_CONNECT_REND || + purpose == CIRCUIT_PURPOSE_C_INTRODUCING) && + circ->path_state == PATH_STATE_BUILD_SUCCEEDED) { + /* Path bias: Cannibalized rends pre-emptively count as a + * successfully built but unused closed circuit. We don't + * wait until the extend (or the close) because the rend + * point could be malicious. + * + * Same deal goes for client side introductions. Clients + * can be manipulated to connect repeatedly to them + * (especially web clients). + * + * If we decide to probe the initial portion of these circs, + * (up to the adversary's final hop), we need to remove this, + * or somehow mark the circuit with a special path state. + */ + + /* This must be called before the purpose change */ + pathbias_check_close(circ, END_CIRC_REASON_FINISHED); + } + + circuit_change_purpose(TO_CIRCUIT(circ), purpose); + /* Reset the start date of this circ, else expire_building + * will see it and think it's been trying to build since it + * began. + * + * Technically, the code should reset this when the + * create cell is finally sent, but we're close enough + * here. */ + tor_gettimeofday(&circ->base_.timestamp_began); + + control_event_circuit_cannibalized(circ, old_purpose, + &old_timestamp_began); + + switch (purpose) { + case CIRCUIT_PURPOSE_C_ESTABLISH_REND: + /* it's ready right now */ + break; + case CIRCUIT_PURPOSE_C_INTRODUCING: + case CIRCUIT_PURPOSE_S_CONNECT_REND: + case CIRCUIT_PURPOSE_C_GENERAL: + case CIRCUIT_PURPOSE_S_HSDIR_POST: + case CIRCUIT_PURPOSE_C_HSDIR_GET: + case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO: + /* need to add a new hop */ + tor_assert(extend_info); + if (circuit_extend_to_new_exit(circ, extend_info) < 0) + return NULL; + break; + default: + log_warn(LD_BUG, + "unexpected purpose %d when cannibalizing a circ.", + purpose); + tor_fragile_assert(); + return NULL; + } + return circ; + } + } + + if (did_circs_fail_last_period && + n_circuit_failures > MAX_CIRCUIT_FAILURES) { + /* too many failed circs in a row. don't try. */ +// log_fn(LOG_INFO,"%d failures so far, not trying.",n_circuit_failures); + return NULL; + } + + /* try a circ. if it fails, circuit_mark_for_close will increment + * n_circuit_failures */ + return circuit_establish_circuit(purpose, extend_info, flags); +} + +/** Record another failure at opening a general circuit. When we have + * too many, we'll stop trying for the remainder of this minute. + */ +static void +circuit_increment_failure_count(void) +{ + ++n_circuit_failures; + log_debug(LD_CIRC,"n_circuit_failures now %d.",n_circuit_failures); +} + +/** Reset the failure count for opening general circuits. This means + * we will try MAX_CIRCUIT_FAILURES times more (if necessary) before + * stopping again. + */ +void +circuit_reset_failure_count(int timeout) +{ + if (timeout && n_circuit_failures > MAX_CIRCUIT_FAILURES) + did_circs_fail_last_period = 1; + else + did_circs_fail_last_period = 0; + n_circuit_failures = 0; +} + +/** Find an open circ that we're happy to use for conn and return 1. If + * there isn't one, and there isn't one on the way, launch one and return + * 0. If it will never work, return -1. + * + * Write the found or in-progress or launched circ into *circp. + */ +static int +circuit_get_open_circ_or_launch(entry_connection_t *conn, + uint8_t desired_circuit_purpose, + origin_circuit_t **circp) +{ + origin_circuit_t *circ; + int check_exit_policy; + int need_uptime, need_internal; + int want_onehop; + const or_options_t *options = get_options(); + + tor_assert(conn); + tor_assert(circp); + if (ENTRY_TO_CONN(conn)->state != AP_CONN_STATE_CIRCUIT_WAIT) { + connection_t *c = ENTRY_TO_CONN(conn); + log_err(LD_BUG, "Connection state mismatch: wanted " + "AP_CONN_STATE_CIRCUIT_WAIT, but got %d (%s)", + c->state, conn_state_to_string(c->type, c->state)); + } + tor_assert(ENTRY_TO_CONN(conn)->state == AP_CONN_STATE_CIRCUIT_WAIT); + + /* Will the exit policy of the exit node apply to this stream? */ + check_exit_policy = + conn->socks_request->command == SOCKS_COMMAND_CONNECT && + !conn->use_begindir && + !connection_edge_is_rendezvous_stream(ENTRY_TO_EDGE_CONN(conn)); + + /* Does this connection want a one-hop circuit? */ + want_onehop = conn->want_onehop; + + /* Do we need a high-uptime circuit? */ + need_uptime = !conn->want_onehop && !conn->use_begindir && + smartlist_contains_int_as_string(options->LongLivedPorts, + conn->socks_request->port); + + /* Do we need an "internal" circuit? */ + if (desired_circuit_purpose != CIRCUIT_PURPOSE_C_GENERAL) + need_internal = 1; + else if (conn->use_begindir || conn->want_onehop) + need_internal = 1; + else + need_internal = 0; + + /* We now know what kind of circuit we need. See if there is an + * open circuit that we can use for this stream */ + circ = circuit_get_best(conn, 1 /* Insist on open circuits */, + desired_circuit_purpose, + need_uptime, need_internal); + + if (circ) { + /* We got a circuit that will work for this stream! We can return it. */ + *circp = circ; + return 1; /* we're happy */ + } + + /* Okay, there's no circuit open that will work for this stream. Let's + * see if there's an in-progress circuit or if we have to launch one */ + + /* Do we know enough directory info to build circuits at all? */ + int have_path = have_enough_path_info(!need_internal); + + if (!want_onehop && (!router_have_minimum_dir_info() || !have_path)) { + /* If we don't have enough directory information, we can't build + * multihop circuits. + */ + if (!connection_get_by_type(CONN_TYPE_DIR)) { + int severity = LOG_NOTICE; + /* Retry some stuff that might help the connection work. */ + /* If we are configured with EntryNodes or UseBridges */ + if (entry_list_is_constrained(options)) { + /* Retry all our guards / bridges. + * guards_retry_optimistic() always returns true here. */ + int rv = guards_retry_optimistic(options); + tor_assert_nonfatal_once(rv); + log_fn(severity, LD_APP|LD_DIR, + "Application request when we haven't %s. " + "Optimistically trying known %s again.", + !router_have_minimum_dir_info() ? + "used client functionality lately" : + "received a consensus with exits", + options->UseBridges ? "bridges" : "entrynodes"); + } else { + /* Getting directory documents doesn't help much if we have a limited + * number of guards */ + tor_assert_nonfatal(!options->UseBridges); + tor_assert_nonfatal(!options->EntryNodes); + /* Retry our directory fetches, so we have a fresh set of guard info */ + log_fn(severity, LD_APP|LD_DIR, + "Application request when we haven't %s. " + "Optimistically trying directory fetches again.", + !router_have_minimum_dir_info() ? + "used client functionality lately" : + "received a consensus with exits"); + routerlist_retry_directory_downloads(time(NULL)); + } + } + /* Since we didn't have enough directory info, we can't attach now. The + * stream will be dealt with when router_have_minimum_dir_info becomes 1, + * or when all directory attempts fail and directory_all_unreachable() + * kills it. + */ + return 0; + } + + /* Check whether the exit policy of the chosen exit, or the exit policies + * of _all_ nodes, would forbid this node. */ + if (check_exit_policy) { + if (!conn->chosen_exit_name) { + struct in_addr in; + tor_addr_t addr, *addrp=NULL; + if (tor_inet_aton(conn->socks_request->address, &in)) { + tor_addr_from_in(&addr, &in); + addrp = &addr; + } + if (router_exit_policy_all_nodes_reject(addrp, + conn->socks_request->port, + need_uptime)) { + log_notice(LD_APP, + "No Tor server allows exit to %s:%d. Rejecting.", + safe_str_client(conn->socks_request->address), + conn->socks_request->port); + return -1; + } + } else { + /* XXXX Duplicates checks in connection_ap_handshake_attach_circuit: + * refactor into a single function. */ + const node_t *node = node_get_by_nickname(conn->chosen_exit_name, 0); + int opt = conn->chosen_exit_optional; + if (node && !connection_ap_can_use_exit(conn, node)) { + log_fn(opt ? LOG_INFO : LOG_WARN, LD_APP, + "Requested exit point '%s' is excluded or " + "would refuse request. %s.", + conn->chosen_exit_name, opt ? "Trying others" : "Closing"); + if (opt) { + conn->chosen_exit_optional = 0; + tor_free(conn->chosen_exit_name); + /* Try again. */ + return circuit_get_open_circ_or_launch(conn, + desired_circuit_purpose, + circp); + } + return -1; + } + } + } + + /* Now, check whether there already a circuit on the way that could handle + * this stream. This check matches the one above, but this time we + * do not require that the circuit will work. */ + circ = circuit_get_best(conn, 0 /* don't insist on open circuits */, + desired_circuit_purpose, + need_uptime, need_internal); + if (circ) + log_debug(LD_CIRC, "one on the way!"); + + if (!circ) { + /* No open or in-progress circuit could handle this stream! We + * will have to launch one! + */ + + /* The chosen exit node, if there is one. */ + extend_info_t *extend_info=NULL; + const int n_pending = count_pending_general_client_circuits(); + + /* Do we have too many pending circuits? */ + if (n_pending >= options->MaxClientCircuitsPending) { + static ratelim_t delay_limit = RATELIM_INIT(10*60); + char *m; + if ((m = rate_limit_log(&delay_limit, approx_time()))) { + log_notice(LD_APP, "We'd like to launch a circuit to handle a " + "connection, but we already have %d general-purpose client " + "circuits pending. Waiting until some finish.%s", + n_pending, m); + tor_free(m); + } + return 0; + } + + /* If this is a hidden service trying to start an introduction point, + * handle that case. */ + if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) { + const edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(conn); + /* need to pick an intro point */ + extend_info = hs_client_get_random_intro_from_edge(edge_conn); + if (!extend_info) { + log_info(LD_REND, "No intro points: re-fetching service descriptor."); + if (edge_conn->rend_data) { + rend_client_refetch_v2_renddesc(edge_conn->rend_data); + } else { + hs_client_refetch_hsdesc(&edge_conn->hs_ident->identity_pk); + } + connection_ap_mark_as_waiting_for_renddesc(conn); + return 0; + } + log_info(LD_REND,"Chose %s as intro point for '%s'.", + extend_info_describe(extend_info), + (edge_conn->rend_data) ? + safe_str_client(rend_data_get_address(edge_conn->rend_data)) : + "service"); + } + + /* If we have specified a particular exit node for our + * connection, then be sure to open a circuit to that exit node. + */ + if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_GENERAL || + desired_circuit_purpose == CIRCUIT_PURPOSE_S_HSDIR_POST || + desired_circuit_purpose == CIRCUIT_PURPOSE_C_HSDIR_GET) { + if (conn->chosen_exit_name) { + const node_t *r; + int opt = conn->chosen_exit_optional; + r = node_get_by_nickname(conn->chosen_exit_name, 0); + if (r && node_has_preferred_descriptor(r, conn->want_onehop ? 1 : 0)) { + /* We might want to connect to an IPv6 bridge for loading + descriptors so we use the preferred address rather than + the primary. */ + extend_info = extend_info_from_node(r, conn->want_onehop ? 1 : 0); + if (!extend_info) { + log_warn(LD_CIRC,"Could not make a one-hop connection to %s. " + "Discarding this circuit.", conn->chosen_exit_name); + return -1; + } + } else { /* ! (r && node_has_preferred_descriptor(...)) */ + log_debug(LD_DIR, "considering %d, %s", + want_onehop, conn->chosen_exit_name); + if (want_onehop && conn->chosen_exit_name[0] == '$') { + /* We're asking for a one-hop circuit to a router that + * we don't have a routerinfo about. Make up an extend_info. */ + /* XXX prop220: we need to make chosen_exit_name able to + * encode both key formats. This is not absolutely critical + * since this is just for one-hop circuits, but we should + * still get it done */ + char digest[DIGEST_LEN]; + char *hexdigest = conn->chosen_exit_name+1; + tor_addr_t addr; + if (strlen(hexdigest) < HEX_DIGEST_LEN || + base16_decode(digest,DIGEST_LEN, + hexdigest,HEX_DIGEST_LEN) != DIGEST_LEN) { + log_info(LD_DIR, "Broken exit digest on tunnel conn. Closing."); + return -1; + } + if (tor_addr_parse(&addr, conn->socks_request->address) < 0) { + log_info(LD_DIR, "Broken address %s on tunnel conn. Closing.", + escaped_safe_str_client(conn->socks_request->address)); + return -1; + } + /* XXXX prop220 add a workaround for ed25519 ID below*/ + extend_info = extend_info_new(conn->chosen_exit_name+1, + digest, + NULL, /* Ed25519 ID */ + NULL, NULL, /* onion keys */ + &addr, conn->socks_request->port); + } else { /* ! (want_onehop && conn->chosen_exit_name[0] == '$') */ + /* We will need an onion key for the router, and we + * don't have one. Refuse or relax requirements. */ + log_fn(opt ? LOG_INFO : LOG_WARN, LD_APP, + "Requested exit point '%s' is not known. %s.", + conn->chosen_exit_name, opt ? "Trying others" : "Closing"); + if (opt) { + conn->chosen_exit_optional = 0; + tor_free(conn->chosen_exit_name); + /* Try again with no requested exit */ + return circuit_get_open_circ_or_launch(conn, + desired_circuit_purpose, + circp); + } + return -1; + } + } + } + } /* Done checking for general circutis with chosen exits. */ + + /* What purpose do we need to launch this circuit with? */ + uint8_t new_circ_purpose; + if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_REND_JOINED) + new_circ_purpose = CIRCUIT_PURPOSE_C_ESTABLISH_REND; + else if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) + new_circ_purpose = CIRCUIT_PURPOSE_C_INTRODUCING; + else + new_circ_purpose = desired_circuit_purpose; + + /* Determine what kind of a circuit to launch, and actually launch it. */ + { + int flags = CIRCLAUNCH_NEED_CAPACITY; + if (want_onehop) flags |= CIRCLAUNCH_ONEHOP_TUNNEL; + if (need_uptime) flags |= CIRCLAUNCH_NEED_UPTIME; + if (need_internal) flags |= CIRCLAUNCH_IS_INTERNAL; + + /* If we are about to pick a v3 RP right now, make sure we pick a + * rendezvous point that supports the v3 protocol! */ + if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_REND_JOINED && + new_circ_purpose == CIRCUIT_PURPOSE_C_ESTABLISH_REND && + ENTRY_TO_EDGE_CONN(conn)->hs_ident) { + flags |= CIRCLAUNCH_IS_V3_RP; + log_info(LD_GENERAL, "Getting rendezvous circuit to v3 service!"); + } + + circ = circuit_launch_by_extend_info(new_circ_purpose, extend_info, + flags); + } + + extend_info_free(extend_info); + + /* Now trigger things that need to happen when we launch circuits */ + + if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_GENERAL || + desired_circuit_purpose == CIRCUIT_PURPOSE_C_HSDIR_GET || + desired_circuit_purpose == CIRCUIT_PURPOSE_S_HSDIR_POST) { + /* We just caused a circuit to get built because of this stream. + * If this stream has caused a _lot_ of circuits to be built, that's + * a bad sign: we should tell the user. */ + if (conn->num_circuits_launched < NUM_CIRCUITS_LAUNCHED_THRESHOLD && + ++conn->num_circuits_launched == NUM_CIRCUITS_LAUNCHED_THRESHOLD) + log_info(LD_CIRC, "The application request to %s:%d has launched " + "%d circuits without finding one it likes.", + escaped_safe_str_client(conn->socks_request->address), + conn->socks_request->port, + conn->num_circuits_launched); + } else { + /* help predict this next time */ + rep_hist_note_used_internal(time(NULL), need_uptime, 1); + if (circ) { + const edge_connection_t *edge_conn = ENTRY_TO_EDGE_CONN(conn); + if (edge_conn->rend_data) { + /* write the service_id into circ */ + circ->rend_data = rend_data_dup(edge_conn->rend_data); + } else if (edge_conn->hs_ident) { + circ->hs_ident = + hs_ident_circuit_new(&edge_conn->hs_ident->identity_pk, + HS_IDENT_CIRCUIT_INTRO); + } + if (circ->base_.purpose == CIRCUIT_PURPOSE_C_ESTABLISH_REND && + circ->base_.state == CIRCUIT_STATE_OPEN) + circuit_has_opened(circ); + } + } + } /* endif (!circ) */ + + /* We either found a good circuit, or launched a new circuit, or failed to + * do so. Report success, and delay. */ + + if (circ) { + /* Mark the circuit with the isolation fields for this connection. + * When the circuit arrives, we'll clear these flags: this is + * just some internal bookkeeping to make sure that we have + * launched enough circuits. + */ + connection_edge_update_circuit_isolation(conn, circ, 0); + } else { + log_info(LD_APP, + "No safe circuit (purpose %d) ready for edge " + "connection; delaying.", + desired_circuit_purpose); + } + *circp = circ; + return 0; +} + +/** Return true iff crypt_path is one of the crypt_paths for + * circ. */ +static int +cpath_is_on_circuit(origin_circuit_t *circ, crypt_path_t *crypt_path) +{ + crypt_path_t *cpath, *cpath_next = NULL; + for (cpath = circ->cpath; cpath_next != circ->cpath; cpath = cpath_next) { + cpath_next = cpath->next; + if (crypt_path == cpath) + return 1; + } + return 0; +} + +/** Return true iff client-side optimistic data is supported. */ +static int +optimistic_data_enabled(void) +{ + const or_options_t *options = get_options(); + if (options->OptimisticData < 0) { + /* Note: this default was 0 before #18815 was merged. We can't take the + * parameter out of the consensus until versions before that are all + * obsolete. */ + const int32_t enabled = + networkstatus_get_param(NULL, "UseOptimisticData", /*default*/ 1, 0, 1); + return (int)enabled; + } + return options->OptimisticData; +} + +/** Attach the AP stream apconn to circ's linked list of + * p_streams. Also set apconn's cpath_layer to cpath, or to the last + * hop in circ's cpath if cpath is NULL. + */ +static void +link_apconn_to_circ(entry_connection_t *apconn, origin_circuit_t *circ, + crypt_path_t *cpath) +{ + const node_t *exitnode = NULL; + + /* add it into the linked list of streams on this circuit */ + log_debug(LD_APP|LD_CIRC, "attaching new conn to circ. n_circ_id %u.", + (unsigned)circ->base_.n_circ_id); + /* reset it, so we can measure circ timeouts */ + ENTRY_TO_CONN(apconn)->timestamp_last_read_allowed = time(NULL); + ENTRY_TO_EDGE_CONN(apconn)->next_stream = circ->p_streams; + ENTRY_TO_EDGE_CONN(apconn)->on_circuit = TO_CIRCUIT(circ); + /* assert_connection_ok(conn, time(NULL)); */ + circ->p_streams = ENTRY_TO_EDGE_CONN(apconn); + + if (connection_edge_is_rendezvous_stream(ENTRY_TO_EDGE_CONN(apconn))) { + /* We are attaching a stream to a rendezvous circuit. That means + * that an attempt to connect to a hidden service just + * succeeded. Tell rendclient.c. */ + hs_client_note_connection_attempt_succeeded(ENTRY_TO_EDGE_CONN(apconn)); + } + + if (cpath) { /* we were given one; use it */ + tor_assert(cpath_is_on_circuit(circ, cpath)); + } else { + /* use the last hop in the circuit */ + tor_assert(circ->cpath); + tor_assert(circ->cpath->prev); + tor_assert(circ->cpath->prev->state == CPATH_STATE_OPEN); + cpath = circ->cpath->prev; + } + ENTRY_TO_EDGE_CONN(apconn)->cpath_layer = cpath; + + circ->isolation_any_streams_attached = 1; + connection_edge_update_circuit_isolation(apconn, circ, 0); + + /* Compute the exitnode if possible, for logging below */ + if (cpath->extend_info) + exitnode = node_get_by_id(cpath->extend_info->identity_digest); + + /* See if we can use optimistic data on this circuit */ + if (optimistic_data_enabled() && + (circ->base_.purpose == CIRCUIT_PURPOSE_C_GENERAL || + circ->base_.purpose == CIRCUIT_PURPOSE_C_HSDIR_GET || + circ->base_.purpose == CIRCUIT_PURPOSE_S_HSDIR_POST || + circ->base_.purpose == CIRCUIT_PURPOSE_C_REND_JOINED)) + apconn->may_use_optimistic_data = 1; + else + apconn->may_use_optimistic_data = 0; + log_info(LD_APP, "Looks like completed circuit to %s %s allow " + "optimistic data for connection to %s", + circ->base_.purpose == CIRCUIT_PURPOSE_C_GENERAL ? + /* node_describe() does the right thing if exitnode is NULL */ + safe_str_client(node_describe(exitnode)) : + "hidden service", + apconn->may_use_optimistic_data ? "does" : "doesn't", + safe_str_client(apconn->socks_request->address)); +} + +/** Return true iff address is matched by one of the entries in + * TrackHostExits. */ +int +hostname_in_track_host_exits(const or_options_t *options, const char *address) +{ + if (!options->TrackHostExits) + return 0; + SMARTLIST_FOREACH_BEGIN(options->TrackHostExits, const char *, cp) { + if (cp[0] == '.') { /* match end */ + if (cp[1] == '\0' || + !strcasecmpend(address, cp) || + !strcasecmp(address, &cp[1])) + return 1; + } else if (strcasecmp(cp, address) == 0) { + return 1; + } + } SMARTLIST_FOREACH_END(cp); + return 0; +} + +/** If an exit wasn't explicitly specified for conn, consider saving + * the exit that we *did* choose for use by future connections to + * conn's destination. + */ +static void +consider_recording_trackhost(const entry_connection_t *conn, + const origin_circuit_t *circ) +{ + const or_options_t *options = get_options(); + char *new_address = NULL; + char fp[HEX_DIGEST_LEN+1]; + + /* Search the addressmap for this conn's destination. */ + /* If they're not in the address map.. */ + if (!options->TrackHostExits || + addressmap_have_mapping(conn->socks_request->address, + options->TrackHostExitsExpire)) + return; /* nothing to track, or already mapped */ + + if (!hostname_in_track_host_exits(options, conn->socks_request->address) || + !circ->build_state->chosen_exit) + return; + + /* write down the fingerprint of the chosen exit, not the nickname, + * because the chosen exit might not be named. */ + base16_encode(fp, sizeof(fp), + circ->build_state->chosen_exit->identity_digest, DIGEST_LEN); + + /* Add this exit/hostname pair to the addressmap. */ + tor_asprintf(&new_address, "%s.%s.exit", + conn->socks_request->address, fp); + + addressmap_register(conn->socks_request->address, new_address, + time(NULL) + options->TrackHostExitsExpire, + ADDRMAPSRC_TRACKEXIT, 0, 0); +} + +/** Attempt to attach the connection conn to circ, and send a + * begin or resolve cell as appropriate. Return values are as for + * connection_ap_handshake_attach_circuit. The stream will exit from the hop + * indicated by cpath, or from the last hop in circ's cpath if + * cpath is NULL. */ +int +connection_ap_handshake_attach_chosen_circuit(entry_connection_t *conn, + origin_circuit_t *circ, + crypt_path_t *cpath) +{ + connection_t *base_conn = ENTRY_TO_CONN(conn); + tor_assert(conn); + tor_assert(base_conn->state == AP_CONN_STATE_CIRCUIT_WAIT || + base_conn->state == AP_CONN_STATE_CONTROLLER_WAIT); + tor_assert(conn->socks_request); + tor_assert(circ); + tor_assert(circ->base_.state == CIRCUIT_STATE_OPEN); + + base_conn->state = AP_CONN_STATE_CIRCUIT_WAIT; + + if (!circ->base_.timestamp_dirty || + ((conn->entry_cfg.isolation_flags & ISO_SOCKSAUTH) && + (conn->entry_cfg.socks_iso_keep_alive) && + (conn->socks_request->usernamelen || + conn->socks_request->passwordlen))) { + /* When stream isolation is in use and controlled by an application + * we are willing to keep using the stream. */ + circ->base_.timestamp_dirty = approx_time(); + } + + pathbias_count_use_attempt(circ); + + /* Now, actually link the connection. */ + link_apconn_to_circ(conn, circ, cpath); + + tor_assert(conn->socks_request); + if (conn->socks_request->command == SOCKS_COMMAND_CONNECT) { + if (!conn->use_begindir) + consider_recording_trackhost(conn, circ); + if (connection_ap_handshake_send_begin(conn) < 0) + return -1; + } else { + if (connection_ap_handshake_send_resolve(conn) < 0) + return -1; + } + + return 1; +} + +/** + * Return an appropriate circuit purpose for non-rend streams. + * We don't handle rends here because a rend stream triggers two + * circuit builds with different purposes, so it is handled elsewhere. + * + * This function just figures out what type of hsdir activity this is, + * and tells us. Everything else is general. + */ +static int +connection_ap_get_nonrend_circ_purpose(const entry_connection_t *conn) +{ + const connection_t *base_conn = ENTRY_TO_CONN(conn); + tor_assert_nonfatal(!connection_edge_is_rendezvous_stream( + ENTRY_TO_EDGE_CONN(conn))); + + if (base_conn->linked_conn && + base_conn->linked_conn->type == CONN_TYPE_DIR) { + /* Set a custom purpose for hsdir activity */ + if (base_conn->linked_conn->purpose == DIR_PURPOSE_UPLOAD_RENDDESC_V2 || + base_conn->linked_conn->purpose == DIR_PURPOSE_UPLOAD_HSDESC) { + return CIRCUIT_PURPOSE_S_HSDIR_POST; + } else if (base_conn->linked_conn->purpose + == DIR_PURPOSE_FETCH_RENDDESC_V2 || + base_conn->linked_conn->purpose + == DIR_PURPOSE_FETCH_HSDESC) { + return CIRCUIT_PURPOSE_C_HSDIR_GET; + } + } + + /* All other purposes are general for now */ + return CIRCUIT_PURPOSE_C_GENERAL; +} + +/** Try to find a safe live circuit for stream conn. If we find one, + * attach the stream, send appropriate cells, and return 1. Otherwise, + * try to launch new circuit(s) for the stream. If we can launch + * circuits, return 0. Otherwise, if we simply can't proceed with + * this stream, return -1. (conn needs to die, and is maybe already marked). + */ +/* XXXX this function should mark for close whenever it returns -1; + * its callers shouldn't have to worry about that. */ +int +connection_ap_handshake_attach_circuit(entry_connection_t *conn) +{ + connection_t *base_conn = ENTRY_TO_CONN(conn); + int retval; + int conn_age; + int want_onehop; + + tor_assert(conn); + tor_assert(base_conn->state == AP_CONN_STATE_CIRCUIT_WAIT); + tor_assert(conn->socks_request); + want_onehop = conn->want_onehop; + + conn_age = (int)(time(NULL) - base_conn->timestamp_created); + + /* Is this connection so old that we should give up on it? */ + if (conn_age >= get_options()->SocksTimeout) { + int severity = (tor_addr_is_null(&base_conn->addr) && !base_conn->port) ? + LOG_INFO : LOG_NOTICE; + log_fn(severity, LD_APP, + "Tried for %d seconds to get a connection to %s:%d. Giving up.", + conn_age, safe_str_client(conn->socks_request->address), + conn->socks_request->port); + return -1; + } + + /* We handle "general" (non-onion) connections much more straightforwardly. + */ + if (!connection_edge_is_rendezvous_stream(ENTRY_TO_EDGE_CONN(conn))) { + /* we're a general conn */ + origin_circuit_t *circ=NULL; + + /* Are we linked to a dir conn that aims to fetch a consensus? + * We check here because the conn might no longer be needed. */ + if (base_conn->linked_conn && + base_conn->linked_conn->type == CONN_TYPE_DIR && + base_conn->linked_conn->purpose == DIR_PURPOSE_FETCH_CONSENSUS) { + + /* Yes we are. Is there a consensus fetch farther along than us? */ + if (networkstatus_consensus_is_already_downloading( + TO_DIR_CONN(base_conn->linked_conn)->requested_resource)) { + /* We're doing the "multiple consensus fetch attempts" game from + * proposal 210, and we're late to the party. Just close this conn. + * The circuit and TLS conn that we made will time out after a while + * if nothing else wants to use them. */ + log_info(LD_DIR, "Closing extra consensus fetch (to %s) since one " + "is already downloading.", base_conn->linked_conn->address); + return -1; + } + } + + /* If we have a chosen exit, we need to use a circuit that's + * open to that exit. See what exit we meant, and whether we can use it. + */ + if (conn->chosen_exit_name) { + const node_t *node = node_get_by_nickname(conn->chosen_exit_name, 0); + int opt = conn->chosen_exit_optional; + if (!node && !want_onehop) { + /* We ran into this warning when trying to extend a circuit to a + * hidden service directory for which we didn't have a router + * descriptor. See flyspray task 767 for more details. We should + * keep this in mind when deciding to use BEGIN_DIR cells for other + * directory requests as well. -KL*/ + log_fn(opt ? LOG_INFO : LOG_WARN, LD_APP, + "Requested exit point '%s' is not known. %s.", + conn->chosen_exit_name, opt ? "Trying others" : "Closing"); + if (opt) { + /* If we are allowed to ignore the .exit request, do so */ + conn->chosen_exit_optional = 0; + tor_free(conn->chosen_exit_name); + return 0; + } + return -1; + } + if (node && !connection_ap_can_use_exit(conn, node)) { + log_fn(opt ? LOG_INFO : LOG_WARN, LD_APP, + "Requested exit point '%s' is excluded or " + "would refuse request. %s.", + conn->chosen_exit_name, opt ? "Trying others" : "Closing"); + if (opt) { + /* If we are allowed to ignore the .exit request, do so */ + conn->chosen_exit_optional = 0; + tor_free(conn->chosen_exit_name); + return 0; + } + return -1; + } + } + + /* Find the circuit that we should use, if there is one. Otherwise + * launch it + */ + retval = circuit_get_open_circ_or_launch(conn, + connection_ap_get_nonrend_circ_purpose(conn), + &circ); + + if (retval < 1) { + /* We were either told "-1" (complete failure) or 0 (circuit in + * progress); we can't attach this stream yet. */ + return retval; + } + + log_debug(LD_APP|LD_CIRC, + "Attaching apconn to circ %u (stream %d sec old).", + (unsigned)circ->base_.n_circ_id, conn_age); + /* print the circ's path, so clients can figure out which circs are + * sucking. */ + circuit_log_path(LOG_INFO,LD_APP|LD_CIRC,circ); + + /* We have found a suitable circuit for our conn. Hurray. Do + * the attachment. */ + return connection_ap_handshake_attach_chosen_circuit(conn, circ, NULL); + + } else { /* we're a rendezvous conn */ + origin_circuit_t *rendcirc=NULL, *introcirc=NULL; + + tor_assert(!ENTRY_TO_EDGE_CONN(conn)->cpath_layer); + + /* start by finding a rendezvous circuit for us */ + + retval = circuit_get_open_circ_or_launch( + conn, CIRCUIT_PURPOSE_C_REND_JOINED, &rendcirc); + if (retval < 0) return -1; /* failed */ + + if (retval > 0) { + tor_assert(rendcirc); + /* one is already established, attach */ + log_info(LD_REND, + "rend joined circ %u (id: %" PRIu32 ") already here. " + "Attaching. (stream %d sec old)", + (unsigned) TO_CIRCUIT(rendcirc)->n_circ_id, + rendcirc->global_identifier, conn_age); + /* Mark rendezvous circuits as 'newly dirty' every time you use + * them, since the process of rebuilding a rendezvous circ is so + * expensive. There is a tradeoff between linkability and + * feasibility, at this point. + */ + rendcirc->base_.timestamp_dirty = time(NULL); + + /* We've also attempted to use them. If they fail, we need to + * probe them for path bias */ + pathbias_count_use_attempt(rendcirc); + + link_apconn_to_circ(conn, rendcirc, NULL); + if (connection_ap_handshake_send_begin(conn) < 0) + return 0; /* already marked, let them fade away */ + return 1; + } + + /* At this point we need to re-check the state, since it's possible that + * our call to circuit_get_open_circ_or_launch() changed the connection's + * state from "CIRCUIT_WAIT" to "RENDDESC_WAIT" because we decided to + * re-fetch the descriptor. + */ + if (ENTRY_TO_CONN(conn)->state != AP_CONN_STATE_CIRCUIT_WAIT) { + log_info(LD_REND, "This connection is no longer ready to attach; its " + "state changed." + "(We probably have to re-fetch its descriptor.)"); + return 0; + } + + if (rendcirc && (rendcirc->base_.purpose == + CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED)) { + log_info(LD_REND, + "pending-join circ %u (id: %" PRIu32 ") already here, with " + "intro ack. Stalling. (stream %d sec old)", + (unsigned) TO_CIRCUIT(rendcirc)->n_circ_id, + rendcirc->global_identifier, conn_age); + return 0; + } + + /* it's on its way. find an intro circ. */ + retval = circuit_get_open_circ_or_launch( + conn, CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT, &introcirc); + if (retval < 0) return -1; /* failed */ + + if (retval > 0) { + /* one has already sent the intro. keep waiting. */ + tor_assert(introcirc); + log_info(LD_REND, "Intro circ %u (id: %" PRIu32 ") present and " + "awaiting ACK. Rend circuit %u (id: %" PRIu32 "). " + "Stalling. (stream %d sec old)", + (unsigned) TO_CIRCUIT(introcirc)->n_circ_id, + introcirc->global_identifier, + rendcirc ? (unsigned) TO_CIRCUIT(rendcirc)->n_circ_id : 0, + rendcirc ? rendcirc->global_identifier : 0, + conn_age); + return 0; + } + + /* now rendcirc and introcirc are each either undefined or not finished */ + + if (rendcirc && introcirc && + rendcirc->base_.purpose == CIRCUIT_PURPOSE_C_REND_READY) { + log_info(LD_REND, + "ready rend circ %u (id: %" PRIu32 ") already here. No" + "intro-ack yet on intro %u (id: %" PRIu32 "). " + "(stream %d sec old)", + (unsigned) TO_CIRCUIT(rendcirc)->n_circ_id, + rendcirc->global_identifier, + (unsigned) TO_CIRCUIT(introcirc)->n_circ_id, + introcirc->global_identifier, conn_age); + + tor_assert(introcirc->base_.purpose == CIRCUIT_PURPOSE_C_INTRODUCING); + if (introcirc->base_.state == CIRCUIT_STATE_OPEN) { + int ret; + log_info(LD_REND, "Found open intro circ %u (id: %" PRIu32 "). " + "Rend circuit %u (id: %" PRIu32 "); Sending " + "introduction. (stream %d sec old)", + (unsigned) TO_CIRCUIT(introcirc)->n_circ_id, + introcirc->global_identifier, + (unsigned) TO_CIRCUIT(rendcirc)->n_circ_id, + rendcirc->global_identifier, conn_age); + ret = hs_client_send_introduce1(introcirc, rendcirc); + switch (ret) { + case 0: /* success */ + rendcirc->base_.timestamp_dirty = time(NULL); + introcirc->base_.timestamp_dirty = time(NULL); + + pathbias_count_use_attempt(introcirc); + pathbias_count_use_attempt(rendcirc); + + assert_circuit_ok(TO_CIRCUIT(rendcirc)); + assert_circuit_ok(TO_CIRCUIT(introcirc)); + return 0; + case -1: /* transient error */ + return 0; + case -2: /* permanent error */ + return -1; + default: /* oops */ + tor_fragile_assert(); + return -1; + } + } + } + + log_info(LD_REND, "Intro %u (id: %" PRIu32 ") and rend circuit %u " + "(id: %" PRIu32 ") circuits are not both ready. " + "Stalling conn. (%d sec old)", + introcirc ? (unsigned) TO_CIRCUIT(introcirc)->n_circ_id : 0, + introcirc ? introcirc->global_identifier : 0, + rendcirc ? (unsigned) TO_CIRCUIT(rendcirc)->n_circ_id : 0, + rendcirc ? rendcirc->global_identifier : 0, conn_age); + return 0; + } +} + +/** Change circ's purpose to new_purpose. */ +void +circuit_change_purpose(circuit_t *circ, uint8_t new_purpose) +{ + uint8_t old_purpose; + /* Don't allow an OR circ to become an origin circ or vice versa. */ + tor_assert(!!(CIRCUIT_IS_ORIGIN(circ)) == + !!(CIRCUIT_PURPOSE_IS_ORIGIN(new_purpose))); + + if (circ->purpose == new_purpose) return; + + if (CIRCUIT_IS_ORIGIN(circ)) { + char old_purpose_desc[80] = ""; + + strncpy(old_purpose_desc, circuit_purpose_to_string(circ->purpose), 80-1); + old_purpose_desc[80-1] = '\0'; + + log_debug(LD_CIRC, + "changing purpose of origin circ %d " + "from \"%s\" (%d) to \"%s\" (%d)", + TO_ORIGIN_CIRCUIT(circ)->global_identifier, + old_purpose_desc, + circ->purpose, + circuit_purpose_to_string(new_purpose), + new_purpose); + } + + old_purpose = circ->purpose; + circ->purpose = new_purpose; + + if (CIRCUIT_IS_ORIGIN(circ)) { + control_event_circuit_purpose_changed(TO_ORIGIN_CIRCUIT(circ), + old_purpose); + } +} + +/** Mark circ so that no more connections can be attached to it. */ +void +mark_circuit_unusable_for_new_conns(origin_circuit_t *circ) +{ + const or_options_t *options = get_options(); + tor_assert(circ); + + /* XXXX This is a kludge; we're only keeping it around in case there's + * something that doesn't check unusable_for_new_conns, and to avoid + * deeper refactoring of our expiration logic. */ + if (! circ->base_.timestamp_dirty) + circ->base_.timestamp_dirty = approx_time(); + if (options->MaxCircuitDirtiness >= circ->base_.timestamp_dirty) + circ->base_.timestamp_dirty = 1; /* prevent underflow */ + else + circ->base_.timestamp_dirty -= options->MaxCircuitDirtiness; + + circ->unusable_for_new_conns = 1; +} + +/** + * Add relay_body_len and RELAY_PAYLOAD_SIZE-relay_body_len to + * the valid delivered written fields and the overhead field, + * respectively. + */ +void +circuit_sent_valid_data(origin_circuit_t *circ, uint16_t relay_body_len) +{ + if (!circ) return; + + tor_assert_nonfatal(relay_body_len <= RELAY_PAYLOAD_SIZE); + + circ->n_delivered_written_circ_bw = + tor_add_u32_nowrap(circ->n_delivered_written_circ_bw, relay_body_len); + circ->n_overhead_written_circ_bw = + tor_add_u32_nowrap(circ->n_overhead_written_circ_bw, + RELAY_PAYLOAD_SIZE-relay_body_len); +} + +/** + * Add relay_body_len and RELAY_PAYLOAD_SIZE-relay_body_len to + * the valid delivered read field and the overhead field, + * respectively. + */ +void +circuit_read_valid_data(origin_circuit_t *circ, uint16_t relay_body_len) +{ + if (!circ) return; + + tor_assert_nonfatal(relay_body_len <= RELAY_PAYLOAD_SIZE); + + circ->n_delivered_read_circ_bw = + tor_add_u32_nowrap(circ->n_delivered_read_circ_bw, relay_body_len); + circ->n_overhead_read_circ_bw = + tor_add_u32_nowrap(circ->n_overhead_read_circ_bw, + RELAY_PAYLOAD_SIZE-relay_body_len); +} diff --cc src/feature/relay/router.c index b376046c8,000000000..dad2c6a50 mode 100644,000000..100644 --- a/src/feature/relay/router.c +++ b/src/feature/relay/router.c @@@ -1,3129 -1,0 +1,3128 @@@ +/* Copyright (c) 2001 Matej Pfajfar. + * Copyright (c) 2001-2004, Roger Dingledine. + * Copyright (c) 2004-2006, Roger Dingledine, Nick Mathewson. + * Copyright (c) 2007-2019, The Tor Project, Inc. */ +/* See LICENSE for licensing information */ + +#define ROUTER_PRIVATE + +#include "core/or/or.h" +#include "app/config/config.h" +#include "app/config/statefile.h" +#include "app/main/main.h" +#include "core/mainloop/connection.h" +#include "core/mainloop/mainloop.h" +#include "core/mainloop/netstatus.h" +#include "core/or/policies.h" +#include "core/or/protover.h" +#include "feature/client/transports.h" +#include "feature/control/control.h" +#include "feature/dirauth/process_descs.h" +#include "feature/dircache/dirserv.h" +#include "feature/dirclient/dirclient.h" +#include "feature/dircommon/directory.h" +#include "feature/dirparse/authcert_parse.h" +#include "feature/dirparse/routerparse.h" +#include "feature/dirparse/signing.h" +#include "feature/hibernate/hibernate.h" +#include "feature/keymgt/loadkey.h" +#include "feature/nodelist/authcert.h" +#include "feature/nodelist/dirlist.h" +#include "feature/nodelist/networkstatus.h" +#include "feature/nodelist/nickname.h" +#include "feature/nodelist/nodelist.h" +#include "feature/nodelist/routerlist.h" +#include "feature/nodelist/torcert.h" +#include "feature/relay/dns.h" +#include "feature/relay/router.h" +#include "feature/relay/routerkeys.h" +#include "feature/relay/routermode.h" +#include "feature/relay/selftest.h" +#include "lib/geoip/geoip.h" +#include "feature/stats/geoip_stats.h" +#include "feature/stats/rephist.h" +#include "lib/crypt_ops/crypto_ed25519.h" +#include "lib/crypt_ops/crypto_format.h" +#include "lib/crypt_ops/crypto_init.h" +#include "lib/crypt_ops/crypto_rand.h" +#include "lib/crypt_ops/crypto_util.h" +#include "lib/encoding/confline.h" +#include "lib/osinfo/uname.h" +#include "lib/tls/tortls.h" + +#include "feature/dirauth/authmode.h" + +#include "app/config/or_state_st.h" +#include "core/or/port_cfg_st.h" +#include "feature/dirclient/dir_server_st.h" +#include "feature/dircommon/dir_connection_st.h" +#include "feature/nodelist/authority_cert_st.h" +#include "feature/nodelist/extrainfo_st.h" +#include "feature/nodelist/node_st.h" +#include "feature/nodelist/routerinfo_st.h" +#include "feature/nodelist/routerstatus_st.h" + +/** + * \file router.c + * \brief Miscellaneous relay functionality, including RSA key maintenance, + * generating and uploading server descriptors, picking an address to + * advertise, and so on. + * + * This module handles the job of deciding whether we are a Tor relay, and if + * so what kind. (Mostly through functions like server_mode() that inspect an + * or_options_t, but in some cases based on our own capabilities, such as when + * we are deciding whether to be a directory cache in + * router_has_bandwidth_to_be_dirserver().) + * + * Also in this module are the functions to generate our own routerinfo_t and + * extrainfo_t, and to encode those to signed strings for upload to the + * directory authorities. + * + * This module also handles key maintenance for RSA and Curve25519-ntor keys, + * and for our TLS context. (These functions should eventually move to + * routerkeys.c along with the code that handles Ed25519 keys now.) + **/ + +/************************************************************/ + +/***** + * Key management: ORs only. + *****/ + +/** Private keys for this OR. There is also an SSL key managed by tortls.c. + */ +static tor_mutex_t *key_lock=NULL; +static time_t onionkey_set_at=0; /**< When was onionkey last changed? */ +/** Current private onionskin decryption key: used to decode CREATE cells. */ +static crypto_pk_t *onionkey=NULL; +/** Previous private onionskin decryption key: used to decode CREATE cells + * generated by clients that have an older version of our descriptor. */ +static crypto_pk_t *lastonionkey=NULL; +/** Current private ntor secret key: used to perform the ntor handshake. */ +static curve25519_keypair_t curve25519_onion_key; +/** Previous private ntor secret key: used to perform the ntor handshake + * with clients that have an older version of our descriptor. */ +static curve25519_keypair_t last_curve25519_onion_key; +/** Private server "identity key": used to sign directory info and TLS + * certificates. Never changes. */ +static crypto_pk_t *server_identitykey=NULL; +/** Digest of server_identitykey. */ +static char server_identitykey_digest[DIGEST_LEN]; +/** Private client "identity key": used to sign bridges' and clients' + * outbound TLS certificates. Regenerated on startup and on IP address + * change. */ +static crypto_pk_t *client_identitykey=NULL; +/** Signing key used for v3 directory material; only set for authorities. */ +static crypto_pk_t *authority_signing_key = NULL; +/** Key certificate to authenticate v3 directory material; only set for + * authorities. */ +static authority_cert_t *authority_key_certificate = NULL; + +/** For emergency V3 authority key migration: An extra signing key that we use + * with our old (obsolete) identity key for a while. */ +static crypto_pk_t *legacy_signing_key = NULL; +/** For emergency V3 authority key migration: An extra certificate to + * authenticate legacy_signing_key with our obsolete identity key.*/ +static authority_cert_t *legacy_key_certificate = NULL; + +/* (Note that v3 authorities also have a separate "authority identity key", + * but this key is never actually loaded by the Tor process. Instead, it's + * used by tor-gencert to sign new signing keys and make new key + * certificates. */ + +/** Return a readonly string with human readable description + * of err. + */ +const char * +routerinfo_err_to_string(int err) +{ + switch (err) { + case TOR_ROUTERINFO_ERROR_NO_EXT_ADDR: + return "No known exit address yet"; + case TOR_ROUTERINFO_ERROR_CANNOT_PARSE: + return "Cannot parse descriptor"; + case TOR_ROUTERINFO_ERROR_NOT_A_SERVER: + return "Not running in server mode"; + case TOR_ROUTERINFO_ERROR_DIGEST_FAILED: + return "Key digest failed"; + case TOR_ROUTERINFO_ERROR_CANNOT_GENERATE: + return "Cannot generate descriptor"; + case TOR_ROUTERINFO_ERROR_DESC_REBUILDING: + return "Descriptor still rebuilding - not ready yet"; + } + + log_warn(LD_BUG, "unknown routerinfo error %d - shouldn't happen", err); + tor_assert_unreached(); + + return "Unknown error"; +} + +/** Return true if we expect given error to be transient. + * Return false otherwise. + */ +int +routerinfo_err_is_transient(int err) +{ + /** + * For simplicity, we consider all errors other than + * "not a server" transient - see discussion on + * https://trac.torproject.org/projects/tor/ticket/27034 + */ + return err != TOR_ROUTERINFO_ERROR_NOT_A_SERVER; +} + +/** Replace the current onion key with k. Does not affect + * lastonionkey; to update lastonionkey correctly, call rotate_onion_key(). + */ +static void +set_onion_key(crypto_pk_t *k) +{ + if (onionkey && crypto_pk_eq_keys(onionkey, k)) { + /* k is already our onion key; free it and return */ + crypto_pk_free(k); + return; + } + tor_mutex_acquire(key_lock); + crypto_pk_free(onionkey); + onionkey = k; + tor_mutex_release(key_lock); + mark_my_descriptor_dirty("set onion key"); +} + +/** Return the current onion key. Requires that the onion key has been + * loaded or generated. */ +crypto_pk_t * +get_onion_key(void) +{ + tor_assert(onionkey); + return onionkey; +} + +/** Store a full copy of the current onion key into *key, and a full + * copy of the most recent onion key into *last. Store NULL into + * a pointer if the corresponding key does not exist. + */ +void +dup_onion_keys(crypto_pk_t **key, crypto_pk_t **last) +{ + tor_assert(key); + tor_assert(last); + tor_mutex_acquire(key_lock); + if (onionkey) + *key = crypto_pk_copy_full(onionkey); + else + *key = NULL; + if (lastonionkey) + *last = crypto_pk_copy_full(lastonionkey); + else + *last = NULL; + tor_mutex_release(key_lock); +} + +/** Expire our old set of onion keys. This is done by setting + * last_curve25519_onion_key and lastonionkey to all zero's and NULL + * respectively. + * + * This function does not perform any grace period checks for the old onion + * keys. + */ +void +expire_old_onion_keys(void) +{ + char *fname = NULL; + + tor_mutex_acquire(key_lock); + + /* Free lastonionkey and set it to NULL. */ + if (lastonionkey) { + crypto_pk_free(lastonionkey); + lastonionkey = NULL; + } + + /* We zero out the keypair. See the tor_mem_is_zero() check made in + * construct_ntor_key_map() below. */ + memset(&last_curve25519_onion_key, 0, sizeof(last_curve25519_onion_key)); + + tor_mutex_release(key_lock); + + fname = get_keydir_fname("secret_onion_key.old"); + if (file_status(fname) == FN_FILE) { + if (tor_unlink(fname) != 0) { + log_warn(LD_FS, "Couldn't unlink old onion key file %s: %s", + fname, strerror(errno)); + } + } + tor_free(fname); + + fname = get_keydir_fname("secret_onion_key_ntor.old"); + if (file_status(fname) == FN_FILE) { + if (tor_unlink(fname) != 0) { + log_warn(LD_FS, "Couldn't unlink old ntor onion key file %s: %s", + fname, strerror(errno)); + } + } + tor_free(fname); +} + +/** Return the current secret onion key for the ntor handshake. Must only + * be called from the main thread. */ +static const curve25519_keypair_t * +get_current_curve25519_keypair(void) +{ + return &curve25519_onion_key; +} +/** Return a map from KEYID (the key itself) to keypairs for use in the ntor + * handshake. Must only be called from the main thread. */ +di_digest256_map_t * +construct_ntor_key_map(void) +{ + di_digest256_map_t *m = NULL; + + if (!tor_mem_is_zero((const char*) + curve25519_onion_key.pubkey.public_key, + CURVE25519_PUBKEY_LEN)) { + dimap_add_entry(&m, + curve25519_onion_key.pubkey.public_key, + tor_memdup(&curve25519_onion_key, + sizeof(curve25519_keypair_t))); + } + if (!tor_mem_is_zero((const char*) + last_curve25519_onion_key.pubkey.public_key, + CURVE25519_PUBKEY_LEN)) { + dimap_add_entry(&m, + last_curve25519_onion_key.pubkey.public_key, + tor_memdup(&last_curve25519_onion_key, + sizeof(curve25519_keypair_t))); + } + + return m; +} +/** Helper used to deallocate a di_digest256_map_t returned by + * construct_ntor_key_map. */ +static void +ntor_key_map_free_helper(void *arg) +{ + curve25519_keypair_t *k = arg; + memwipe(k, 0, sizeof(*k)); + tor_free(k); +} +/** Release all storage from a keymap returned by construct_ntor_key_map. */ +void +ntor_key_map_free_(di_digest256_map_t *map) +{ + if (!map) + return; + dimap_free(map, ntor_key_map_free_helper); +} + +/** Return the time when the onion key was last set. This is either the time + * when the process launched, or the time of the most recent key rotation since + * the process launched. + */ +time_t +get_onion_key_set_at(void) +{ + return onionkey_set_at; +} + +/** Set the current server identity key to k. + */ +void +set_server_identity_key(crypto_pk_t *k) +{ + crypto_pk_free(server_identitykey); + server_identitykey = k; + if (crypto_pk_get_digest(server_identitykey, + server_identitykey_digest) < 0) { + log_err(LD_BUG, "Couldn't compute our own identity key digest."); + tor_assert(0); + } +} + +/** Make sure that we have set up our identity keys to match or not match as + * appropriate, and die with an assertion if we have not. */ +static void +assert_identity_keys_ok(void) +{ + if (1) + return; + tor_assert(client_identitykey); + if (public_server_mode(get_options())) { + /* assert that we have set the client and server keys to be equal */ + tor_assert(server_identitykey); + tor_assert(crypto_pk_eq_keys(client_identitykey, server_identitykey)); + } else { + /* assert that we have set the client and server keys to be unequal */ + if (server_identitykey) + tor_assert(!crypto_pk_eq_keys(client_identitykey, server_identitykey)); + } +} + +/** Returns the current server identity key; requires that the key has + * been set, and that we are running as a Tor server. + */ +crypto_pk_t * +get_server_identity_key(void) +{ + tor_assert(server_identitykey); + tor_assert(server_mode(get_options())); + assert_identity_keys_ok(); + return server_identitykey; +} + +/** Return true iff we are a server and the server identity key + * has been set. */ +int +server_identity_key_is_set(void) +{ + return server_mode(get_options()) && server_identitykey != NULL; +} + +/** Set the current client identity key to k. + */ +void +set_client_identity_key(crypto_pk_t *k) +{ + crypto_pk_free(client_identitykey); + client_identitykey = k; +} + +/** Returns the current client identity key for use on outgoing TLS + * connections; requires that the key has been set. + */ +crypto_pk_t * +get_tlsclient_identity_key(void) +{ + tor_assert(client_identitykey); + assert_identity_keys_ok(); + return client_identitykey; +} + +/** Return true iff the client identity key has been set. */ +int +client_identity_key_is_set(void) +{ + return client_identitykey != NULL; +} + +/** Return the key certificate for this v3 (voting) authority, or NULL + * if we have no such certificate. */ +MOCK_IMPL(authority_cert_t *, +get_my_v3_authority_cert, (void)) +{ + return authority_key_certificate; +} + +/** Return the v3 signing key for this v3 (voting) authority, or NULL + * if we have no such key. */ +crypto_pk_t * +get_my_v3_authority_signing_key(void) +{ + return authority_signing_key; +} + +/** If we're an authority, and we're using a legacy authority identity key for + * emergency migration purposes, return the certificate associated with that + * key. */ +authority_cert_t * +get_my_v3_legacy_cert(void) +{ + return legacy_key_certificate; +} + +/** If we're an authority, and we're using a legacy authority identity key for + * emergency migration purposes, return that key. */ +crypto_pk_t * +get_my_v3_legacy_signing_key(void) +{ + return legacy_signing_key; +} + +/** Replace the previous onion key with the current onion key, and generate + * a new previous onion key. Immediately after calling this function, + * the OR should: + * - schedule all previous cpuworkers to shut down _after_ processing + * pending work. (This will cause fresh cpuworkers to be generated.) + * - generate and upload a fresh routerinfo. + */ +void +rotate_onion_key(void) +{ + char *fname, *fname_prev; + crypto_pk_t *prkey = NULL; + or_state_t *state = get_or_state(); + curve25519_keypair_t new_curve25519_keypair; + time_t now; + fname = get_keydir_fname("secret_onion_key"); + fname_prev = get_keydir_fname("secret_onion_key.old"); + /* There isn't much point replacing an old key with an empty file */ + if (file_status(fname) == FN_FILE) { + if (replace_file(fname, fname_prev)) + goto error; + } + if (!(prkey = crypto_pk_new())) { + log_err(LD_GENERAL,"Error constructing rotated onion key"); + goto error; + } + if (crypto_pk_generate_key(prkey)) { + log_err(LD_BUG,"Error generating onion key"); + goto error; + } + if (crypto_pk_write_private_key_to_filename(prkey, fname)) { + log_err(LD_FS,"Couldn't write generated onion key to \"%s\".", fname); + goto error; + } + tor_free(fname); + tor_free(fname_prev); + fname = get_keydir_fname("secret_onion_key_ntor"); + fname_prev = get_keydir_fname("secret_onion_key_ntor.old"); + if (curve25519_keypair_generate(&new_curve25519_keypair, 1) < 0) + goto error; + /* There isn't much point replacing an old key with an empty file */ + if (file_status(fname) == FN_FILE) { + if (replace_file(fname, fname_prev)) + goto error; + } + if (curve25519_keypair_write_to_file(&new_curve25519_keypair, fname, + "onion") < 0) { + log_err(LD_FS,"Couldn't write curve25519 onion key to \"%s\".",fname); + goto error; + } + log_info(LD_GENERAL, "Rotating onion key"); + tor_mutex_acquire(key_lock); + crypto_pk_free(lastonionkey); + lastonionkey = onionkey; + onionkey = prkey; + memcpy(&last_curve25519_onion_key, &curve25519_onion_key, + sizeof(curve25519_keypair_t)); + memcpy(&curve25519_onion_key, &new_curve25519_keypair, + sizeof(curve25519_keypair_t)); + now = time(NULL); + state->LastRotatedOnionKey = onionkey_set_at = now; + tor_mutex_release(key_lock); + mark_my_descriptor_dirty("rotated onion key"); + or_state_mark_dirty(state, get_options()->AvoidDiskWrites ? now+3600 : 0); + goto done; + error: + log_warn(LD_GENERAL, "Couldn't rotate onion key."); + if (prkey) + crypto_pk_free(prkey); + done: + memwipe(&new_curve25519_keypair, 0, sizeof(new_curve25519_keypair)); + tor_free(fname); + tor_free(fname_prev); +} + +/** Log greeting message that points to new relay lifecycle document the + * first time this function has been called. + */ +static void +log_new_relay_greeting(void) +{ + static int already_logged = 0; + + if (already_logged) + return; + + tor_log(LOG_NOTICE, LD_GENERAL, "You are running a new relay. " + "Thanks for helping the Tor network! If you wish to know " + "what will happen in the upcoming weeks regarding its usage, " + "have a look at https://blog.torproject.org/blog/lifecycle-of" + "-a-new-relay"); + + already_logged = 1; +} + +/** Load a curve25519 keypair from the file fname, writing it into + * keys_out. If the file isn't found, or is empty, and generate + * is true, create a new keypair and write it into the file. If there are + * errors, log them at level severity. Generate files using tag + * in their ASCII wrapper. */ +static int +init_curve25519_keypair_from_file(curve25519_keypair_t *keys_out, + const char *fname, + int generate, + int severity, + const char *tag) +{ + switch (file_status(fname)) { + case FN_DIR: + case FN_ERROR: + tor_log(severity, LD_FS,"Can't read key from \"%s\"", fname); + goto error; + /* treat empty key files as if the file doesn't exist, and, if generate + * is set, replace the empty file in curve25519_keypair_write_to_file() */ + case FN_NOENT: + case FN_EMPTY: + if (generate) { + if (!have_lockfile()) { + if (try_locking(get_options(), 0)<0) { + /* Make sure that --list-fingerprint only creates new keys + * if there is no possibility for a deadlock. */ + tor_log(severity, LD_FS, "Another Tor process has locked \"%s\". " + "Not writing any new keys.", fname); + /*XXXX The 'other process' might make a key in a second or two; + * maybe we should wait for it. */ + goto error; + } + } + log_info(LD_GENERAL, "No key found in \"%s\"; generating fresh key.", + fname); + if (curve25519_keypair_generate(keys_out, 1) < 0) + goto error; + if (curve25519_keypair_write_to_file(keys_out, fname, tag)<0) { + tor_log(severity, LD_FS, + "Couldn't write generated key to \"%s\".", fname); + memwipe(keys_out, 0, sizeof(*keys_out)); + goto error; + } + } else { + log_info(LD_GENERAL, "No key found in \"%s\"", fname); + } + return 0; + case FN_FILE: + { + char *tag_in=NULL; + if (curve25519_keypair_read_from_file(keys_out, &tag_in, fname) < 0) { + tor_log(severity, LD_GENERAL,"Error loading private key."); + tor_free(tag_in); + goto error; + } + if (!tag_in || strcmp(tag_in, tag)) { + tor_log(severity, LD_GENERAL,"Unexpected tag %s on private key.", + escaped(tag_in)); + tor_free(tag_in); + goto error; + } + tor_free(tag_in); + return 0; + } + default: + tor_assert(0); + } + + error: + return -1; +} + +/** Try to load the vote-signing private key and certificate for being a v3 + * directory authority, and make sure they match. If legacy, load a + * legacy key/cert set for emergency key migration; otherwise load the regular + * key/cert set. On success, store them into *key_out and + * *cert_out respectively, and return 0. On failure, return -1. */ +static int +load_authority_keyset(int legacy, crypto_pk_t **key_out, + authority_cert_t **cert_out) +{ + int r = -1; + char *fname = NULL, *cert = NULL; + const char *eos = NULL; + crypto_pk_t *signing_key = NULL; + authority_cert_t *parsed = NULL; + + fname = get_keydir_fname( + legacy ? "legacy_signing_key" : "authority_signing_key"); + signing_key = init_key_from_file(fname, 0, LOG_ERR, NULL); + if (!signing_key) { + log_warn(LD_DIR, "No version 3 directory key found in %s", fname); + goto done; + } + tor_free(fname); + fname = get_keydir_fname( + legacy ? "legacy_certificate" : "authority_certificate"); + cert = read_file_to_str(fname, 0, NULL); + if (!cert) { + log_warn(LD_DIR, "Signing key found, but no certificate found in %s", + fname); + goto done; + } + parsed = authority_cert_parse_from_string(cert, &eos); + if (!parsed) { + log_warn(LD_DIR, "Unable to parse certificate in %s", fname); + goto done; + } + if (!crypto_pk_eq_keys(signing_key, parsed->signing_key)) { + log_warn(LD_DIR, "Stored signing key does not match signing key in " + "certificate"); + goto done; + } + + crypto_pk_free(*key_out); + authority_cert_free(*cert_out); + + *key_out = signing_key; + *cert_out = parsed; + r = 0; + signing_key = NULL; + parsed = NULL; + + done: + tor_free(fname); + tor_free(cert); + crypto_pk_free(signing_key); + authority_cert_free(parsed); + return r; +} + +/** Load the v3 (voting) authority signing key and certificate, if they are + * present. Return -1 if anything is missing, mismatched, or unloadable; + * return 0 on success. */ +static int +init_v3_authority_keys(void) +{ + if (load_authority_keyset(0, &authority_signing_key, + &authority_key_certificate)<0) + return -1; + + if (get_options()->V3AuthUseLegacyKey && + load_authority_keyset(1, &legacy_signing_key, + &legacy_key_certificate)<0) + return -1; + + return 0; +} + +/** If we're a v3 authority, check whether we have a certificate that's + * likely to expire soon. Warn if we do, but not too often. */ +void +v3_authority_check_key_expiry(void) +{ + time_t now, expires; + static time_t last_warned = 0; + int badness, time_left, warn_interval; + if (!authdir_mode_v3(get_options()) || !authority_key_certificate) + return; + + now = time(NULL); + expires = authority_key_certificate->expires; + time_left = (int)( expires - now ); + if (time_left <= 0) { + badness = LOG_ERR; + warn_interval = 60*60; + } else if (time_left <= 24*60*60) { + badness = LOG_WARN; + warn_interval = 60*60; + } else if (time_left <= 24*60*60*7) { + badness = LOG_WARN; + warn_interval = 24*60*60; + } else if (time_left <= 24*60*60*30) { + badness = LOG_WARN; + warn_interval = 24*60*60*5; + } else { + return; + } + + if (last_warned + warn_interval > now) + return; + + if (time_left <= 0) { + tor_log(badness, LD_DIR, "Your v3 authority certificate has expired." + " Generate a new one NOW."); + } else if (time_left <= 24*60*60) { + tor_log(badness, LD_DIR, "Your v3 authority certificate expires in %d " + "hours; Generate a new one NOW.", time_left/(60*60)); + } else { + tor_log(badness, LD_DIR, "Your v3 authority certificate expires in %d " + "days; Generate a new one soon.", time_left/(24*60*60)); + } + last_warned = now; +} + +/** Get the lifetime of an onion key in days. This value is defined by the + * network consesus parameter "onion-key-rotation-days". Always returns a value + * between MIN_ONION_KEY_LIFETIME_DAYS and + * MAX_ONION_KEY_LIFETIME_DAYS. + */ +static int +get_onion_key_rotation_days_(void) +{ + return networkstatus_get_param(NULL, + "onion-key-rotation-days", + DEFAULT_ONION_KEY_LIFETIME_DAYS, + MIN_ONION_KEY_LIFETIME_DAYS, + MAX_ONION_KEY_LIFETIME_DAYS); +} + +/** Get the current lifetime of an onion key in seconds. This value is defined + * by the network consesus parameter "onion-key-rotation-days", but the value + * is converted to seconds. + */ +int +get_onion_key_lifetime(void) +{ + return get_onion_key_rotation_days_()*24*60*60; +} + +/** Get the grace period of an onion key in seconds. This value is defined by + * the network consesus parameter "onion-key-grace-period-days", but the value + * is converted to seconds. + */ +int +get_onion_key_grace_period(void) +{ + int grace_period; + grace_period = networkstatus_get_param(NULL, + "onion-key-grace-period-days", + DEFAULT_ONION_KEY_GRACE_PERIOD_DAYS, + MIN_ONION_KEY_GRACE_PERIOD_DAYS, + get_onion_key_rotation_days_()); + return grace_period*24*60*60; +} + +/** Set up Tor's TLS contexts, based on our configuration and keys. Return 0 + * on success, and -1 on failure. */ +int +router_initialize_tls_context(void) +{ + unsigned int flags = 0; + const or_options_t *options = get_options(); + int lifetime = options->SSLKeyLifetime; + if (public_server_mode(options)) + flags |= TOR_TLS_CTX_IS_PUBLIC_SERVER; + if (!lifetime) { /* we should guess a good ssl cert lifetime */ + + /* choose between 5 and 365 days, and round to the day */ + unsigned int five_days = 5*24*3600; + unsigned int one_year = 365*24*3600; + lifetime = crypto_rand_int_range(five_days, one_year); + lifetime -= lifetime % (24*3600); + + if (crypto_rand_int(2)) { + /* Half the time we expire at midnight, and half the time we expire + * one second before midnight. (Some CAs wobble their expiry times a + * bit in practice, perhaps to reduce collision attacks; see ticket + * 8443 for details about observed certs in the wild.) */ + lifetime--; + } + } + + /* It's ok to pass lifetime in as an unsigned int, since + * config_parse_interval() checked it. */ + return tor_tls_context_init(flags, + get_tlsclient_identity_key(), + server_mode(options) ? + get_server_identity_key() : NULL, + (unsigned int)lifetime); +} + +/** Compute fingerprint (or hashed fingerprint if hashed is 1) and write + * it to 'fingerprint' (or 'hashed-fingerprint'). Return 0 on success, or + * -1 if Tor should die, + */ +STATIC int +router_write_fingerprint(int hashed) +{ + char *keydir = NULL, *cp = NULL; + const char *fname = hashed ? "hashed-fingerprint" : + "fingerprint"; + char fingerprint[FINGERPRINT_LEN+1]; + const or_options_t *options = get_options(); + char *fingerprint_line = NULL; + int result = -1; + + keydir = get_datadir_fname(fname); + log_info(LD_GENERAL,"Dumping %sfingerprint to \"%s\"...", + hashed ? "hashed " : "", keydir); + if (!hashed) { + if (crypto_pk_get_fingerprint(get_server_identity_key(), + fingerprint, 0) < 0) { + log_err(LD_GENERAL,"Error computing fingerprint"); + goto done; + } + } else { + if (crypto_pk_get_hashed_fingerprint(get_server_identity_key(), + fingerprint) < 0) { + log_err(LD_GENERAL,"Error computing hashed fingerprint"); + goto done; + } + } + + tor_asprintf(&fingerprint_line, "%s %s\n", options->Nickname, fingerprint); + + /* Check whether we need to write the (hashed-)fingerprint file. */ + + cp = read_file_to_str(keydir, RFTS_IGNORE_MISSING, NULL); + if (!cp || strcmp(cp, fingerprint_line)) { + if (write_str_to_file(keydir, fingerprint_line, 0)) { + log_err(LD_FS, "Error writing %sfingerprint line to file", + hashed ? "hashed " : ""); + goto done; + } + } + + log_notice(LD_GENERAL, "Your Tor %s identity key fingerprint is '%s %s'", + hashed ? "bridge's hashed" : "server's", options->Nickname, + fingerprint); + + result = 0; + done: + tor_free(cp); + tor_free(keydir); + tor_free(fingerprint_line); + return result; +} + +static int +init_keys_common(void) +{ + if (!key_lock) + key_lock = tor_mutex_new(); + + /* There are a couple of paths that put us here before we've asked + * openssl to initialize itself. */ + if (crypto_global_init(get_options()->HardwareAccel, + get_options()->AccelName, + get_options()->AccelDir)) { + log_err(LD_BUG, "Unable to initialize OpenSSL. Exiting."); + return -1; + } + + return 0; +} + +int +init_keys_client(void) +{ + crypto_pk_t *prkey; + if (init_keys_common() < 0) + return -1; + + if (!(prkey = crypto_pk_new())) + return -1; + if (crypto_pk_generate_key(prkey)) { + crypto_pk_free(prkey); + return -1; + } + set_client_identity_key(prkey); + /* Create a TLS context. */ + if (router_initialize_tls_context() < 0) { + log_err(LD_GENERAL,"Error creating TLS context for Tor client."); + return -1; + } + return 0; +} + +/** Initialize all OR private keys, and the TLS context, as necessary. + * On OPs, this only initializes the tls context. Return 0 on success, + * or -1 if Tor should die. + */ +int +init_keys(void) +{ + char *keydir; + const char *mydesc; + crypto_pk_t *prkey; + char digest[DIGEST_LEN]; + char v3_digest[DIGEST_LEN]; + const or_options_t *options = get_options(); + dirinfo_type_t type; + time_t now = time(NULL); + dir_server_t *ds; + int v3_digest_set = 0; + authority_cert_t *cert = NULL; + + /* OP's don't need persistent keys; just make up an identity and + * initialize the TLS context. */ + if (!server_mode(options)) { + return init_keys_client(); + } + if (init_keys_common() < 0) + return -1; + + if (create_keys_directory(options) < 0) + return -1; + + /* 1a. Read v3 directory authority key/cert information. */ + memset(v3_digest, 0, sizeof(v3_digest)); + if (authdir_mode_v3(options)) { + if (init_v3_authority_keys()<0) { + log_err(LD_GENERAL, "We're configured as a V3 authority, but we " + "were unable to load our v3 authority keys and certificate! " + "Use tor-gencert to generate them. Dying."); + return -1; + } + cert = get_my_v3_authority_cert(); + if (cert) { + if (crypto_pk_get_digest(get_my_v3_authority_cert()->identity_key, + v3_digest) < 0) { + log_err(LD_BUG, "Couldn't compute my v3 authority identity key " + "digest."); + return -1; + } + v3_digest_set = 1; + } + } + + /* 1b. Read identity key. Make it if none is found. */ + keydir = get_keydir_fname("secret_id_key"); + log_info(LD_GENERAL,"Reading/making identity key \"%s\"...",keydir); + bool created = false; + prkey = init_key_from_file(keydir, 1, LOG_ERR, &created); + tor_free(keydir); + if (!prkey) return -1; + if (created) + log_new_relay_greeting(); + set_server_identity_key(prkey); + + /* 1c. If we are configured as a bridge, generate a client key; + * otherwise, set the server identity key as our client identity + * key. */ + if (public_server_mode(options)) { + set_client_identity_key(crypto_pk_dup_key(prkey)); /* set above */ + } else { + if (!(prkey = crypto_pk_new())) + return -1; + if (crypto_pk_generate_key(prkey)) { + crypto_pk_free(prkey); + return -1; + } + set_client_identity_key(prkey); + } + + /* 1d. Load all ed25519 keys */ + const int new_signing_key = load_ed_keys(options,now); + if (new_signing_key < 0) + return -1; + + /* 2. Read onion key. Make it if none is found. */ + keydir = get_keydir_fname("secret_onion_key"); + log_info(LD_GENERAL,"Reading/making onion key \"%s\"...",keydir); + prkey = init_key_from_file(keydir, 1, LOG_ERR, &created); + if (created) + log_new_relay_greeting(); + tor_free(keydir); + if (!prkey) return -1; + set_onion_key(prkey); + if (options->command == CMD_RUN_TOR) { + /* only mess with the state file if we're actually running Tor */ + or_state_t *state = get_or_state(); + if (state->LastRotatedOnionKey > 100 && state->LastRotatedOnionKey < now) { + /* We allow for some parsing slop, but we don't want to risk accepting + * values in the distant future. If we did, we might never rotate the + * onion key. */ + onionkey_set_at = state->LastRotatedOnionKey; + } else { + /* We have no LastRotatedOnionKey set; either we just created the key + * or it's a holdover from 0.1.2.4-alpha-dev or earlier. In either case, + * start the clock ticking now so that we will eventually rotate it even + * if we don't stay up for the full lifetime of an onion key. */ + state->LastRotatedOnionKey = onionkey_set_at = now; + or_state_mark_dirty(state, options->AvoidDiskWrites ? + time(NULL)+3600 : 0); + } + } + + keydir = get_keydir_fname("secret_onion_key.old"); + if (!lastonionkey && file_status(keydir) == FN_FILE) { + /* Load keys from non-empty files only. + * Missing old keys won't be replaced with freshly generated keys. */ + prkey = init_key_from_file(keydir, 0, LOG_ERR, 0); + if (prkey) + lastonionkey = prkey; + } + tor_free(keydir); + + { + /* 2b. Load curve25519 onion keys. */ + int r; + keydir = get_keydir_fname("secret_onion_key_ntor"); + r = init_curve25519_keypair_from_file(&curve25519_onion_key, + keydir, 1, LOG_ERR, "onion"); + tor_free(keydir); + if (r<0) + return -1; + + keydir = get_keydir_fname("secret_onion_key_ntor.old"); + if (tor_mem_is_zero((const char *) + last_curve25519_onion_key.pubkey.public_key, + CURVE25519_PUBKEY_LEN) && + file_status(keydir) == FN_FILE) { + /* Load keys from non-empty files only. + * Missing old keys won't be replaced with freshly generated keys. */ + init_curve25519_keypair_from_file(&last_curve25519_onion_key, + keydir, 0, LOG_ERR, "onion"); + } + tor_free(keydir); + } + + /* 3. Initialize link key and TLS context. */ + if (router_initialize_tls_context() < 0) { + log_err(LD_GENERAL,"Error initializing TLS context"); + return -1; + } + + /* 3b. Get an ed25519 link certificate. Note that we need to do this + * after we set up the TLS context */ + if (generate_ed_link_cert(options, now, new_signing_key > 0) < 0) { + log_err(LD_GENERAL,"Couldn't make link cert"); + return -1; + } + + /* 4. Build our router descriptor. */ + /* Must be called after keys are initialized. */ + mydesc = router_get_my_descriptor(); + if (authdir_mode_v3(options)) { + const char *m = NULL; + routerinfo_t *ri; + /* We need to add our own fingerprint so it gets recognized. */ + if (dirserv_add_own_fingerprint(get_server_identity_key())) { + log_err(LD_GENERAL,"Error adding own fingerprint to set of relays"); + return -1; + } + if (mydesc) { + was_router_added_t added; + ri = router_parse_entry_from_string(mydesc, NULL, 1, 0, NULL, NULL); + if (!ri) { + log_err(LD_GENERAL,"Generated a routerinfo we couldn't parse."); + return -1; + } + added = dirserv_add_descriptor(ri, &m, "self"); + if (!WRA_WAS_ADDED(added)) { + if (!WRA_WAS_OUTDATED(added)) { + log_err(LD_GENERAL, "Unable to add own descriptor to directory: %s", + m?m:"<unknown error>"); + return -1; + } else { + /* If the descriptor was outdated, that's ok. This can happen + * when some config options are toggled that affect workers, but + * we don't really need new keys yet so the descriptor doesn't + * change and the old one is still fresh. */ + log_info(LD_GENERAL, "Couldn't add own descriptor to directory " + "after key init: %s This is usually not a problem.", + m?m:"<unknown error>"); + } + } + } + } + + /* 5. Dump fingerprint and possibly hashed fingerprint to files. */ + if (router_write_fingerprint(0)) { + log_err(LD_FS, "Error writing fingerprint to file"); + return -1; + } + if (!public_server_mode(options) && router_write_fingerprint(1)) { + log_err(LD_FS, "Error writing hashed fingerprint to file"); + return -1; + } + + if (!authdir_mode(options)) + return 0; + /* 6. [authdirserver only] load approved-routers file */ + if (dirserv_load_fingerprint_file() < 0) { + log_err(LD_GENERAL,"Error loading fingerprints"); + return -1; + } + /* 6b. [authdirserver only] add own key to approved directories. */ + crypto_pk_get_digest(get_server_identity_key(), digest); + type = ((options->V3AuthoritativeDir ? + (V3_DIRINFO|MICRODESC_DIRINFO|EXTRAINFO_DIRINFO) : NO_DIRINFO) | + (options->BridgeAuthoritativeDir ? BRIDGE_DIRINFO : NO_DIRINFO)); + + ds = router_get_trusteddirserver_by_digest(digest); + if (!ds) { + ds = trusted_dir_server_new(options->Nickname, NULL, + router_get_advertised_dir_port(options, 0), + router_get_advertised_or_port(options), + NULL, + digest, + v3_digest, + type, 0.0); + if (!ds) { + log_err(LD_GENERAL,"We want to be a directory authority, but we " + "couldn't add ourselves to the authority list. Failing."); + return -1; + } + dir_server_add(ds); + } + if (ds->type != type) { + log_warn(LD_DIR, "Configured authority type does not match authority " + "type in DirAuthority list. Adjusting. (%d v %d)", + type, ds->type); + ds->type = type; + } + if (v3_digest_set && (ds->type & V3_DIRINFO) && + tor_memneq(v3_digest, ds->v3_identity_digest, DIGEST_LEN)) { + log_warn(LD_DIR, "V3 identity key does not match identity declared in " + "DirAuthority line. Adjusting."); + memcpy(ds->v3_identity_digest, v3_digest, DIGEST_LEN); + } + + if (cert) { /* add my own cert to the list of known certs */ + log_info(LD_DIR, "adding my own v3 cert"); + if (trusted_dirs_load_certs_from_string( + cert->cache_info.signed_descriptor_body, + TRUSTED_DIRS_CERTS_SRC_SELF, 0, + NULL)<0) { + log_warn(LD_DIR, "Unable to parse my own v3 cert! Failing."); + return -1; + } + } + + return 0; /* success */ +} + +/** The lower threshold of remaining bandwidth required to advertise (or + * automatically provide) directory services */ +/* XXX Should this be increased? */ +#define MIN_BW_TO_ADVERTISE_DIRSERVER 51200 + +/** Return true iff we have enough configured bandwidth to advertise or + * automatically provide directory services from cache directory + * information. */ +int +router_has_bandwidth_to_be_dirserver(const or_options_t *options) +{ + if (options->BandwidthRate < MIN_BW_TO_ADVERTISE_DIRSERVER) { + return 0; + } + if (options->RelayBandwidthRate > 0 && + options->RelayBandwidthRate < MIN_BW_TO_ADVERTISE_DIRSERVER) { + return 0; + } + return 1; +} + +/** Helper: Return 1 if we have sufficient resources for serving directory + * requests, return 0 otherwise. + * dir_port is either 0 or the configured DirPort number. + * If AccountingMax is set less than our advertised bandwidth, then don't + * serve requests. Likewise, if our advertised bandwidth is less than + * MIN_BW_TO_ADVERTISE_DIRSERVER, don't bother trying to serve requests. + */ +static int +router_should_be_dirserver(const or_options_t *options, int dir_port) +{ + static int advertising=1; /* start out assuming we will advertise */ + int new_choice=1; + const char *reason = NULL; + + if (accounting_is_enabled(options) && + get_options()->AccountingRule != ACCT_IN) { + /* Don't spend bytes for directory traffic if we could end up hibernating, + * but allow DirPort otherwise. Some relay operators set AccountingMax + * because they're confused or to get statistics. Directory traffic has a + * much larger effect on output than input so there is no reason to turn it + * off if using AccountingRule in. */ + int interval_length = accounting_get_interval_length(); + uint32_t effective_bw = get_effective_bwrate(options); + uint64_t acc_bytes; + if (!interval_length) { + log_warn(LD_BUG, "An accounting interval is not allowed to be zero " + "seconds long. Raising to 1."); + interval_length = 1; + } + log_info(LD_GENERAL, "Calculating whether to advertise %s: effective " + "bwrate: %u, AccountingMax: %"PRIu64", " + "accounting interval length %d", + dir_port ? "dirport" : "begindir", + effective_bw, (options->AccountingMax), + interval_length); + + acc_bytes = options->AccountingMax; + if (get_options()->AccountingRule == ACCT_SUM) + acc_bytes /= 2; + if (effective_bw >= + acc_bytes / interval_length) { + new_choice = 0; + reason = "AccountingMax enabled"; + } + } else if (! router_has_bandwidth_to_be_dirserver(options)) { + /* if we're advertising a small amount */ + new_choice = 0; + reason = "BandwidthRate under 50KB"; + } + + if (advertising != new_choice) { + if (new_choice == 1) { + if (dir_port > 0) + log_notice(LD_DIR, "Advertising DirPort as %d", dir_port); + else + log_notice(LD_DIR, "Advertising directory service support"); + } else { + tor_assert(reason); + log_notice(LD_DIR, "Not advertising Dir%s (Reason: %s)", + dir_port ? "Port" : "ectory Service support", reason); + } + advertising = new_choice; + } + + return advertising; +} + +/** Look at a variety of factors, and return 0 if we don't want to + * advertise the fact that we have a DirPort open or begindir support, else + * return 1. + * + * Where dir_port or supports_tunnelled_dir_requests are not relevant, they + * must be 0. + * + * Log a helpful message if we change our mind about whether to publish. + */ +static int +decide_to_advertise_dir_impl(const or_options_t *options, + uint16_t dir_port, + int supports_tunnelled_dir_requests) +{ + /* Part one: reasons to publish or not publish that aren't + * worth mentioning to the user, either because they're obvious + * or because they're normal behavior. */ + + /* short circuit the rest of the function */ + if (!dir_port && !supports_tunnelled_dir_requests) + return 0; + if (authdir_mode(options)) /* always publish */ + return 1; + if (net_is_disabled()) + return 0; + if (dir_port && !router_get_advertised_dir_port(options, dir_port)) + return 0; + if (supports_tunnelled_dir_requests && + !router_get_advertised_or_port(options)) + return 0; + + /* Part two: consider config options that could make us choose to + * publish or not publish that the user might find surprising. */ + return router_should_be_dirserver(options, dir_port); +} + +/** Front-end to decide_to_advertise_dir_impl(): return 0 if we don't want to + * advertise the fact that we have a DirPort open, else return the + * DirPort we want to advertise. + */ +int +router_should_advertise_dirport(const or_options_t *options, uint16_t dir_port) +{ + /* supports_tunnelled_dir_requests is not relevant, pass 0 */ + return decide_to_advertise_dir_impl(options, dir_port, 0) ? dir_port : 0; +} + +/** Front-end to decide_to_advertise_dir_impl(): return 0 if we don't want to + * advertise the fact that we support begindir requests, else return 1. + */ +static int +router_should_advertise_begindir(const or_options_t *options, + int supports_tunnelled_dir_requests) +{ + /* dir_port is not relevant, pass 0 */ + return decide_to_advertise_dir_impl(options, 0, + supports_tunnelled_dir_requests); +} + +/** Return true iff the combination of options in options and parameters + * in the consensus mean that we don't want to allow exits from circuits + * we got from addresses not known to be servers. */ +int +should_refuse_unknown_exits(const or_options_t *options) +{ + if (options->RefuseUnknownExits != -1) { + return options->RefuseUnknownExits; + } else { + return networkstatus_get_param(NULL, "refuseunknownexits", 1, 0, 1); + } +} + +/** Decide if we're a publishable server. We are a publishable server if: + * - We don't have the ClientOnly option set + * and + * - We have the PublishServerDescriptor option set to non-empty + * and + * - We have ORPort set + * and + * - We believe our ORPort and DirPort (if present) are reachable from + * the outside; or + * - We believe our ORPort is reachable from the outside, and we can't + * check our DirPort because the consensus has no exits; or + * - We are an authoritative directory server. + */ +static int +decide_if_publishable_server(void) +{ + const or_options_t *options = get_options(); + + if (options->ClientOnly) + return 0; + if (options->PublishServerDescriptor_ == NO_DIRINFO) + return 0; + if (!server_mode(options)) + return 0; + if (authdir_mode(options)) + return 1; + if (!router_get_advertised_or_port(options)) + return 0; + if (!check_whether_orport_reachable(options)) + return 0; + if (router_have_consensus_path() == CONSENSUS_PATH_INTERNAL) { + /* All set: there are no exits in the consensus (maybe this is a tiny + * test network), so we can't check our DirPort reachability. */ + return 1; + } else { + return check_whether_dirport_reachable(options); + } +} + +/** Initiate server descriptor upload as reasonable (if server is publishable, + * etc). force is as for router_upload_dir_desc_to_dirservers. + * + * We need to rebuild the descriptor if it's dirty even if we're not + * uploading, because our reachability testing *uses* our descriptor to + * determine what IP address and ports to test. + */ +void +consider_publishable_server(int force) +{ + int rebuilt; + + if (!server_mode(get_options())) + return; + + rebuilt = router_rebuild_descriptor(0); + if (decide_if_publishable_server()) { + set_server_advertised(1); + if (rebuilt == 0) + router_upload_dir_desc_to_dirservers(force); + } else { + set_server_advertised(0); + } +} + +/** Return the port of the first active listener of type + * listener_type. */ +/** XXX not a very good interface. it's not reliable when there are + multiple listeners. */ +uint16_t +router_get_active_listener_port_by_type_af(int listener_type, + sa_family_t family) +{ + /* Iterate all connections, find one of the right kind and return + the port. Not very sophisticated or fast, but effective. */ + smartlist_t *conns = get_connection_array(); + SMARTLIST_FOREACH_BEGIN(conns, connection_t *, conn) { + if (conn->type == listener_type && !conn->marked_for_close && + conn->socket_family == family) { + return conn->port; + } + } SMARTLIST_FOREACH_END(conn); + + return 0; +} + +/** Return the port that we should advertise as our ORPort; this is either + * the one configured in the ORPort option, or the one we actually bound to + * if ORPort is "auto". + */ +uint16_t +router_get_advertised_or_port(const or_options_t *options) +{ + return router_get_advertised_or_port_by_af(options, AF_INET); +} + +/** As router_get_advertised_or_port(), but allows an address family argument. + */ +uint16_t +router_get_advertised_or_port_by_af(const or_options_t *options, + sa_family_t family) +{ + int port = get_first_advertised_port_by_type_af(CONN_TYPE_OR_LISTENER, + family); + (void)options; + + /* If the port is in 'auto' mode, we have to use + router_get_listener_port_by_type(). */ + if (port == CFG_AUTO_PORT) + return router_get_active_listener_port_by_type_af(CONN_TYPE_OR_LISTENER, + family); + + return port; +} + +/** Return the port that we should advertise as our DirPort; + * this is one of three possibilities: + * The one that is passed as dirport if the DirPort option is 0, or + * the one configured in the DirPort option, + * or the one we actually bound to if DirPort is "auto". */ +uint16_t +router_get_advertised_dir_port(const or_options_t *options, uint16_t dirport) +{ + int dirport_configured = get_primary_dir_port(); + (void)options; + + if (!dirport_configured) + return dirport; + + if (dirport_configured == CFG_AUTO_PORT) + return router_get_active_listener_port_by_type_af(CONN_TYPE_DIR_LISTENER, + AF_INET); + + return dirport_configured; +} + +/* + * OR descriptor generation. + */ + +/** My routerinfo. */ +static routerinfo_t *desc_routerinfo = NULL; +/** My extrainfo */ +static extrainfo_t *desc_extrainfo = NULL; +/** Why did we most recently decide to regenerate our descriptor? Used to + * tell the authorities why we're sending it to them. */ +static const char *desc_gen_reason = "uninitialized reason"; +/** Since when has our descriptor been "clean"? 0 if we need to regenerate it + * now. */ +static time_t desc_clean_since = 0; +/** Why did we mark the descriptor dirty? */ +static const char *desc_dirty_reason = "Tor just started"; +/** Boolean: do we need to regenerate the above? */ +static int desc_needs_upload = 0; + +/** OR only: If force is true, or we haven't uploaded this + * descriptor successfully yet, try to upload our signed descriptor to + * all the directory servers we know about. + */ +void +router_upload_dir_desc_to_dirservers(int force) +{ + const routerinfo_t *ri; + extrainfo_t *ei; + char *msg; + size_t desc_len, extra_len = 0, total_len; + dirinfo_type_t auth = get_options()->PublishServerDescriptor_; + + ri = router_get_my_routerinfo(); + if (!ri) { + log_info(LD_GENERAL, "No descriptor; skipping upload"); + return; + } + ei = router_get_my_extrainfo(); + if (auth == NO_DIRINFO) + return; + if (!force && !desc_needs_upload) + return; + + log_info(LD_OR, "Uploading relay descriptor to directory authorities%s", + force ? " (forced)" : ""); + + desc_needs_upload = 0; + + desc_len = ri->cache_info.signed_descriptor_len; + extra_len = ei ? ei->cache_info.signed_descriptor_len : 0; + total_len = desc_len + extra_len + 1; + msg = tor_malloc(total_len); + memcpy(msg, ri->cache_info.signed_descriptor_body, desc_len); + if (ei) { + memcpy(msg+desc_len, ei->cache_info.signed_descriptor_body, extra_len); + } + msg[desc_len+extra_len] = 0; + + directory_post_to_dirservers(DIR_PURPOSE_UPLOAD_DIR, + (auth & BRIDGE_DIRINFO) ? + ROUTER_PURPOSE_BRIDGE : + ROUTER_PURPOSE_GENERAL, + auth, msg, desc_len, extra_len); + tor_free(msg); +} + +/** OR only: Check whether my exit policy says to allow connection to + * conn. Return 0 if we accept; non-0 if we reject. + */ +int +router_compare_to_my_exit_policy(const tor_addr_t *addr, uint16_t port) +{ + const routerinfo_t *me = router_get_my_routerinfo(); + if (!me) /* make sure routerinfo exists */ + return -1; + + /* make sure it's resolved to something. this way we can't get a + 'maybe' below. */ + if (tor_addr_is_null(addr)) + return -1; + + /* look at router_get_my_routerinfo()->exit_policy for both the v4 and the + * v6 policies. The exit_policy field in router_get_my_routerinfo() is a + * bit unusual, in that it contains IPv6 and IPv6 entries. We don't want to + * look at router_get_my_routerinfo()->ipv6_exit_policy, since that's a port + * summary. */ + if ((tor_addr_family(addr) == AF_INET || + tor_addr_family(addr) == AF_INET6)) { + return compare_tor_addr_to_addr_policy(addr, port, + me->exit_policy) != ADDR_POLICY_ACCEPTED; +#if 0 + } else if (tor_addr_family(addr) == AF_INET6) { + return get_options()->IPv6Exit && + desc_routerinfo->ipv6_exit_policy && + compare_tor_addr_to_short_policy(addr, port, + me->ipv6_exit_policy) != ADDR_POLICY_ACCEPTED; +#endif /* 0 */ + } else { + return -1; + } +} + +/** Return true iff my exit policy is reject *:*. Return -1 if we don't + * have a descriptor */ +MOCK_IMPL(int, +router_my_exit_policy_is_reject_star,(void)) +{ + const routerinfo_t *me = router_get_my_routerinfo(); + if (!me) /* make sure routerinfo exists */ + return -1; + + return me->policy_is_reject_star; +} + +/** Return true iff I'm a server and digest is equal to + * my server identity key digest. */ +int +router_digest_is_me(const char *digest) +{ + return (server_identitykey && + tor_memeq(server_identitykey_digest, digest, DIGEST_LEN)); +} + +/** Return my identity digest. */ +const uint8_t * +router_get_my_id_digest(void) +{ + return (const uint8_t *)server_identitykey_digest; +} + +/** Return true iff I'm a server and digest is equal to + * my identity digest. */ +int +router_extrainfo_digest_is_me(const char *digest) +{ + extrainfo_t *ei = router_get_my_extrainfo(); + if (!ei) + return 0; + + return tor_memeq(digest, + ei->cache_info.signed_descriptor_digest, + DIGEST_LEN); +} + +/** A wrapper around router_digest_is_me(). */ +int +router_is_me(const routerinfo_t *router) +{ + return router_digest_is_me(router->cache_info.identity_digest); +} + +/** Return a routerinfo for this OR, rebuilding a fresh one if + * necessary. Return NULL on error, or if called on an OP. */ +MOCK_IMPL(const routerinfo_t *, +router_get_my_routerinfo,(void)) +{ + return router_get_my_routerinfo_with_err(NULL); +} + +/** Return routerinfo of this OR. Rebuild it from + * scratch if needed. Set *err to 0 on success or to + * appropriate TOR_ROUTERINFO_ERROR_* value on failure. + */ +MOCK_IMPL(const routerinfo_t *, +router_get_my_routerinfo_with_err,(int *err)) +{ + if (!server_mode(get_options())) { + if (err) + *err = TOR_ROUTERINFO_ERROR_NOT_A_SERVER; + + return NULL; + } + + if (!desc_clean_since) { + int rebuild_err = router_rebuild_descriptor(0); + if (rebuild_err < 0) { + if (err) + *err = rebuild_err; + + return NULL; + } + } + + if (!desc_routerinfo) { + if (err) + *err = TOR_ROUTERINFO_ERROR_DESC_REBUILDING; + + return NULL; + } + + if (err) + *err = 0; + + return desc_routerinfo; +} + +/** OR only: Return a signed server descriptor for this OR, rebuilding a fresh + * one if necessary. Return NULL on error. + */ +const char * +router_get_my_descriptor(void) +{ + const char *body; + const routerinfo_t *me = router_get_my_routerinfo(); + if (! me) + return NULL; + tor_assert(me->cache_info.saved_location == SAVED_NOWHERE); + body = signed_descriptor_get_body(&me->cache_info); + /* Make sure this is nul-terminated. */ + tor_assert(!body[me->cache_info.signed_descriptor_len]); + log_debug(LD_GENERAL,"my desc is '%s'", body); + return body; +} + +/** Return the extrainfo document for this OR, or NULL if we have none. + * Rebuilt it (and the server descriptor) if necessary. */ +extrainfo_t * +router_get_my_extrainfo(void) +{ + if (!server_mode(get_options())) + return NULL; + if (router_rebuild_descriptor(0)) + return NULL; + return desc_extrainfo; +} + +/** Return a human-readable string describing what triggered us to generate + * our current descriptor, or NULL if we don't know. */ +const char * +router_get_descriptor_gen_reason(void) +{ + return desc_gen_reason; +} + +/** A list of nicknames that we've warned about including in our family + * declaration verbatim rather than as digests. */ +static smartlist_t *warned_nonexistent_family = NULL; + +static int router_guess_address_from_dir_headers(uint32_t *guess); + +/** Make a current best guess at our address, either because + * it's configured in torrc, or because we've learned it from + * dirserver headers. Place the answer in *addr and return + * 0 on success, else return -1 if we have no guess. + * + * If cache_only is true, just return any cached answers, and + * don't try to get any new answers. + */ +MOCK_IMPL(int, +router_pick_published_address,(const or_options_t *options, uint32_t *addr, + int cache_only)) +{ + /* First, check the cached output from resolve_my_address(). */ + *addr = get_last_resolved_addr(); + if (*addr) + return 0; + + /* Second, consider doing a resolve attempt right here. */ + if (!cache_only) { + if (resolve_my_address(LOG_INFO, options, addr, NULL, NULL) >= 0) { + log_info(LD_CONFIG,"Success: chose address '%s'.", fmt_addr32(*addr)); + return 0; + } + } + + /* Third, check the cached output from router_new_address_suggestion(). */ + if (router_guess_address_from_dir_headers(addr) >= 0) + return 0; + + /* We have no useful cached answers. Return failure. */ + return -1; +} + +/* Like router_check_descriptor_address_consistency, but specifically for the + * ORPort or DirPort. + * listener_type is either CONN_TYPE_OR_LISTENER or CONN_TYPE_DIR_LISTENER. */ +static void +router_check_descriptor_address_port_consistency(uint32_t ipv4h_desc_addr, + int listener_type) +{ + tor_assert(listener_type == CONN_TYPE_OR_LISTENER || + listener_type == CONN_TYPE_DIR_LISTENER); + + /* The first advertised Port may be the magic constant CFG_AUTO_PORT. + */ + int port_v4_cfg = get_first_advertised_port_by_type_af(listener_type, + AF_INET); + if (port_v4_cfg != 0 && + !port_exists_by_type_addr32h_port(listener_type, + ipv4h_desc_addr, port_v4_cfg, 1)) { + const tor_addr_t *port_addr = get_first_advertised_addr_by_type_af( + listener_type, + AF_INET); + /* If we're building a descriptor with no advertised address, + * something is terribly wrong. */ + tor_assert(port_addr); + + tor_addr_t desc_addr; + char port_addr_str[TOR_ADDR_BUF_LEN]; + char desc_addr_str[TOR_ADDR_BUF_LEN]; + + tor_addr_to_str(port_addr_str, port_addr, TOR_ADDR_BUF_LEN, 0); + + tor_addr_from_ipv4h(&desc_addr, ipv4h_desc_addr); + tor_addr_to_str(desc_addr_str, &desc_addr, TOR_ADDR_BUF_LEN, 0); + + const char *listener_str = (listener_type == CONN_TYPE_OR_LISTENER ? + "OR" : "Dir"); + log_warn(LD_CONFIG, "The IPv4 %sPort address %s does not match the " + "descriptor address %s. If you have a static public IPv4 " + "address, use 'Address <IPv4>' and 'OutboundBindAddress " + "<IPv4>'. If you are behind a NAT, use two %sPort lines: " + "'%sPort <PublicPort> NoListen' and '%sPort <InternalPort> " + "NoAdvertise'.", + listener_str, port_addr_str, desc_addr_str, listener_str, + listener_str, listener_str); + } +} + +/* Tor relays only have one IPv4 address in the descriptor, which is derived + * from the Address torrc option, or guessed using various methods in + * router_pick_published_address(). + * Warn the operator if there is no ORPort on the descriptor address + * ipv4h_desc_addr. + * Warn the operator if there is no DirPort on the descriptor address. + * This catches a few common config errors: + * - operators who expect ORPorts and DirPorts to be advertised on the + * ports' listen addresses, rather than the torrc Address (or guessed + * addresses in the absence of an Address config). This includes + * operators who attempt to put their ORPort and DirPort on different + * addresses; + * - discrepancies between guessed addresses and configured listen + * addresses (when the Address option isn't set). + * If a listener is listening on all IPv4 addresses, it is assumed that it + * is listening on the configured Address, and no messages are logged. + * If an operators has specified NoAdvertise ORPorts in a NAT setting, + * no messages are logged, unless they have specified other advertised + * addresses. + * The message tells operators to configure an ORPort and DirPort that match + * the Address (using NoListen if needed). + */ +static void +router_check_descriptor_address_consistency(uint32_t ipv4h_desc_addr) +{ + router_check_descriptor_address_port_consistency(ipv4h_desc_addr, + CONN_TYPE_OR_LISTENER); + router_check_descriptor_address_port_consistency(ipv4h_desc_addr, + CONN_TYPE_DIR_LISTENER); +} + +/** Build a fresh routerinfo, signed server descriptor, and extra-info document + * for this OR. Set r to the generated routerinfo, e to the generated + * extra-info document. Return 0 on success, -1 on temporary error. Failure to + * generate an extra-info document is not an error and is indicated by setting + * e to NULL. Caller is responsible for freeing generated documents if 0 is + * returned. + */ +int +router_build_fresh_descriptor(routerinfo_t **r, extrainfo_t **e) +{ + routerinfo_t *ri; + extrainfo_t *ei; + uint32_t addr; + char platform[256]; + int hibernating = we_are_hibernating(); + const or_options_t *options = get_options(); + + if (router_pick_published_address(options, &addr, 0) < 0) { + log_warn(LD_CONFIG, "Don't know my address while generating descriptor"); + return TOR_ROUTERINFO_ERROR_NO_EXT_ADDR; + } + + /* Log a message if the address in the descriptor doesn't match the ORPort + * and DirPort addresses configured by the operator. */ + router_check_descriptor_address_consistency(addr); + + ri = tor_malloc_zero(sizeof(routerinfo_t)); + ri->cache_info.routerlist_index = -1; + ri->nickname = tor_strdup(options->Nickname); + ri->addr = addr; + ri->or_port = router_get_advertised_or_port(options); + ri->dir_port = router_get_advertised_dir_port(options, 0); + ri->supports_tunnelled_dir_requests = + directory_permits_begindir_requests(options); + ri->cache_info.published_on = time(NULL); + /* get_onion_key() must invoke from main thread */ + router_set_rsa_onion_pkey(get_onion_key(), &ri->onion_pkey, + &ri->onion_pkey_len); + + ri->onion_curve25519_pkey = + tor_memdup(&get_current_curve25519_keypair()->pubkey, + sizeof(curve25519_public_key_t)); + + /* For now, at most one IPv6 or-address is being advertised. */ + { + const port_cfg_t *ipv6_orport = NULL; + SMARTLIST_FOREACH_BEGIN(get_configured_ports(), const port_cfg_t *, p) { + if (p->type == CONN_TYPE_OR_LISTENER && + ! p->server_cfg.no_advertise && + ! p->server_cfg.bind_ipv4_only && + tor_addr_family(&p->addr) == AF_INET6) { + /* Like IPv4, if the relay is configured using the default + * authorities, disallow internal IPs. Otherwise, allow them. */ + const int default_auth = using_default_dir_authorities(options); + if (! tor_addr_is_internal(&p->addr, 0) || ! default_auth) { + ipv6_orport = p; + break; + } else { + char addrbuf[TOR_ADDR_BUF_LEN]; + log_warn(LD_CONFIG, + "Unable to use configured IPv6 address \"%s\" in a " + "descriptor. Skipping it. " + "Try specifying a globally reachable address explicitly.", + tor_addr_to_str(addrbuf, &p->addr, sizeof(addrbuf), 1)); + } + } + } SMARTLIST_FOREACH_END(p); + if (ipv6_orport) { + tor_addr_copy(&ri->ipv6_addr, &ipv6_orport->addr); + ri->ipv6_orport = ipv6_orport->port; + } + } + + ri->identity_pkey = crypto_pk_dup_key(get_server_identity_key()); + if (BUG(crypto_pk_get_digest(ri->identity_pkey, + ri->cache_info.identity_digest) < 0)) { + routerinfo_free(ri); + return TOR_ROUTERINFO_ERROR_DIGEST_FAILED; + } + ri->cache_info.signing_key_cert = + tor_cert_dup(get_master_signing_key_cert()); + + get_platform_str(platform, sizeof(platform)); + ri->platform = tor_strdup(platform); + + ri->protocol_list = tor_strdup(protover_get_supported_protocols()); + + /* compute ri->bandwidthrate as the min of various options */ + ri->bandwidthrate = get_effective_bwrate(options); + + /* and compute ri->bandwidthburst similarly */ + ri->bandwidthburst = get_effective_bwburst(options); + + /* Report bandwidth, unless we're hibernating or shutting down */ + ri->bandwidthcapacity = hibernating ? 0 : rep_hist_bandwidth_assess(); + + if (dns_seems_to_be_broken() || has_dns_init_failed()) { + /* DNS is screwed up; don't claim to be an exit. */ + policies_exit_policy_append_reject_star(&ri->exit_policy); + } else { + policies_parse_exit_policy_from_options(options,ri->addr,&ri->ipv6_addr, + &ri->exit_policy); + } + ri->policy_is_reject_star = + policy_is_reject_star(ri->exit_policy, AF_INET, 1) && + policy_is_reject_star(ri->exit_policy, AF_INET6, 1); + + if (options->IPv6Exit) { + char *p_tmp = policy_summarize(ri->exit_policy, AF_INET6); + if (p_tmp) + ri->ipv6_exit_policy = parse_short_policy(p_tmp); + tor_free(p_tmp); + } + + if (options->MyFamily && ! options->BridgeRelay) { + if (!warned_nonexistent_family) + warned_nonexistent_family = smartlist_new(); + ri->declared_family = smartlist_new(); + config_line_t *family; + for (family = options->MyFamily; family; family = family->next) { + char *name = family->value; + const node_t *member; + if (!strcasecmp(name, options->Nickname)) + continue; /* Don't list ourself, that's redundant */ + else + member = node_get_by_nickname(name, 0); + if (!member) { + int is_legal = is_legal_nickname_or_hexdigest(name); + if (!smartlist_contains_string(warned_nonexistent_family, name) && + !is_legal_hexdigest(name)) { + if (is_legal) + log_warn(LD_CONFIG, + "I have no descriptor for the router named \"%s\" in my " + "declared family; I'll use the nickname as is, but " + "this may confuse clients.", name); + else + log_warn(LD_CONFIG, "There is a router named \"%s\" in my " + "declared family, but that isn't a legal nickname. " + "Skipping it.", escaped(name)); + smartlist_add_strdup(warned_nonexistent_family, name); + } + if (is_legal) { + smartlist_add_strdup(ri->declared_family, name); + } + } else if (router_digest_is_me(member->identity)) { + /* Don't list ourself in our own family; that's redundant */ + /* XXX shouldn't be possible */ + } else { + char *fp = tor_malloc(HEX_DIGEST_LEN+2); + fp[0] = '$'; + base16_encode(fp+1,HEX_DIGEST_LEN+1, + member->identity, DIGEST_LEN); + smartlist_add(ri->declared_family, fp); + if (smartlist_contains_string(warned_nonexistent_family, name)) + smartlist_string_remove(warned_nonexistent_family, name); + } + } + + /* remove duplicates from the list */ + smartlist_sort_strings(ri->declared_family); + smartlist_uniq_strings(ri->declared_family); + } + + /* Now generate the extrainfo. */ + ei = tor_malloc_zero(sizeof(extrainfo_t)); + ei->cache_info.is_extrainfo = 1; + strlcpy(ei->nickname, get_options()->Nickname, sizeof(ei->nickname)); + ei->cache_info.published_on = ri->cache_info.published_on; + ei->cache_info.signing_key_cert = + tor_cert_dup(get_master_signing_key_cert()); + + memcpy(ei->cache_info.identity_digest, ri->cache_info.identity_digest, + DIGEST_LEN); + if (extrainfo_dump_to_string(&ei->cache_info.signed_descriptor_body, + ei, get_server_identity_key(), + get_master_signing_keypair()) < 0) { + log_warn(LD_BUG, "Couldn't generate extra-info descriptor."); + extrainfo_free(ei); + ei = NULL; + } else { + ei->cache_info.signed_descriptor_len = + strlen(ei->cache_info.signed_descriptor_body); + router_get_extrainfo_hash(ei->cache_info.signed_descriptor_body, + ei->cache_info.signed_descriptor_len, + ei->cache_info.signed_descriptor_digest); + crypto_digest256((char*) ei->digest256, + ei->cache_info.signed_descriptor_body, + ei->cache_info.signed_descriptor_len, + DIGEST_SHA256); + } + + /* Now finish the router descriptor. */ + if (ei) { + memcpy(ri->cache_info.extra_info_digest, + ei->cache_info.signed_descriptor_digest, + DIGEST_LEN); + memcpy(ri->cache_info.extra_info_digest256, + ei->digest256, + DIGEST256_LEN); + } else { + /* ri was allocated with tor_malloc_zero, so there is no need to + * zero ri->cache_info.extra_info_digest here. */ + } + if (! (ri->cache_info.signed_descriptor_body = + router_dump_router_to_string(ri, get_server_identity_key(), + get_onion_key(), + get_current_curve25519_keypair(), + get_master_signing_keypair())) ) { + log_warn(LD_BUG, "Couldn't generate router descriptor."); + routerinfo_free(ri); + extrainfo_free(ei); + return TOR_ROUTERINFO_ERROR_CANNOT_GENERATE; + } + ri->cache_info.signed_descriptor_len = + strlen(ri->cache_info.signed_descriptor_body); + + ri->purpose = + options->BridgeRelay ? ROUTER_PURPOSE_BRIDGE : ROUTER_PURPOSE_GENERAL; + if (options->BridgeRelay) { + /* Bridges shouldn't be able to send their descriptors unencrypted, + anyway, since they don't have a DirPort, and always connect to the + bridge authority anonymously. But just in case they somehow think of + sending them on an unencrypted connection, don't allow them to try. */ + ri->cache_info.send_unencrypted = 0; + if (ei) + ei->cache_info.send_unencrypted = 0; + } else { + ri->cache_info.send_unencrypted = 1; + if (ei) + ei->cache_info.send_unencrypted = 1; + } + + router_get_router_hash(ri->cache_info.signed_descriptor_body, + strlen(ri->cache_info.signed_descriptor_body), + ri->cache_info.signed_descriptor_digest); + + if (ei) { + tor_assert(! + routerinfo_incompatible_with_extrainfo(ri->identity_pkey, ei, + &ri->cache_info, NULL)); + } + + *r = ri; + *e = ei; + return 0; +} + +/** If force is true, or our descriptor is out-of-date, rebuild a fresh + * routerinfo, signed server descriptor, and extra-info document for this OR. + * Return 0 on success, -1 on temporary error. + */ +int +router_rebuild_descriptor(int force) +{ + int err = 0; + routerinfo_t *ri; + extrainfo_t *ei; + uint32_t addr; + const or_options_t *options = get_options(); + + if (desc_clean_since && !force) + return 0; + + if (router_pick_published_address(options, &addr, 0) < 0 || + router_get_advertised_or_port(options) == 0) { + /* Stop trying to rebuild our descriptor every second. We'll + * learn that it's time to try again when ip_address_changed() + * marks it dirty. */ + desc_clean_since = time(NULL); + return TOR_ROUTERINFO_ERROR_DESC_REBUILDING; + } + + log_info(LD_OR, "Rebuilding relay descriptor%s", force ? " (forced)" : ""); + + err = router_build_fresh_descriptor(&ri, &ei); + if (err < 0) { + return err; + } + + routerinfo_free(desc_routerinfo); + desc_routerinfo = ri; + extrainfo_free(desc_extrainfo); + desc_extrainfo = ei; + + desc_clean_since = time(NULL); + desc_needs_upload = 1; + desc_gen_reason = desc_dirty_reason; + if (BUG(desc_gen_reason == NULL)) { + desc_gen_reason = "descriptor was marked dirty earlier, for no reason."; + } + desc_dirty_reason = NULL; + control_event_my_descriptor_changed(); + return 0; +} + +/** If our router descriptor ever goes this long without being regenerated + * because something changed, we force an immediate regenerate-and-upload. */ +#define FORCE_REGENERATE_DESCRIPTOR_INTERVAL (18*60*60) + +/** If our router descriptor seems to be missing or unacceptable according + * to the authorities, regenerate and reupload it _this_ often. */ +#define FAST_RETRY_DESCRIPTOR_INTERVAL (90*60) + +/** Mark descriptor out of date if it's been "too long" since we last tried + * to upload one. */ +void +mark_my_descriptor_dirty_if_too_old(time_t now) +{ + networkstatus_t *ns; + const routerstatus_t *rs; + const char *retry_fast_reason = NULL; /* Set if we should retry frequently */ + const time_t slow_cutoff = now - FORCE_REGENERATE_DESCRIPTOR_INTERVAL; + const time_t fast_cutoff = now - FAST_RETRY_DESCRIPTOR_INTERVAL; + + /* If it's already dirty, don't mark it. */ + if (! desc_clean_since) + return; + + /* If it's older than FORCE_REGENERATE_DESCRIPTOR_INTERVAL, it's always + * time to rebuild it. */ + if (desc_clean_since < slow_cutoff) { + mark_my_descriptor_dirty("time for new descriptor"); + return; + } + /* Now we see whether we want to be retrying frequently or no. The + * rule here is that we'll retry frequently if we aren't listed in the + * live consensus we have, or if the publication time of the + * descriptor listed for us in the consensus is very old. */ + ns = networkstatus_get_live_consensus(now); + if (ns) { + rs = networkstatus_vote_find_entry(ns, server_identitykey_digest); + if (rs == NULL) + retry_fast_reason = "not listed in consensus"; + else if (rs->published_on < slow_cutoff) + retry_fast_reason = "version listed in consensus is quite old"; + } + + if (retry_fast_reason && desc_clean_since < fast_cutoff) + mark_my_descriptor_dirty(retry_fast_reason); +} + +/** Call when the current descriptor is out of date. */ +void +mark_my_descriptor_dirty(const char *reason) +{ + const or_options_t *options = get_options(); + if (BUG(reason == NULL)) { + reason = "marked descriptor dirty for unspecified reason"; + } + if (server_mode(options) && options->PublishServerDescriptor_) + log_info(LD_OR, "Decided to publish new relay descriptor: %s", reason); + desc_clean_since = 0; + if (!desc_dirty_reason) + desc_dirty_reason = reason; +} + +/** How frequently will we republish our descriptor because of large (factor + * of 2) shifts in estimated bandwidth? Note: We don't use this constant + * if our previous bandwidth estimate was exactly 0. */ +#define MAX_BANDWIDTH_CHANGE_FREQ (3*60*60) + +/** Maximum uptime to republish our descriptor because of large shifts in + * estimated bandwidth. */ +#define MAX_UPTIME_BANDWIDTH_CHANGE (24*60*60) + +/** By which factor bandwidth shifts have to change to be considered large. */ +#define BANDWIDTH_CHANGE_FACTOR 2 + +/** Check whether bandwidth has changed a lot since the last time we announced + * bandwidth while the uptime is smaller than MAX_UPTIME_BANDWIDTH_CHANGE. + * If so, mark our descriptor dirty. */ +void +check_descriptor_bandwidth_changed(time_t now) +{ + static time_t last_changed = 0; + uint64_t prev, cur; + const int hibernating = we_are_hibernating(); + + /* If the relay uptime is bigger than MAX_UPTIME_BANDWIDTH_CHANGE, + * the next regularly scheduled descriptor update (18h) will be enough */ + if (get_uptime() > MAX_UPTIME_BANDWIDTH_CHANGE && !hibernating) + return; + + const routerinfo_t *my_ri = router_get_my_routerinfo(); + + if (!my_ri) + return; + + prev = my_ri->bandwidthcapacity; + + /* Consider ourselves to have zero bandwidth if we're hibernating or + * shutting down. */ + cur = hibernating ? 0 : rep_hist_bandwidth_assess(); + + if ((prev != cur && (!prev || !cur)) || + cur > (prev * BANDWIDTH_CHANGE_FACTOR) || + cur < (prev / BANDWIDTH_CHANGE_FACTOR) ) { + if (last_changed+MAX_BANDWIDTH_CHANGE_FREQ < now || !prev) { + log_info(LD_GENERAL, + "Measured bandwidth has changed; rebuilding descriptor."); + mark_my_descriptor_dirty("bandwidth has changed"); + last_changed = now; + } + } +} + +/** Note at log level severity that our best guess of address has changed from + * prev to cur. */ +static void +log_addr_has_changed(int severity, + const tor_addr_t *prev, + const tor_addr_t *cur, + const char *source) +{ + char addrbuf_prev[TOR_ADDR_BUF_LEN]; + char addrbuf_cur[TOR_ADDR_BUF_LEN]; + + if (BUG(!server_mode(get_options()))) + return; + + if (tor_addr_to_str(addrbuf_prev, prev, sizeof(addrbuf_prev), 1) == NULL) + strlcpy(addrbuf_prev, "???", TOR_ADDR_BUF_LEN); + if (tor_addr_to_str(addrbuf_cur, cur, sizeof(addrbuf_cur), 1) == NULL) + strlcpy(addrbuf_cur, "???", TOR_ADDR_BUF_LEN); + + if (!tor_addr_is_null(prev)) + log_fn(severity, LD_GENERAL, + "Our IP Address has changed from %s to %s; " + "rebuilding descriptor (source: %s).", + addrbuf_prev, addrbuf_cur, source); + else + log_notice(LD_GENERAL, + "Guessed our IP address as %s (source: %s).", + addrbuf_cur, source); +} + +/** Check whether our own address as defined by the Address configuration + * has changed. This is for routers that get their address from a service + * like dyndns. If our address has changed, mark our descriptor dirty. */ +void +check_descriptor_ipaddress_changed(time_t now) +{ + uint32_t prev, cur; + const or_options_t *options = get_options(); + const char *method = NULL; + char *hostname = NULL; + const routerinfo_t *my_ri = router_get_my_routerinfo(); + + (void) now; + + if (my_ri == NULL) /* make sure routerinfo exists */ + return; + + /* XXXX ipv6 */ + prev = my_ri->addr; + if (resolve_my_address(LOG_INFO, options, &cur, &method, &hostname) < 0) { + log_info(LD_CONFIG,"options->Address didn't resolve into an IP."); + return; + } + + if (prev != cur) { + char *source; + tor_addr_t tmp_prev, tmp_cur; + + tor_addr_from_ipv4h(&tmp_prev, prev); + tor_addr_from_ipv4h(&tmp_cur, cur); + + tor_asprintf(&source, "METHOD=%s%s%s", method, + hostname ? " HOSTNAME=" : "", + hostname ? hostname : ""); + + log_addr_has_changed(LOG_NOTICE, &tmp_prev, &tmp_cur, source); + tor_free(source); + + ip_address_changed(0); + } + + tor_free(hostname); +} + +/** The most recently guessed value of our IP address, based on directory + * headers. */ +static tor_addr_t last_guessed_ip = TOR_ADDR_NULL; + +/** A directory server d_conn told us our IP address is + * suggestion. + * If this address is different from the one we think we are now, and + * if our computer doesn't actually know its IP address, then switch. */ +void +router_new_address_suggestion(const char *suggestion, + const dir_connection_t *d_conn) +{ + tor_addr_t addr; + uint32_t cur = 0; /* Current IPv4 address. */ + const or_options_t *options = get_options(); + + /* first, learn what the IP address actually is */ + if (tor_addr_parse(&addr, suggestion) == -1) { + log_debug(LD_DIR, "Malformed X-Your-Address-Is header %s. Ignoring.", + escaped(suggestion)); + return; + } + + log_debug(LD_DIR, "Got X-Your-Address-Is: %s.", suggestion); + + if (!server_mode(options)) { + tor_addr_copy(&last_guessed_ip, &addr); + return; + } + + /* XXXX ipv6 */ + cur = get_last_resolved_addr(); + if (cur || + resolve_my_address(LOG_INFO, options, &cur, NULL, NULL) >= 0) { + /* We're all set -- we already know our address. Great. */ + tor_addr_from_ipv4h(&last_guessed_ip, cur); /* store it in case we + need it later */ + return; + } + if (tor_addr_is_internal(&addr, 0)) { + /* Don't believe anybody who says our IP is, say, 127.0.0.1. */ + return; + } + if (tor_addr_eq(&d_conn->base_.addr, &addr)) { + /* Don't believe anybody who says our IP is their IP. */ + log_debug(LD_DIR, "A directory server told us our IP address is %s, " + "but they are just reporting their own IP address. Ignoring.", + suggestion); + return; + } + + /* Okay. We can't resolve our own address, and X-Your-Address-Is is giving + * us an answer different from what we had the last time we managed to + * resolve it. */ + if (!tor_addr_eq(&last_guessed_ip, &addr)) { + control_event_server_status(LOG_NOTICE, + "EXTERNAL_ADDRESS ADDRESS=%s METHOD=DIRSERV", + suggestion); + log_addr_has_changed(LOG_NOTICE, &last_guessed_ip, &addr, + d_conn->base_.address); + ip_address_changed(0); + tor_addr_copy(&last_guessed_ip, &addr); /* router_rebuild_descriptor() + will fetch it */ + } +} + +/** We failed to resolve our address locally, but we'd like to build + * a descriptor and publish / test reachability. If we have a guess + * about our address based on directory headers, answer it and return + * 0; else return -1. */ +static int +router_guess_address_from_dir_headers(uint32_t *guess) +{ + if (!tor_addr_is_null(&last_guessed_ip)) { + *guess = tor_addr_to_ipv4h(&last_guessed_ip); + return 0; + } + return -1; +} + +/** Set platform (max length len) to a NUL-terminated short + * string describing the version of Tor and the operating system we're + * currently running on. + */ +STATIC void +get_platform_str(char *platform, size_t len) +{ + tor_snprintf(platform, len, "Tor %s on %s", + get_short_version(), get_uname()); +} + +/* XXX need to audit this thing and count fenceposts. maybe + * refactor so we don't have to keep asking if we're + * near the end of maxlen? + */ +#define DEBUG_ROUTER_DUMP_ROUTER_TO_STRING + +/** OR only: Given a routerinfo for this router, and an identity key to sign + * with, encode the routerinfo as a signed server descriptor and return a new + * string encoding the result, or NULL on failure. + */ +char * +router_dump_router_to_string(routerinfo_t *router, + const crypto_pk_t *ident_key, + const crypto_pk_t *tap_key, + const curve25519_keypair_t *ntor_keypair, + const ed25519_keypair_t *signing_keypair) +{ + char *address = NULL; + char *onion_pkey = NULL; /* Onion key, PEM-encoded. */ + crypto_pk_t *rsa_pubkey = NULL; + char *identity_pkey = NULL; /* Identity key, PEM-encoded. */ + char digest[DIGEST256_LEN]; + char published[ISO_TIME_LEN+1]; + char fingerprint[FINGERPRINT_LEN+1]; + char *extra_info_line = NULL; + size_t onion_pkeylen, identity_pkeylen; + char *family_line = NULL; + char *extra_or_address = NULL; + const or_options_t *options = get_options(); + smartlist_t *chunks = NULL; + char *output = NULL; + const int emit_ed_sigs = signing_keypair && + router->cache_info.signing_key_cert; + char *ed_cert_line = NULL; + char *rsa_tap_cc_line = NULL; + char *ntor_cc_line = NULL; + char *proto_line = NULL; + + /* Make sure the identity key matches the one in the routerinfo. */ + if (!crypto_pk_eq_keys(ident_key, router->identity_pkey)) { + log_warn(LD_BUG,"Tried to sign a router with a private key that didn't " + "match router's public key!"); + goto err; + } + if (emit_ed_sigs) { + if (!router->cache_info.signing_key_cert->signing_key_included || + !ed25519_pubkey_eq(&router->cache_info.signing_key_cert->signed_key, + &signing_keypair->pubkey)) { + log_warn(LD_BUG, "Tried to sign a router descriptor with a mismatched " + "ed25519 key chain %d", + router->cache_info.signing_key_cert->signing_key_included); + goto err; + } + } + + /* record our fingerprint, so we can include it in the descriptor */ + if (crypto_pk_get_fingerprint(router->identity_pkey, fingerprint, 1)<0) { + log_err(LD_BUG,"Error computing fingerprint"); + goto err; + } + + if (emit_ed_sigs) { + /* Encode ed25519 signing cert */ + char ed_cert_base64[256]; + char ed_fp_base64[ED25519_BASE64_LEN+1]; + if (base64_encode(ed_cert_base64, sizeof(ed_cert_base64), + (const char*)router->cache_info.signing_key_cert->encoded, + router->cache_info.signing_key_cert->encoded_len, + BASE64_ENCODE_MULTILINE) < 0) { + log_err(LD_BUG,"Couldn't base64-encode signing key certificate!"); + goto err; + } + if (ed25519_public_to_base64(ed_fp_base64, + &router->cache_info.signing_key_cert->signing_key)<0) { + log_err(LD_BUG,"Couldn't base64-encode identity key\n"); + goto err; + } + tor_asprintf(&ed_cert_line, "identity-ed25519\n" + "-----BEGIN ED25519 CERT-----\n" + "%s" + "-----END ED25519 CERT-----\n" + "master-key-ed25519 %s\n", + ed_cert_base64, ed_fp_base64); + } + + /* PEM-encode the onion key */ + rsa_pubkey = router_get_rsa_onion_pkey(router->onion_pkey, + router->onion_pkey_len); + if (crypto_pk_write_public_key_to_string(rsa_pubkey, + &onion_pkey,&onion_pkeylen)<0) { + log_warn(LD_BUG,"write onion_pkey to string failed!"); + goto err; + } + + /* PEM-encode the identity key */ + if (crypto_pk_write_public_key_to_string(router->identity_pkey, + &identity_pkey,&identity_pkeylen)<0) { + log_warn(LD_BUG,"write identity_pkey to string failed!"); + goto err; + } + + /* Cross-certify with RSA key */ + if (tap_key && router->cache_info.signing_key_cert && + router->cache_info.signing_key_cert->signing_key_included) { + char buf[256]; + int tap_cc_len = 0; + uint8_t *tap_cc = + make_tap_onion_key_crosscert(tap_key, + &router->cache_info.signing_key_cert->signing_key, + router->identity_pkey, + &tap_cc_len); + if (!tap_cc) { + log_warn(LD_BUG,"make_tap_onion_key_crosscert failed!"); + goto err; + } + + if (base64_encode(buf, sizeof(buf), (const char*)tap_cc, tap_cc_len, + BASE64_ENCODE_MULTILINE) < 0) { + log_warn(LD_BUG,"base64_encode(rsa_crosscert) failed!"); + tor_free(tap_cc); + goto err; + } + tor_free(tap_cc); + + tor_asprintf(&rsa_tap_cc_line, + "onion-key-crosscert\n" + "-----BEGIN CROSSCERT-----\n" + "%s" + "-----END CROSSCERT-----\n", buf); + } + + /* Cross-certify with onion keys */ + if (ntor_keypair && router->cache_info.signing_key_cert && + router->cache_info.signing_key_cert->signing_key_included) { + int sign = 0; + char buf[256]; + /* XXXX Base the expiration date on the actual onion key expiration time?*/ + tor_cert_t *cert = + make_ntor_onion_key_crosscert(ntor_keypair, + &router->cache_info.signing_key_cert->signing_key, + router->cache_info.published_on, + get_onion_key_lifetime(), &sign); + if (!cert) { + log_warn(LD_BUG,"make_ntor_onion_key_crosscert failed!"); + goto err; + } + tor_assert(sign == 0 || sign == 1); + + if (base64_encode(buf, sizeof(buf), + (const char*)cert->encoded, cert->encoded_len, + BASE64_ENCODE_MULTILINE)<0) { + log_warn(LD_BUG,"base64_encode(ntor_crosscert) failed!"); + tor_cert_free(cert); + goto err; + } + tor_cert_free(cert); + + tor_asprintf(&ntor_cc_line, + "ntor-onion-key-crosscert %d\n" + "-----BEGIN ED25519 CERT-----\n" + "%s" + "-----END ED25519 CERT-----\n", sign, buf); + } + + /* Encode the publication time. */ + format_iso_time(published, router->cache_info.published_on); + + if (router->declared_family && smartlist_len(router->declared_family)) { + char *family = smartlist_join_strings(router->declared_family, + " ", 0, NULL); + tor_asprintf(&family_line, "family %s\n", family); + tor_free(family); + } else { + family_line = tor_strdup(""); + } + + if (!tor_digest_is_zero(router->cache_info.extra_info_digest)) { + char extra_info_digest[HEX_DIGEST_LEN+1]; + base16_encode(extra_info_digest, sizeof(extra_info_digest), + router->cache_info.extra_info_digest, DIGEST_LEN); + if (!tor_digest256_is_zero(router->cache_info.extra_info_digest256)) { + char d256_64[BASE64_DIGEST256_LEN+1]; + digest256_to_base64(d256_64, router->cache_info.extra_info_digest256); + tor_asprintf(&extra_info_line, "extra-info-digest %s %s\n", + extra_info_digest, d256_64); + } else { + tor_asprintf(&extra_info_line, "extra-info-digest %s\n", + extra_info_digest); + } + } + + if (router->ipv6_orport && + tor_addr_family(&router->ipv6_addr) == AF_INET6) { + char addr[TOR_ADDR_BUF_LEN]; + const char *a; + a = tor_addr_to_str(addr, &router->ipv6_addr, sizeof(addr), 1); + if (a) { + tor_asprintf(&extra_or_address, + "or-address %s:%d\n", a, router->ipv6_orport); + log_debug(LD_OR, "My or-address line is <%s>", extra_or_address); + } + } + + if (router->protocol_list) { + tor_asprintf(&proto_line, "proto %s\n", router->protocol_list); + } else { + proto_line = tor_strdup(""); + } + + address = tor_dup_ip(router->addr); + chunks = smartlist_new(); + + /* Generate the easy portion of the router descriptor. */ + smartlist_add_asprintf(chunks, + "router %s %s %d 0 %d\n" + "%s" + "%s" + "platform %s\n" + "%s" + "published %s\n" + "fingerprint %s\n" + "uptime %ld\n" + "bandwidth %d %d %d\n" + "%s%s" + "onion-key\n%s" + "signing-key\n%s" + "%s%s" + "%s%s%s", + router->nickname, + address, + router->or_port, + router_should_advertise_dirport(options, router->dir_port), + ed_cert_line ? ed_cert_line : "", + extra_or_address ? extra_or_address : "", + router->platform, + proto_line, + published, + fingerprint, + get_uptime(), + (int) router->bandwidthrate, + (int) router->bandwidthburst, + (int) router->bandwidthcapacity, + extra_info_line ? extra_info_line : "", + (options->DownloadExtraInfo || options->V3AuthoritativeDir) ? + "caches-extra-info\n" : "", + onion_pkey, identity_pkey, + rsa_tap_cc_line ? rsa_tap_cc_line : "", + ntor_cc_line ? ntor_cc_line : "", + family_line, + we_are_hibernating() ? "hibernating 1\n" : "", + "hidden-service-dir\n"); + + if (options->ContactInfo && strlen(options->ContactInfo)) { + const char *ci = options->ContactInfo; + if (strchr(ci, '\n') || strchr(ci, '\r')) + ci = escaped(ci); + smartlist_add_asprintf(chunks, "contact %s\n", ci); + } + + if (options->BridgeRelay) { + const char *bd; + if (options->BridgeDistribution && strlen(options->BridgeDistribution)) { + bd = options->BridgeDistribution; + } else { + bd = "any"; + } + if (strchr(bd, '\n') || strchr(bd, '\r')) + bd = escaped(bd); + smartlist_add_asprintf(chunks, "bridge-distribution-request %s\n", bd); + } + + if (router->onion_curve25519_pkey) { + char kbuf[128]; + base64_encode(kbuf, sizeof(kbuf), + (const char *)router->onion_curve25519_pkey->public_key, + CURVE25519_PUBKEY_LEN, BASE64_ENCODE_MULTILINE); + smartlist_add_asprintf(chunks, "ntor-onion-key %s", kbuf); + } else { + /* Authorities will start rejecting relays without ntor keys in 0.2.9 */ + log_err(LD_BUG, "A relay must have an ntor onion key"); + goto err; + } + + /* Write the exit policy to the end of 's'. */ + if (!router->exit_policy || !smartlist_len(router->exit_policy)) { + smartlist_add_strdup(chunks, "reject *:*\n"); + } else if (router->exit_policy) { + char *exit_policy = router_dump_exit_policy_to_string(router,1,0); + + if (!exit_policy) + goto err; + + smartlist_add_asprintf(chunks, "%s\n", exit_policy); + tor_free(exit_policy); + } + + if (router->ipv6_exit_policy) { + char *p6 = write_short_policy(router->ipv6_exit_policy); + if (p6 && strcmp(p6, "reject 1-65535")) { + smartlist_add_asprintf(chunks, + "ipv6-policy %s\n", p6); + } + tor_free(p6); + } + + if (router_should_advertise_begindir(options, + router->supports_tunnelled_dir_requests)) { + smartlist_add_strdup(chunks, "tunnelled-dir-server\n"); + } + + /* Sign the descriptor with Ed25519 */ + if (emit_ed_sigs) { + smartlist_add_strdup(chunks, "router-sig-ed25519 "); + crypto_digest_smartlist_prefix(digest, DIGEST256_LEN, + ED_DESC_SIGNATURE_PREFIX, + chunks, "", DIGEST_SHA256); + ed25519_signature_t sig; + char buf[ED25519_SIG_BASE64_LEN+1]; + if (ed25519_sign(&sig, (const uint8_t*)digest, DIGEST256_LEN, + signing_keypair) < 0) + goto err; + if (ed25519_signature_to_base64(buf, &sig) < 0) + goto err; + + smartlist_add_asprintf(chunks, "%s\n", buf); + } + + /* Sign the descriptor with RSA */ + smartlist_add_strdup(chunks, "router-signature\n"); + + crypto_digest_smartlist(digest, DIGEST_LEN, chunks, "", DIGEST_SHA1); + + { + char *sig; + if (!(sig = router_get_dirobj_signature(digest, DIGEST_LEN, ident_key))) { + log_warn(LD_BUG, "Couldn't sign router descriptor"); + goto err; + } + smartlist_add(chunks, sig); + } + + /* include a last '\n' */ + smartlist_add_strdup(chunks, "\n"); + + output = smartlist_join_strings(chunks, "", 0, NULL); + +#ifdef DEBUG_ROUTER_DUMP_ROUTER_TO_STRING + { + char *s_dup; + const char *cp; + routerinfo_t *ri_tmp; + cp = s_dup = tor_strdup(output); + ri_tmp = router_parse_entry_from_string(cp, NULL, 1, 0, NULL, NULL); + if (!ri_tmp) { + log_err(LD_BUG, + "We just generated a router descriptor we can't parse."); + log_err(LD_BUG, "Descriptor was: <<%s>>", output); + goto err; + } + tor_free(s_dup); + routerinfo_free(ri_tmp); + } +#endif /* defined(DEBUG_ROUTER_DUMP_ROUTER_TO_STRING) */ + + goto done; + + err: + tor_free(output); /* sets output to NULL */ + done: + if (chunks) { + SMARTLIST_FOREACH(chunks, char *, cp, tor_free(cp)); + smartlist_free(chunks); + } + crypto_pk_free(rsa_pubkey); + tor_free(address); + tor_free(family_line); + tor_free(onion_pkey); + tor_free(identity_pkey); + tor_free(extra_or_address); + tor_free(ed_cert_line); + tor_free(rsa_tap_cc_line); + tor_free(ntor_cc_line); + tor_free(extra_info_line); + tor_free(proto_line); + + return output; +} + +/** + * OR only: Given router, produce a string with its exit policy. + * If include_ipv4 is true, include IPv4 entries. + * If include_ipv6 is true, include IPv6 entries. + */ +char * +router_dump_exit_policy_to_string(const routerinfo_t *router, + int include_ipv4, + int include_ipv6) +{ + if ((!router->exit_policy) || (router->policy_is_reject_star)) { + return tor_strdup("reject *:*"); + } + + return policy_dump_to_string(router->exit_policy, + include_ipv4, + include_ipv6); +} + +/** Load the contents of filename, find the last line starting with + * end_line, ensure that its timestamp is not more than 25 hours in + * the past or more than 1 hour in the future with respect to now, + * and write the file contents starting with that line to *out. + * Return 1 for success, 0 if the file does not exist or is empty, or -1 + * if the file does not contain a line matching these criteria or other + * failure. */ +static int +load_stats_file(const char *filename, const char *end_line, time_t now, + char **out) +{ + int r = -1; + char *fname = get_datadir_fname(filename); + char *contents, *start = NULL, *tmp, timestr[ISO_TIME_LEN+1]; + time_t written; + switch (file_status(fname)) { + case FN_FILE: + /* X022 Find an alternative to reading the whole file to memory. */ + if ((contents = read_file_to_str(fname, 0, NULL))) { + tmp = strstr(contents, end_line); + /* Find last block starting with end_line */ + while (tmp) { + start = tmp; + tmp = strstr(tmp + 1, end_line); + } + if (!start) + goto notfound; + if (strlen(start) < strlen(end_line) + 1 + sizeof(timestr)) + goto notfound; + strlcpy(timestr, start + 1 + strlen(end_line), sizeof(timestr)); + if (parse_iso_time(timestr, &written) < 0) + goto notfound; + if (written < now - (25*60*60) || written > now + (1*60*60)) + goto notfound; + *out = tor_strdup(start); + r = 1; + } + notfound: + tor_free(contents); + break; + /* treat empty stats files as if the file doesn't exist */ + case FN_NOENT: + case FN_EMPTY: + r = 0; + break; + case FN_ERROR: + case FN_DIR: + default: + break; + } + tor_free(fname); + return r; +} + +/** Write the contents of extrainfo and aggregated statistics to + * *s_out, signing them with ident_key. Return 0 on + * success, negative on failure. */ +int +extrainfo_dump_to_string(char **s_out, extrainfo_t *extrainfo, + crypto_pk_t *ident_key, + const ed25519_keypair_t *signing_keypair) +{ + const or_options_t *options = get_options(); + char identity[HEX_DIGEST_LEN+1]; + char published[ISO_TIME_LEN+1]; + char digest[DIGEST_LEN]; + char *bandwidth_usage; + int result; + static int write_stats_to_extrainfo = 1; + char sig[DIROBJ_MAX_SIG_LEN+1]; + char *s = NULL, *pre, *contents, *cp, *s_dup = NULL; + time_t now = time(NULL); + smartlist_t *chunks = smartlist_new(); + extrainfo_t *ei_tmp = NULL; + const int emit_ed_sigs = signing_keypair && + extrainfo->cache_info.signing_key_cert; + char *ed_cert_line = NULL; + + base16_encode(identity, sizeof(identity), + extrainfo->cache_info.identity_digest, DIGEST_LEN); + format_iso_time(published, extrainfo->cache_info.published_on); + bandwidth_usage = rep_hist_get_bandwidth_lines(); + if (emit_ed_sigs) { + if (!extrainfo->cache_info.signing_key_cert->signing_key_included || + !ed25519_pubkey_eq(&extrainfo->cache_info.signing_key_cert->signed_key, + &signing_keypair->pubkey)) { + log_warn(LD_BUG, "Tried to sign a extrainfo descriptor with a " + "mismatched ed25519 key chain %d", + extrainfo->cache_info.signing_key_cert->signing_key_included); + goto err; + } + char ed_cert_base64[256]; + if (base64_encode(ed_cert_base64, sizeof(ed_cert_base64), + (const char*)extrainfo->cache_info.signing_key_cert->encoded, + extrainfo->cache_info.signing_key_cert->encoded_len, + BASE64_ENCODE_MULTILINE) < 0) { + log_err(LD_BUG,"Couldn't base64-encode signing key certificate!"); + goto err; + } + tor_asprintf(&ed_cert_line, "identity-ed25519\n" + "-----BEGIN ED25519 CERT-----\n" + "%s" + "-----END ED25519 CERT-----\n", ed_cert_base64); + } else { + ed_cert_line = tor_strdup(""); + } + + tor_asprintf(&pre, "extra-info %s %s\n%spublished %s\n%s", + extrainfo->nickname, identity, + ed_cert_line, + published, bandwidth_usage); + smartlist_add(chunks, pre); + + if (geoip_is_loaded(AF_INET)) + smartlist_add_asprintf(chunks, "geoip-db-digest %s\n", + geoip_db_digest(AF_INET)); + if (geoip_is_loaded(AF_INET6)) + smartlist_add_asprintf(chunks, "geoip6-db-digest %s\n", + geoip_db_digest(AF_INET6)); + + if (options->ExtraInfoStatistics && write_stats_to_extrainfo) { + log_info(LD_GENERAL, "Adding stats to extra-info descriptor."); + if (options->DirReqStatistics && + load_stats_file("stats"PATH_SEPARATOR"dirreq-stats", + "dirreq-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + if (options->HiddenServiceStatistics && + load_stats_file("stats"PATH_SEPARATOR"hidserv-stats", + "hidserv-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + if (options->EntryStatistics && + load_stats_file("stats"PATH_SEPARATOR"entry-stats", + "entry-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + if (options->CellStatistics && + load_stats_file("stats"PATH_SEPARATOR"buffer-stats", + "cell-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + if (options->ExitPortStatistics && + load_stats_file("stats"PATH_SEPARATOR"exit-stats", + "exit-stats-end", now, &contents) > 0) { + smartlist_add(chunks, contents); + } + if (options->ConnDirectionStatistics && + load_stats_file("stats"PATH_SEPARATOR"conn-stats", + "conn-bi-direct", now, &contents) > 0) { + smartlist_add(chunks, contents); + } - } - - if (options->PaddingStatistics) { - contents = rep_hist_get_padding_count_lines(); - if (contents) - smartlist_add(chunks, contents); ++ if (options->PaddingStatistics) { ++ contents = rep_hist_get_padding_count_lines(); ++ if (contents) ++ smartlist_add(chunks, contents); ++ } + } + + /* Add information about the pluggable transports we support. */ + if (options->ServerTransportPlugin) { + char *pluggable_transports = pt_get_extra_info_descriptor_string(); + if (pluggable_transports) + smartlist_add(chunks, pluggable_transports); + } + + if (should_record_bridge_info(options) && write_stats_to_extrainfo) { + const char *bridge_stats = geoip_get_bridge_stats_extrainfo(now); + if (bridge_stats) { + smartlist_add_strdup(chunks, bridge_stats); + } + } + + if (emit_ed_sigs) { + char sha256_digest[DIGEST256_LEN]; + smartlist_add_strdup(chunks, "router-sig-ed25519 "); + crypto_digest_smartlist_prefix(sha256_digest, DIGEST256_LEN, + ED_DESC_SIGNATURE_PREFIX, + chunks, "", DIGEST_SHA256); + ed25519_signature_t ed_sig; + char buf[ED25519_SIG_BASE64_LEN+1]; + if (ed25519_sign(&ed_sig, (const uint8_t*)sha256_digest, DIGEST256_LEN, + signing_keypair) < 0) + goto err; + if (ed25519_signature_to_base64(buf, &ed_sig) < 0) + goto err; + + smartlist_add_asprintf(chunks, "%s\n", buf); + } + + smartlist_add_strdup(chunks, "router-signature\n"); + s = smartlist_join_strings(chunks, "", 0, NULL); + + while (strlen(s) > MAX_EXTRAINFO_UPLOAD_SIZE - DIROBJ_MAX_SIG_LEN) { + /* So long as there are at least two chunks (one for the initial + * extra-info line and one for the router-signature), we can keep removing + * things. */ + if (smartlist_len(chunks) > 2) { + /* We remove the next-to-last element (remember, len-1 is the last + element), since we need to keep the router-signature element. */ + int idx = smartlist_len(chunks) - 2; + char *e = smartlist_get(chunks, idx); + smartlist_del_keeporder(chunks, idx); + log_warn(LD_GENERAL, "We just generated an extra-info descriptor " + "with statistics that exceeds the 50 KB " + "upload limit. Removing last added " + "statistics."); + tor_free(e); + tor_free(s); + s = smartlist_join_strings(chunks, "", 0, NULL); + } else { + log_warn(LD_BUG, "We just generated an extra-info descriptors that " + "exceeds the 50 KB upload limit."); + goto err; + } + } + + memset(sig, 0, sizeof(sig)); + if (router_get_extrainfo_hash(s, strlen(s), digest) < 0 || + router_append_dirobj_signature(sig, sizeof(sig), digest, DIGEST_LEN, + ident_key) < 0) { + log_warn(LD_BUG, "Could not append signature to extra-info " + "descriptor."); + goto err; + } + smartlist_add_strdup(chunks, sig); + tor_free(s); + s = smartlist_join_strings(chunks, "", 0, NULL); + + cp = s_dup = tor_strdup(s); + ei_tmp = extrainfo_parse_entry_from_string(cp, NULL, 1, NULL, NULL); + if (!ei_tmp) { + if (write_stats_to_extrainfo) { + log_warn(LD_GENERAL, "We just generated an extra-info descriptor " + "with statistics that we can't parse. Not " + "adding statistics to this or any future " + "extra-info descriptors."); + write_stats_to_extrainfo = 0; + result = extrainfo_dump_to_string(s_out, extrainfo, ident_key, + signing_keypair); + goto done; + } else { + log_warn(LD_BUG, "We just generated an extrainfo descriptor we " + "can't parse."); + goto err; + } + } + + *s_out = s; + s = NULL; /* prevent free */ + result = 0; + goto done; + + err: + result = -1; + + done: + tor_free(s); + SMARTLIST_FOREACH(chunks, char *, chunk, tor_free(chunk)); + smartlist_free(chunks); + tor_free(s_dup); + tor_free(ed_cert_line); + extrainfo_free(ei_tmp); + tor_free(bandwidth_usage); + + return result; +} + +/** Forget that we have issued any router-related warnings, so that we'll + * warn again if we see the same errors. */ +void +router_reset_warnings(void) +{ + if (warned_nonexistent_family) { + SMARTLIST_FOREACH(warned_nonexistent_family, char *, cp, tor_free(cp)); + smartlist_clear(warned_nonexistent_family); + } +} + +/** Release all static resources held in router.c */ +void +router_free_all(void) +{ + crypto_pk_free(onionkey); + crypto_pk_free(lastonionkey); + crypto_pk_free(server_identitykey); + crypto_pk_free(client_identitykey); + + tor_mutex_free(key_lock); + routerinfo_free(desc_routerinfo); + extrainfo_free(desc_extrainfo); + crypto_pk_free(authority_signing_key); + authority_cert_free(authority_key_certificate); + crypto_pk_free(legacy_signing_key); + authority_cert_free(legacy_key_certificate); + + memwipe(&curve25519_onion_key, 0, sizeof(curve25519_onion_key)); + memwipe(&last_curve25519_onion_key, 0, sizeof(last_curve25519_onion_key)); + + if (warned_nonexistent_family) { + SMARTLIST_FOREACH(warned_nonexistent_family, char *, cp, tor_free(cp)); + smartlist_free(warned_nonexistent_family); + } +} +/* From the given RSA key object, convert it to ASN-1 encoded format and set + * the newly allocated object in onion_pkey_out. The length of the key is set + * in onion_pkey_len_out. */ +void +router_set_rsa_onion_pkey(const crypto_pk_t *pk, char **onion_pkey_out, + size_t *onion_pkey_len_out) +{ + int len; + char buf[1024]; + + tor_assert(pk); + tor_assert(onion_pkey_out); + tor_assert(onion_pkey_len_out); + + len = crypto_pk_asn1_encode(pk, buf, sizeof(buf)); + if (BUG(len < 0)) { + goto done; + } + + *onion_pkey_out = tor_memdup(buf, len); + *onion_pkey_len_out = len; + + done: + return; +} + +/* From an ASN-1 encoded onion pkey, return a newly allocated RSA key object. + * It is the caller responsability to free the returned object. + * + * Return NULL if the pkey is NULL, malformed or if the length is 0. */ +crypto_pk_t * +router_get_rsa_onion_pkey(const char *pkey, size_t pkey_len) +{ + if (!pkey || pkey_len == 0) { + return NULL; + } + return crypto_pk_asn1_decode(pkey, pkey_len); +} [View Less]

1 0

[translation/support-portal_completed] Update translations for support-portal_completed
by translation＠torproject.org 19 Apr '19

19 Apr '19

commit 982425971b96d6e0c0a921d4b5ab07f841f1e391 Author: Translation commit bot <translation(a)torproject.org> Date: Fri Apr 19 01:50:48 2019 +0000 Update translations for support-portal_completed --- contents+pt-BR.po | 2774 +++++++++++++++++++++++++++-------------------------- 1 file changed, 1430 insertions(+), 1344 deletions(-) diff --git a/contents+pt-BR.po b/contents+pt-BR.po index 1e849cd06..c1b9f905c 100644 --- a/contents+pt-BR.po +++ b/contents+pt-BR.po @@ -1,23 +1,24 @@ … [View More]

1 0

[translation/support-portal] Update translations for support-portal
by translation＠torproject.org 19 Apr '19

19 Apr '19

commit 927a1c6fff92af65c0496a4a2238dc9c4c74b49a Author: Translation commit bot <translation(a)torproject.org> Date: Fri Apr 19 01:50:40 2019 +0000 Update translations for support-portal --- contents+pt-BR.po | 52 +++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 41 insertions(+), 11 deletions(-) diff --git a/contents+pt-BR.po b/contents+pt-BR.po index 76b46f741..c1b9f905c 100644 --- a/contents+pt-BR.po +++ b/contents+pt-BR.po @@ -8,9 +8,9 @@ # Alexei Gonç… [View More]alves de Oliveira <alexis(a)gessicahellmann.com>, 2019 # Greg Strider <gboufleur(a)gmail.com>, 2019 # Emma Peel, 2019 -# Chacal Exodius, 2019 # Communia <ameaneantie(a)riseup.net>, 2019 # Eduardo Addad de Oliveira <eduardoaddad(a)hotmail.com>, 2019 +# Chacal E., 2019 # msgid "" msgstr "" @@ -18,7 +18,7 @@ msgstr "" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2019-04-15 12:30+CET\n" "PO-Revision-Date: 2018-10-02 22:41+0000\n" -"Last-Translator: Eduardo Addad de Oliveira <eduardoaddad(a)hotmail.com>, 2019\n" +"Last-Translator: Chacal E., 2019\n" "Language-Team: Portuguese (Brazil) (https://www.transifex.com/otf/teams/1519/pt_BR/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -346,6 +346,10 @@ msgid "" "accompanied by a file with the same name as the package and the extension " "\".asc\". These .asc files are GPG signatures." msgstr "" +"Cada arquivo em nossa [página de download] " +"(https://www.torproject.org/download/) é acompanhado por um arquivo com o " +"mesmo nome do pacote e a extensão \".asc\". Esses arquivos .asc são " +"assinaturas GPG." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -455,12 +459,14 @@ msgstr "A equipe do Tor Browser assina os lançamentos do Tor Browser." #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "Import its key (0x4E2C6E8793298290) by starting cmd.exe and typing:" msgstr "" +"Importe sua chave (0x4E2C6E8793298290) iniciando o cmd.exe e digitando:" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "" "gpg.exe --keyserver pool.sks-keyservers.net --recv-keys 0x4E2C6E8793298290" msgstr "" +"gpg.exe --keyserver pool.sks-keyservers.net --recv-keys 0x4E2C6E8793298290" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -496,6 +502,8 @@ msgid "" "uid [ unknown] Tor Browser Developers (signing key) " "<torbrowser&at;torproject.org>" msgstr "" +"uid [ desconhecido] Tor Browser Developers (chave de " +"assinatura) <torbrowser&at;torproject.org>" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -505,7 +513,7 @@ msgstr "sub rsa4096/0xEB774491D9FF06E2 2018-05-26 [S] [expires: 2020-09-12]" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "Key fingerprint = 1107 75B5 D101 FB36 BC6C 911B EB77 4491 D9FF 06E2" -msgstr "" +msgstr "Key fingerprint = 1107 75B5 D101 FB36 BC6C 911B EB77 4491 D9FF 06E2" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -521,6 +529,8 @@ msgstr "" msgid "" "Assuming you downloaded the package and its signature to your Desktop, run:" msgstr "" +"Supondo que você baixou o pacote e sua assinatura para o seu Desktop, " +"execute:" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -529,6 +539,9 @@ msgid "" ".8_en-US.exe.asc C:\\Users\\Alice\\Desktop\\torbrowser-install-8.0.8_en-" "US.exe" msgstr "" +"gpg.exe --verify C:\\Users\\Alice\\Desktop\\torbrowser-install-win64-8.0" +".8_en-US.exe.asc C:\\Users\\Alice\\Desktop\\torbrowser-install-8.0.8_en-" +"US.exe" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -538,12 +551,12 @@ msgstr "Por favor, substitua \"Alice\" com o seu nome de usuário." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "The output should say \"Good signature\":" -msgstr "" +msgstr "A saída deve dizer \"Good signature\":" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "gpg: Signature made Tue 12 Feb 2019 08:27:41 AM EST" -msgstr "" +msgstr "gpg: Signature made Tue 12 Feb 2019 08:27:41 AM EST" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -556,6 +569,8 @@ msgid "" "gpg: Good signature from \"Tor Browser Developers (signing key) " "<torbrowser(a)torproject.org>\" [unknown]" msgstr "" +"gpg: boa assinatura de \"Tor Browser Developers (chave de assinatura) " +"<torbrowser(a)torproject.org>\" [desconhecido]" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -569,22 +584,24 @@ msgid "" "gpg: There is no indication that the signature belongs to the " "owner." msgstr "" +"gpg: Não há indicação de que a assinatura pertença ao proprietário." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "" "Primary key fingerprint: EF6E 286D DA85 EA2A 4BA7 DE68 4E2C 6E87 9329 8290" msgstr "" +"Primary key fingerprint: EF6E 286D DA85 EA2A 4BA7 DE68 4E2C 6E87 9329 8290" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "Subkey fingerprint: 1107 75B5 D101 FB36 BC6C 911B EB77 4491 D9FF 06E2" -msgstr "" +msgstr "Subkey fingerprint: 1107 75B5 D101 FB36 BC6C 911B EB77 4491 D9FF 06E2" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "Currently valid subkey fingerprints are:" -msgstr "" +msgstr "As impressões digitais de subchave atualmente válidas são:" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -606,6 +623,8 @@ msgid "" "This means that GnuPG verified that the key made that signature, but it's up" " to you to decide if that key really belongs to the developer." msgstr "" +"Isso significa que o GnuPG verificou que a chave fez essa assinatura, mas " +"cabe a você decidir se essa chave realmente pertence ao desenvolvedor." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -632,6 +651,8 @@ msgid "" "If you are using Mac OS X, you can install it from " "https://www.gpgtools.org/." msgstr "" +"Se você estiver usando o Mac OS X, você pode instalá-lo em " +"https://www.gpgtools.org/." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -639,6 +660,8 @@ msgid "" "If you are using Linux, then you probably already have GnuPG in your system," " as most Linux distributions come with it preinstalled." msgstr "" +"Se você está usando Linux, então provavelmente você já tem o GnuPG em seu " +"sistema, já que a maioria das distribuições Linux vêm pré-instaladas." #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -654,11 +677,14 @@ msgid "" "Import its key (0x4E2C6E8793298290) by starting the terminal (under " "\"Applications\" in Mac OS X) and typing:" msgstr "" +"Importe sua chave (0x4E2C6E8793298290) iniciando o terminal (em " +"\"Aplicativos\" no Mac OS X) e digitando:" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "gpg --keyserver pool.sks-keyservers.net --recv-keys 0x4E2C6E8793298290" msgstr "" +"gpg --keyserver pool.sks-keyservers.net --recv-keys 0x4E2C6E8793298290" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -671,6 +697,8 @@ msgid "" "Assuming you downloaded the package and its signature to your Downloads " "folder, run:" msgstr "" +"Supondo que você baixou o pacote e sua assinatura para sua pasta Downloads, " +"execute:" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) @@ -680,27 +708,29 @@ msgstr "#### Para quem usa Mac OS X:" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "gpg --verify ~/Downloads/TorBrowser-8.0.8-osx64_en-US.dmg{.asc,}" -msgstr "" +msgstr "gpg --verify ~/Downloads/TorBrowser-8.0.8-osx64_en-US.dmg{.asc,}" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "#### For Linux users (change 64 to 32 if you have the 32-bit package):" msgstr "" +"#### Para usuários do Linux (altere 64 para 32 se você tiver o pacote de 32 " +"bits):" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "gpg --verify tor-browser-linux64-8.0.8_en-US.tar.xz{.asc,}" -msgstr "" +msgstr "gpg --verify tor-browser-linux64-8.0.8_en-US.tar.xz{.asc,}" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "gpg: Signature made Wed 15 Nov 2017 05:52:38 PM CET" -msgstr "" +msgstr "gpg: Signature made Wed 15 Nov 2017 05:52:38 PM CET" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) msgid "gpg: using RSA key 0xD1483FA6C3C07136" -msgstr "" +msgstr "gpg: usando a chave RSA 0xD1483FA6C3C07136" #: https//support.torproject.org/tbb/how-to-verify-signature/ #: (content/tbb/how-to-verify-signature/contents+en.lrquestion.description) [View Less]

1 0

[tor/master] practracker: accept more lines in microdescs_parse_from_string()
by teor＠torproject.org 19 Apr '19

19 Apr '19

commit bffba9d26f62d934c686576424a79eab953bd490 Author: teor <teor(a)torproject.org> Date: Fri Apr 19 10:34:16 2019 +1000 practracker: accept more lines in microdescs_parse_from_string() Part of 28223. --- scripts/maint/practracker/exceptions.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/maint/practracker/exceptions.txt b/scripts/maint/practracker/exceptions.txt index ecc58012c..fefd6c3a7 100644 --- a/scripts/maint/practracker/exceptions.… [View More]

1 0

[translation/support-portal] Update translations for support-portal
by translation＠torproject.org 18 Apr '19

18 Apr '19

commit 73b805516e2dbf5fe9920462039d8aadc90e741e Author: Translation commit bot <translation(a)torproject.org> Date: Thu Apr 18 23:50:48 2019 +0000 Update translations for support-portal --- contents+zh-CN.po | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/contents+zh-CN.po b/contents+zh-CN.po index 6ed9248f0..ab5d89adc 100644 --- a/contents+zh-CN.po +++ b/contents+zh-CN.po @@ -2,9 +2,9 @@ # Herman Koe <hkoe.academic(a)gmail.com>, 2018 # MD Rights … [View More]

1 0

[translation/tpo-web] Update translations for tpo-web
by translation＠torproject.org 18 Apr '19

18 Apr '19

commit b3f39c721373e8dd2972ac9549fb1b06d045bfb5 Author: Translation commit bot <translation(a)torproject.org> Date: Thu Apr 18 23:50:26 2019 +0000 Update translations for tpo-web --- contents+zh-CN.po | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/contents+zh-CN.po b/contents+zh-CN.po index 3b2eb47dd..0c492d6f7 100644 --- a/contents+zh-CN.po +++ b/contents+zh-CN.po @@ -2,10 +2,10 @@ # Chi-Hsun Tsai, 2019 # Xiaoyu Huang <007pig(a)gmail.com>, 2019 … [View More]

1 0