July 2014 - tor-commits - lists.torproject.org

[tor/master] Fix a bug where streams would linger forever when we had no dirinfo
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit ed3d7892c721c9495215ecad2e18c026d29fbb9b Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 9 16:14:14 2014 -0400 Fix a bug where streams would linger forever when we had no dirinfo fixes bug 8387; fix on 0.1.1.11-alpha (code), or on 0.2.4.10-alpha (behavior). --- changes/bug8387 | 11 +++++++++++ src/or/circuituse.c | 27 ++++++++++++++++++++++----- src/or/circuituse.h | 1 + src/or/main.c | 5 ++++- 4 files changed, 38 insertions(+), 6 deletions(-) diff --git a/changes/bug8387 b/changes/bug8387 new file mode 100644 index 0000000..2ec0487 --- /dev/null +++ b/changes/bug8387 @@ -0,0 +1,11 @@ + o Major bugfixes (client): + + - Perform circuit cleanup operations even when circuit + construction operations are disabled (because the network is + disabled, or because there isn't enough directory information). + Previously, when we were not building predictive circuits, we + were not closing expired circuits either. + + Fixes bug 8387; bugfix on 0.1.1.11-alpha. This bug became visible + in 0.2.4.10-alpha when we became more strict about when we have + "enough directory information to build circuits". diff --git a/src/or/circuituse.c b/src/or/circuituse.c index c2d2b2e..9933b9c 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -969,7 +969,6 @@ circuit_predict_and_launch_new(void) void circuit_build_needed_circs(time_t now) { - static time_t time_to_new_circuit = 0; const or_options_t *options = get_options(); /* launch a new circ for any pending streams that need one */ @@ -978,14 +977,34 @@ circuit_build_needed_circs(time_t now) /* make sure any hidden services have enough intro points */ rend_services_introduce(); - if (time_to_new_circuit < now) { + circuit_expire_old_circs_as_needed(now); + + if (!options->DisablePredictedCircuits) + circuit_predict_and_launch_new(); +} + +/** + * Called once a second either directly or from + * circuit_build_needed_circs(). As appropriate (once per NewCircuitPeriod) + * resets failure counts and expires old circuits. + */ +void +circuit_expire_old_circs_as_needed(time_t now) +{ + static time_t time_to_expire_and_reset = 0; + + if (time_to_expire_and_reset < now) { circuit_reset_failure_count(1); - time_to_new_circuit = now + options->NewCircuitPeriod; + time_to_expire_and_reset = now + get_options()->NewCircuitPeriod; if (proxy_mode(get_options())) addressmap_clean(now); circuit_expire_old_circuits_clientside(); #if 0 /* disable for now, until predict-and-launch-new can cull leftovers */ + + /* If we ever re-enable, this has to move into + * circuit_build_needed_circs */ + circ = circuit_get_youngest_clean_open(CIRCUIT_PURPOSE_C_GENERAL); if (get_options()->RunTesting && circ && @@ -995,8 +1014,6 @@ circuit_build_needed_circs(time_t now) } #endif } - if (!options->DisablePredictedCircuits) - circuit_predict_and_launch_new(); } /** If the stream conn is a member of any of the linked diff --git a/src/or/circuituse.h b/src/or/circuituse.h index 11e5a64..1559a40 100644 --- a/src/or/circuituse.h +++ b/src/or/circuituse.h @@ -21,6 +21,7 @@ int circuit_conforms_to_options(const origin_circuit_t *circ, const or_options_t *options); #endif void circuit_build_needed_circs(time_t now); +void circuit_expire_old_circs_as_needed(time_t now); void circuit_detach_stream(circuit_t *circ, edge_connection_t *conn); void circuit_expire_old_circuits_serverside(time_t now); diff --git a/src/or/main.c b/src/or/main.c index bd23141..495a71f 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -1485,8 +1485,11 @@ run_scheduled_events(time_t now) * and we make a new circ if there are no clean circuits. */ have_dir_info = router_have_minimum_dir_info(); - if (have_dir_info && !net_is_disabled()) + if (have_dir_info && !net_is_disabled()) { circuit_build_needed_circs(now); + } else { + circuit_expire_old_circs_as_needed(now); + } /* every 10 seconds, but not at the same second as other such events */ if (now % 10 == 5)

1 0

[tor/master] Remove executable bit from control.c, router.c
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 35791f4238bd6853c038d5fc68976dd74c93c577 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 9 08:54:08 2014 -0400 Remove executable bit from control.c, router.c Fix for 12512. --- 0 files changed diff --git a/src/or/control.c b/src/or/control.c old mode 100755 new mode 100644 diff --git a/src/or/router.c b/src/or/router.c old mode 100755 new mode 100644

1 0

[tor/master] Add another 8387 diagnostic
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 32495ee3099c3fb512a986a8e5db32ebf153b439 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 10:05:00 2014 +0200 Add another 8387 diagnostic When we run into bug 8387 (if we run into it again), report when we last called circuit_expire_old_circuits_clientside(). This will let us know -- if my fix for 8387 doesn't work -- whether my diagnosis was at least correct. Suggested by Andrea. --- src/or/circuituse.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/or/circuituse.c b/src/or/circuituse.c index 600aede..714754a 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -783,6 +783,10 @@ circuit_expire_building(void) } } +/** For debugging #8387: track when we last called + * circuit_expire_old_circuits_clientside. */ +static time_t last_expired_clientside_circuits = 0; + /** * As a diagnostic for bug 8387, log information about how many one-hop * circuits we have around that have been there for at least age @@ -894,6 +898,10 @@ circuit_log_ancient_one_hop_circuits(int age) } } SMARTLIST_FOREACH_END(ocirc); + log_notice(LD_HEARTBEAT, "It has been %ld seconds since I last called " + "circuit_expire_old_circuits_clientside().", + (long)(now - last_expired_clientside_circuits)); + done: smartlist_free(log_these); } @@ -1220,6 +1228,7 @@ circuit_expire_old_circuits_clientside(void) tor_gettimeofday(&now); cutoff = now; + last_expired_clientside_circuits = now.tv_sec; if (! circuit_build_times_disabled() && circuit_build_times_needs_circuits(get_circuit_build_times())) {

1 0

[tor/maint-0.2.5] configure.ac: fix build with --enable-bufferevents
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 589de5ec20bf6f15813fc3bddb7a814309184800 Author: Anthony G. Basile <blueness(a)gentoo.org> Date: Fri Jun 20 08:47:26 2014 -0400 configure.ac: fix build with --enable-bufferevents When building with bufferevents enabled, configure.ac throws an error if "$ac_cv_header_event2_bufferevent_ssl_h" is not set to "yes". However, nowhere was AC_CHECK_HEADERS(event2/bufferevent_ssl.h) done. This commit adds the check. --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 38d9222..1d8cfa7 100644 --- a/configure.ac +++ b/configure.ac @@ -454,7 +454,7 @@ AC_CHECK_MEMBERS([struct event.min_heap_idx], , , [#include <event.h> ]) -AC_CHECK_HEADERS(event2/event.h event2/dns.h) +AC_CHECK_HEADERS(event2/event.h event2/dns.h event2/bufferevent_ssl.h) LIBS="$save_LIBS" LDFLAGS="$save_LDFLAGS"

1 0

[tor/maint-0.2.5] src/or/connection.c: expose bucket_millis_empty for bufferevents test
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit d504a4e36f571a5b6ed84529d3781d08df82f117 Author: Anthony G. Basile <blueness(a)gentoo.org> Date: Thu Jun 26 11:15:36 2014 -0400 src/or/connection.c: expose bucket_millis_empty for bufferevents test Currently tor fails to build its test when enabled with bufferevents because an #ifndef USE_BUFFEREVENTS hides bucket_millis_empty() and friends. This is fine if we don't run tests, but if we do, we need these functions in src/or/libtor-testing.a when linking src/test/test. This patch moves the functions outside the #ifndef and exposes them. See downstream bug: https://bugs.gentoo.org/show_bug.cgi?id=510124 --- src/or/connection.c | 60 +++++++++++++++++++++++++-------------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/src/or/connection.c b/src/or/connection.c index 0b03092..4788bdf 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -2650,14 +2650,6 @@ record_num_bytes_transferred(connection_t *conn, } #endif -#ifndef USE_BUFFEREVENTS -/** Last time at which the global or relay buckets were emptied in msec - * since midnight. */ -static uint32_t global_relayed_read_emptied = 0, - global_relayed_write_emptied = 0, - global_read_emptied = 0, - global_write_emptied = 0; - /** Helper: convert given tvnow time value to milliseconds since * midnight. */ static uint32_t @@ -2667,6 +2659,28 @@ msec_since_midnight(const struct timeval *tvnow) ((uint32_t)tvnow->tv_usec / (uint32_t)1000L)); } +/** Helper: return the time in milliseconds since last_empty_time + * when a bucket ran empty that previously had tokens_before tokens + * now has tokens_after tokens after refilling at timestamp + * tvnow, capped at milliseconds_elapsed milliseconds since + * last refilling that bucket. Return 0 if the bucket has not been empty + * since the last refill or has not been refilled. */ +uint32_t +bucket_millis_empty(int tokens_before, uint32_t last_empty_time, + int tokens_after, int milliseconds_elapsed, + const struct timeval *tvnow) +{ + uint32_t result = 0, refilled; + if (tokens_before <= 0 && tokens_after > tokens_before) { + refilled = msec_since_midnight(tvnow); + result = (uint32_t)((refilled + 86400L * 1000L - last_empty_time) % + (86400L * 1000L)); + if (result > (uint32_t)milliseconds_elapsed) + result = (uint32_t)milliseconds_elapsed; + } + return result; +} + /** Check if a bucket which had tokens_before tokens and which got * tokens_removed tokens removed at timestamp tvnow has run * out of tokens, and if so, note the milliseconds since midnight in @@ -2680,6 +2694,14 @@ connection_buckets_note_empty_ts(uint32_t *timestamp_var, *timestamp_var = msec_since_midnight(tvnow); } +#ifndef USE_BUFFEREVENTS +/** Last time at which the global or relay buckets were emptied in msec + * since midnight. */ +static uint32_t global_relayed_read_emptied = 0, + global_relayed_write_emptied = 0, + global_read_emptied = 0, + global_write_emptied = 0; + /** We just read num_read and wrote num_written bytes * onto conn. Decrement buckets appropriately. */ static void @@ -2838,28 +2860,6 @@ connection_bucket_refill_helper(int *bucket, int rate, int burst, } } -/** Helper: return the time in milliseconds since last_empty_time - * when a bucket ran empty that previously had tokens_before tokens - * now has tokens_after tokens after refilling at timestamp - * tvnow, capped at milliseconds_elapsed milliseconds since - * last refilling that bucket. Return 0 if the bucket has not been empty - * since the last refill or has not been refilled. */ -uint32_t -bucket_millis_empty(int tokens_before, uint32_t last_empty_time, - int tokens_after, int milliseconds_elapsed, - const struct timeval *tvnow) -{ - uint32_t result = 0, refilled; - if (tokens_before <= 0 && tokens_after > tokens_before) { - refilled = msec_since_midnight(tvnow); - result = (uint32_t)((refilled + 86400L * 1000L - last_empty_time) % - (86400L * 1000L)); - if (result > (uint32_t)milliseconds_elapsed) - result = (uint32_t)milliseconds_elapsed; - } - return result; -} - /** Time has passed; increment buckets appropriately. */ void connection_bucket_refill(int milliseconds_elapsed, time_t now)

1 0

[tor/maint-0.2.5] changes file for 12474, 12438.
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 9e46855538e9b848b7c37fda502e5d109079cee0 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 11:00:49 2014 +0200 changes file for 12474, 12438. --- changes/bufferevent_compilation | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/changes/bufferevent_compilation b/changes/bufferevent_compilation new file mode 100644 index 0000000..0690e65 --- /dev/null +++ b/changes/bufferevent_compilation @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix compilation when building with bufferevents enabled. (This + configuration is still not expected to work, however.) + Fixes bugs 12438, 12474; bugfixes on 0.2.5.1-alpha. Patches + from Anthony G. Basile.

1 0

[tor/maint-0.2.5] diagnostic for 12184: Add a call to channel_dump_statistics
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 5d2045ee8ba42a294291b187234be6ead6864cab Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 10:34:39 2014 +0200 diagnostic for 12184: Add a call to channel_dump_statistics --- changes/further-12184-diagnostic | 2 ++ src/or/circuitbuild.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/changes/further-12184-diagnostic b/changes/further-12184-diagnostic new file mode 100644 index 0000000..89e9f46 --- /dev/null +++ b/changes/further-12184-diagnostic @@ -0,0 +1,2 @@ + o Minor features (diagnostic): + - Slightly enhance the diagnostic message for bug 12184. diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index e743a25..283afee 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -183,6 +183,8 @@ get_unique_circ_id_by_chan(channel_t *chan) if (0) circuitmux_assert_okay(chan->cmux); + channel_dump_statistics(chan, LOG_WARN); + return 0; }

1 0

[tor/maint-0.2.5] Fix a bug where streams would linger forever when we had no dirinfo
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit ed3d7892c721c9495215ecad2e18c026d29fbb9b Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 9 16:14:14 2014 -0400 Fix a bug where streams would linger forever when we had no dirinfo fixes bug 8387; fix on 0.1.1.11-alpha (code), or on 0.2.4.10-alpha (behavior). --- changes/bug8387 | 11 +++++++++++ src/or/circuituse.c | 27 ++++++++++++++++++++++----- src/or/circuituse.h | 1 + src/or/main.c | 5 ++++- 4 files changed, 38 insertions(+), 6 deletions(-) diff --git a/changes/bug8387 b/changes/bug8387 new file mode 100644 index 0000000..2ec0487 --- /dev/null +++ b/changes/bug8387 @@ -0,0 +1,11 @@ + o Major bugfixes (client): + + - Perform circuit cleanup operations even when circuit + construction operations are disabled (because the network is + disabled, or because there isn't enough directory information). + Previously, when we were not building predictive circuits, we + were not closing expired circuits either. + + Fixes bug 8387; bugfix on 0.1.1.11-alpha. This bug became visible + in 0.2.4.10-alpha when we became more strict about when we have + "enough directory information to build circuits". diff --git a/src/or/circuituse.c b/src/or/circuituse.c index c2d2b2e..9933b9c 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -969,7 +969,6 @@ circuit_predict_and_launch_new(void) void circuit_build_needed_circs(time_t now) { - static time_t time_to_new_circuit = 0; const or_options_t *options = get_options(); /* launch a new circ for any pending streams that need one */ @@ -978,14 +977,34 @@ circuit_build_needed_circs(time_t now) /* make sure any hidden services have enough intro points */ rend_services_introduce(); - if (time_to_new_circuit < now) { + circuit_expire_old_circs_as_needed(now); + + if (!options->DisablePredictedCircuits) + circuit_predict_and_launch_new(); +} + +/** + * Called once a second either directly or from + * circuit_build_needed_circs(). As appropriate (once per NewCircuitPeriod) + * resets failure counts and expires old circuits. + */ +void +circuit_expire_old_circs_as_needed(time_t now) +{ + static time_t time_to_expire_and_reset = 0; + + if (time_to_expire_and_reset < now) { circuit_reset_failure_count(1); - time_to_new_circuit = now + options->NewCircuitPeriod; + time_to_expire_and_reset = now + get_options()->NewCircuitPeriod; if (proxy_mode(get_options())) addressmap_clean(now); circuit_expire_old_circuits_clientside(); #if 0 /* disable for now, until predict-and-launch-new can cull leftovers */ + + /* If we ever re-enable, this has to move into + * circuit_build_needed_circs */ + circ = circuit_get_youngest_clean_open(CIRCUIT_PURPOSE_C_GENERAL); if (get_options()->RunTesting && circ && @@ -995,8 +1014,6 @@ circuit_build_needed_circs(time_t now) } #endif } - if (!options->DisablePredictedCircuits) - circuit_predict_and_launch_new(); } /** If the stream conn is a member of any of the linked diff --git a/src/or/circuituse.h b/src/or/circuituse.h index 11e5a64..1559a40 100644 --- a/src/or/circuituse.h +++ b/src/or/circuituse.h @@ -21,6 +21,7 @@ int circuit_conforms_to_options(const origin_circuit_t *circ, const or_options_t *options); #endif void circuit_build_needed_circs(time_t now); +void circuit_expire_old_circs_as_needed(time_t now); void circuit_detach_stream(circuit_t *circ, edge_connection_t *conn); void circuit_expire_old_circuits_serverside(time_t now); diff --git a/src/or/main.c b/src/or/main.c index bd23141..495a71f 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -1485,8 +1485,11 @@ run_scheduled_events(time_t now) * and we make a new circ if there are no clean circuits. */ have_dir_info = router_have_minimum_dir_info(); - if (have_dir_info && !net_is_disabled()) + if (have_dir_info && !net_is_disabled()) { circuit_build_needed_circs(now); + } else { + circuit_expire_old_circs_as_needed(now); + } /* every 10 seconds, but not at the same second as other such events */ if (now % 10 == 5)

1 0

[tor/maint-0.2.5] Merge remote-tracking branch 'public/bug8387_024' into maint-0.2.5
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 856114ab1c5a8a7a1b7993ee34adbb7cbf4eaa10 Merge: 35791f4 ed3d789 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 10:01:56 2014 +0200 Merge remote-tracking branch 'public/bug8387_024' into maint-0.2.5 changes/bug8387 | 11 +++++++++++ src/or/circuituse.c | 27 ++++++++++++++++++++++----- src/or/circuituse.h | 1 + src/or/main.c | 5 ++++- 4 files changed, 38 insertions(+), 6 deletions(-)

1 0

[tor/maint-0.2.5] Add another 8387 diagnostic
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 32495ee3099c3fb512a986a8e5db32ebf153b439 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 10:05:00 2014 +0200 Add another 8387 diagnostic When we run into bug 8387 (if we run into it again), report when we last called circuit_expire_old_circuits_clientside(). This will let us know -- if my fix for 8387 doesn't work -- whether my diagnosis was at least correct. Suggested by Andrea. --- src/or/circuituse.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/or/circuituse.c b/src/or/circuituse.c index 600aede..714754a 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -783,6 +783,10 @@ circuit_expire_building(void) } } +/** For debugging #8387: track when we last called + * circuit_expire_old_circuits_clientside. */ +static time_t last_expired_clientside_circuits = 0; + /** * As a diagnostic for bug 8387, log information about how many one-hop * circuits we have around that have been there for at least age @@ -894,6 +898,10 @@ circuit_log_ancient_one_hop_circuits(int age) } } SMARTLIST_FOREACH_END(ocirc); + log_notice(LD_HEARTBEAT, "It has been %ld seconds since I last called " + "circuit_expire_old_circuits_clientside().", + (long)(now - last_expired_clientside_circuits)); + done: smartlist_free(log_these); } @@ -1220,6 +1228,7 @@ circuit_expire_old_circuits_clientside(void) tor_gettimeofday(&now); cutoff = now; + last_expired_clientside_circuits = now.tv_sec; if (! circuit_build_times_disabled() && circuit_build_times_needs_circuits(get_circuit_build_times())) {

1 0