July 2014 - tor-commits - lists.torproject.org

[tor/master] Add stanza on lcov coverage target
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit eed942aed9175971b510f0f720274171f1895d01 Author: Kevin Murray <spam(a)kdmurray.id.au> Date: Wed Jul 9 18:55:44 2014 +1000 Add stanza on lcov coverage target Adds a note about using `make coverage-html` to generate coverage reports. Signed-off-by: Kevin Murray <spam(a)kdmurray.id.au> --- doc/HACKING | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/doc/HACKING b/doc/HACKING index 2562f7e..6b21426 100644 --- a/doc/HACKING +++ b/doc/HACKING @@ -114,6 +114,26 @@ valgrind --leak-check=yes --error-limit=no --show-reachable=yes src/or/tor pass --undef-value-errors=no to valgrind, or rebuild your openssl with -DPURIFY.) +Running lcov for unit test coverage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Lcov is a utility that generates pretty HTML reports of test code coverage. +To generate such a report: + +----- + ./configure --enable-coverage + make + make coverage-html + $BROWSER ./coverage_html/index.html +----- + +This will run the tor unit test suite `./src/test/test` and generate the HTML +coverage code report under the directory ./coverage_html/. To change the +output directory, use `make coverage-html HTML_COVER_DIR=./funky_new_cov_dir`. + +Coverage diffs using lcov are not currently implemented, but are being +investigated (as of July 2014). + Running gcov for unit test coverage ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 0

[tor/master] When making a tempdir for windows unit tests, avoid an extra backslash
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 3f683aadcd03c311c1d987ab847a77dbe3fb2bf1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 11:31:22 2014 +0200 When making a tempdir for windows unit tests, avoid an extra backslash The extra \ is harmless, but mildly unpleasant. Fixes 12392; bugfix on 0.2.2.25-alpha where we started using GetTempDir(). Based on a patch by Gisle Vanem. --- src/test/test.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/test/test.c b/src/test/test.c index 8bce9c9..429e79c 100644 --- a/src/test/test.c +++ b/src/test/test.c @@ -106,11 +106,18 @@ setup_directory(void) { char buf[MAX_PATH]; const char *tmp = buf; + const char *extra_backslash = ""; /* If this fails, we're probably screwed anyway */ if (!GetTempPathA(sizeof(buf),buf)) - tmp = "c:\\windows\\temp"; + tmp = "c:\\windows\\temp\\"; + if (strcmpend(tmp, "\\")) { + /* According to MSDN, it should be impossible for GetTempPath to give us + * an answer that doesn't end with \. But let's make sure. */ + extra_backslash = "\\"; + } tor_snprintf(temp_dir, sizeof(temp_dir), - "%s\\tor_test_%d_%s", tmp, (int)getpid(), rnd32); + "%s%stor_test_%d_%s", tmp, extra_backslash, + (int)getpid(), rnd32); r = mkdir(temp_dir); } #else

1 0

[tor/master] Apply GeoIPExcludeUnknown before checking transitions
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 4da4c4c63f02e9551eaeb8ad9ce5c6f2d1f34ef9 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 11:14:59 2014 +0200 Apply GeoIPExcludeUnknown before checking transitions Otherwise, it always seems as though our Exclude* options have changed, since we're comparing modified to unmodified values. Patch from qwerty1. Fixes bug 9801. Bugfix on 0.2.4.10-alpha, where GeoIPExcludeUnknown was introduced. --- changes/bug9801 | 5 +++++ src/or/config.c | 28 ++++++++++++++-------------- 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/changes/bug9801 b/changes/bug9801 new file mode 100644 index 0000000..6b23b71 --- /dev/null +++ b/changes/bug9801 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - When GeoIPExcludeUnkonwn is enabled, do not incorrectly decide + that our options have changed every time we SIGHUP. Fixes bug + 9801; bugfix on 0.2.4.10-alpha. Patch from "qwerty1". + diff --git a/src/or/config.c b/src/or/config.c index 10df839..2c3dfc2 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -1592,6 +1592,20 @@ options_act(const or_options_t *old_options) return -1; } + config_maybe_load_geoip_files_(options, old_options); + + if (geoip_is_loaded(AF_INET) && options->GeoIPExcludeUnknown) { + /* ExcludeUnknown is true or "auto" */ + const int is_auto = options->GeoIPExcludeUnknown == -1; + int changed; + + changed = routerset_add_unknown_ccs(&options->ExcludeNodes, is_auto); + changed += routerset_add_unknown_ccs(&options->ExcludeExitNodes, is_auto); + + if (changed) + routerset_add_unknown_ccs(&options->ExcludeExitNodesUnion_, is_auto); + } + /* Check for transitions that need action. */ if (old_options) { int revise_trackexithosts = 0; @@ -1687,20 +1701,6 @@ options_act(const or_options_t *old_options) connection_or_update_token_buckets(get_connection_array(), options); } - config_maybe_load_geoip_files_(options, old_options); - - if (geoip_is_loaded(AF_INET) && options->GeoIPExcludeUnknown) { - /* ExcludeUnknown is true or "auto" */ - const int is_auto = options->GeoIPExcludeUnknown == -1; - int changed; - - changed = routerset_add_unknown_ccs(&options->ExcludeNodes, is_auto); - changed += routerset_add_unknown_ccs(&options->ExcludeExitNodes, is_auto); - - if (changed) - routerset_add_unknown_ccs(&options->ExcludeExitNodesUnion_, is_auto); - } - if (options->CellStatistics || options->DirReqStatistics || options->EntryStatistics || options->ExitPortStatistics || options->ConnDirectionStatistics ||

1 0

[tor/master] diagnostic for 12184: Add a call to channel_dump_statistics
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 5d2045ee8ba42a294291b187234be6ead6864cab Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 10:34:39 2014 +0200 diagnostic for 12184: Add a call to channel_dump_statistics --- changes/further-12184-diagnostic | 2 ++ src/or/circuitbuild.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/changes/further-12184-diagnostic b/changes/further-12184-diagnostic new file mode 100644 index 0000000..89e9f46 --- /dev/null +++ b/changes/further-12184-diagnostic @@ -0,0 +1,2 @@ + o Minor features (diagnostic): + - Slightly enhance the diagnostic message for bug 12184. diff --git a/src/or/circuitbuild.c b/src/or/circuitbuild.c index e743a25..283afee 100644 --- a/src/or/circuitbuild.c +++ b/src/or/circuitbuild.c @@ -183,6 +183,8 @@ get_unique_circ_id_by_chan(channel_t *chan) if (0) circuitmux_assert_okay(chan->cmux); + channel_dump_statistics(chan, LOG_WARN); + return 0; }

1 0

[tor/master] configure.ac: fix build with --enable-bufferevents
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 589de5ec20bf6f15813fc3bddb7a814309184800 Author: Anthony G. Basile <blueness(a)gentoo.org> Date: Fri Jun 20 08:47:26 2014 -0400 configure.ac: fix build with --enable-bufferevents When building with bufferevents enabled, configure.ac throws an error if "$ac_cv_header_event2_bufferevent_ssl_h" is not set to "yes". However, nowhere was AC_CHECK_HEADERS(event2/bufferevent_ssl.h) done. This commit adds the check. --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 38d9222..1d8cfa7 100644 --- a/configure.ac +++ b/configure.ac @@ -454,7 +454,7 @@ AC_CHECK_MEMBERS([struct event.min_heap_idx], , , [#include <event.h> ]) -AC_CHECK_HEADERS(event2/event.h event2/dns.h) +AC_CHECK_HEADERS(event2/event.h event2/dns.h event2/bufferevent_ssl.h) LIBS="$save_LIBS" LDFLAGS="$save_LDFLAGS"

1 0

[tor/master] src/or/connection.c: expose bucket_millis_empty for bufferevents test
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit d504a4e36f571a5b6ed84529d3781d08df82f117 Author: Anthony G. Basile <blueness(a)gentoo.org> Date: Thu Jun 26 11:15:36 2014 -0400 src/or/connection.c: expose bucket_millis_empty for bufferevents test Currently tor fails to build its test when enabled with bufferevents because an #ifndef USE_BUFFEREVENTS hides bucket_millis_empty() and friends. This is fine if we don't run tests, but if we do, we need these functions in src/or/libtor-testing.a when linking src/test/test. This patch moves the functions outside the #ifndef and exposes them. See downstream bug: https://bugs.gentoo.org/show_bug.cgi?id=510124 --- src/or/connection.c | 60 +++++++++++++++++++++++++-------------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/src/or/connection.c b/src/or/connection.c index 0b03092..4788bdf 100644 --- a/src/or/connection.c +++ b/src/or/connection.c @@ -2650,14 +2650,6 @@ record_num_bytes_transferred(connection_t *conn, } #endif -#ifndef USE_BUFFEREVENTS -/** Last time at which the global or relay buckets were emptied in msec - * since midnight. */ -static uint32_t global_relayed_read_emptied = 0, - global_relayed_write_emptied = 0, - global_read_emptied = 0, - global_write_emptied = 0; - /** Helper: convert given tvnow time value to milliseconds since * midnight. */ static uint32_t @@ -2667,6 +2659,28 @@ msec_since_midnight(const struct timeval *tvnow) ((uint32_t)tvnow->tv_usec / (uint32_t)1000L)); } +/** Helper: return the time in milliseconds since last_empty_time + * when a bucket ran empty that previously had tokens_before tokens + * now has tokens_after tokens after refilling at timestamp + * tvnow, capped at milliseconds_elapsed milliseconds since + * last refilling that bucket. Return 0 if the bucket has not been empty + * since the last refill or has not been refilled. */ +uint32_t +bucket_millis_empty(int tokens_before, uint32_t last_empty_time, + int tokens_after, int milliseconds_elapsed, + const struct timeval *tvnow) +{ + uint32_t result = 0, refilled; + if (tokens_before <= 0 && tokens_after > tokens_before) { + refilled = msec_since_midnight(tvnow); + result = (uint32_t)((refilled + 86400L * 1000L - last_empty_time) % + (86400L * 1000L)); + if (result > (uint32_t)milliseconds_elapsed) + result = (uint32_t)milliseconds_elapsed; + } + return result; +} + /** Check if a bucket which had tokens_before tokens and which got * tokens_removed tokens removed at timestamp tvnow has run * out of tokens, and if so, note the milliseconds since midnight in @@ -2680,6 +2694,14 @@ connection_buckets_note_empty_ts(uint32_t *timestamp_var, *timestamp_var = msec_since_midnight(tvnow); } +#ifndef USE_BUFFEREVENTS +/** Last time at which the global or relay buckets were emptied in msec + * since midnight. */ +static uint32_t global_relayed_read_emptied = 0, + global_relayed_write_emptied = 0, + global_read_emptied = 0, + global_write_emptied = 0; + /** We just read num_read and wrote num_written bytes * onto conn. Decrement buckets appropriately. */ static void @@ -2838,28 +2860,6 @@ connection_bucket_refill_helper(int *bucket, int rate, int burst, } } -/** Helper: return the time in milliseconds since last_empty_time - * when a bucket ran empty that previously had tokens_before tokens - * now has tokens_after tokens after refilling at timestamp - * tvnow, capped at milliseconds_elapsed milliseconds since - * last refilling that bucket. Return 0 if the bucket has not been empty - * since the last refill or has not been refilled. */ -uint32_t -bucket_millis_empty(int tokens_before, uint32_t last_empty_time, - int tokens_after, int milliseconds_elapsed, - const struct timeval *tvnow) -{ - uint32_t result = 0, refilled; - if (tokens_before <= 0 && tokens_after > tokens_before) { - refilled = msec_since_midnight(tvnow); - result = (uint32_t)((refilled + 86400L * 1000L - last_empty_time) % - (86400L * 1000L)); - if (result > (uint32_t)milliseconds_elapsed) - result = (uint32_t)milliseconds_elapsed; - } - return result; -} - /** Time has passed; increment buckets appropriately. */ void connection_bucket_refill(int milliseconds_elapsed, time_t now)

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.5'
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 7591ce64fbaae8d480224a93d62abda10011c2ba Merge: 7bc7188 9e46855 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 11:01:20 2014 +0200 Merge remote-tracking branch 'origin/maint-0.2.5' changes/bufferevent_compilation | 5 ++++ changes/bug8387 | 11 +++++++ changes/further-12184-diagnostic | 2 ++ configure.ac | 2 +- src/or/circuitbuild.c | 2 ++ src/or/circuituse.c | 36 +++++++++++++++++++---- src/or/circuituse.h | 1 + src/or/connection.c | 60 +++++++++++++++++++------------------- src/or/main.c | 5 +++- 9 files changed, 87 insertions(+), 37 deletions(-) diff --cc src/or/router.c index 4fcd4a5,2cdbb0c..4fcd4a5 mode 100755,100644..100644 --- a/src/or/router.c +++ b/src/or/router.c

1 0

[tor/master] changes file for 12474, 12438.
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 9e46855538e9b848b7c37fda502e5d109079cee0 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 11:00:49 2014 +0200 changes file for 12474, 12438. --- changes/bufferevent_compilation | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/changes/bufferevent_compilation b/changes/bufferevent_compilation new file mode 100644 index 0000000..0690e65 --- /dev/null +++ b/changes/bufferevent_compilation @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Fix compilation when building with bufferevents enabled. (This + configuration is still not expected to work, however.) + Fixes bugs 12438, 12474; bugfixes on 0.2.5.1-alpha. Patches + from Anthony G. Basile.

1 0

[tor/master] Merge remote-tracking branch 'public/bug8387_024' into maint-0.2.5
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit 856114ab1c5a8a7a1b7993ee34adbb7cbf4eaa10 Merge: 35791f4 ed3d789 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 16 10:01:56 2014 +0200 Merge remote-tracking branch 'public/bug8387_024' into maint-0.2.5 changes/bug8387 | 11 +++++++++++ src/or/circuituse.c | 27 ++++++++++++++++++++++----- src/or/circuituse.h | 1 + src/or/main.c | 5 ++++- 4 files changed, 38 insertions(+), 6 deletions(-)

1 0

[tor/master] Fix a bug where streams would linger forever when we had no dirinfo
by nickm＠torproject.org 16 Jul '14

16 Jul '14

commit ed3d7892c721c9495215ecad2e18c026d29fbb9b Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Jul 9 16:14:14 2014 -0400 Fix a bug where streams would linger forever when we had no dirinfo fixes bug 8387; fix on 0.1.1.11-alpha (code), or on 0.2.4.10-alpha (behavior). --- changes/bug8387 | 11 +++++++++++ src/or/circuituse.c | 27 ++++++++++++++++++++++----- src/or/circuituse.h | 1 + src/or/main.c | 5 ++++- 4 files changed, 38 insertions(+), 6 deletions(-) diff --git a/changes/bug8387 b/changes/bug8387 new file mode 100644 index 0000000..2ec0487 --- /dev/null +++ b/changes/bug8387 @@ -0,0 +1,11 @@ + o Major bugfixes (client): + + - Perform circuit cleanup operations even when circuit + construction operations are disabled (because the network is + disabled, or because there isn't enough directory information). + Previously, when we were not building predictive circuits, we + were not closing expired circuits either. + + Fixes bug 8387; bugfix on 0.1.1.11-alpha. This bug became visible + in 0.2.4.10-alpha when we became more strict about when we have + "enough directory information to build circuits". diff --git a/src/or/circuituse.c b/src/or/circuituse.c index c2d2b2e..9933b9c 100644 --- a/src/or/circuituse.c +++ b/src/or/circuituse.c @@ -969,7 +969,6 @@ circuit_predict_and_launch_new(void) void circuit_build_needed_circs(time_t now) { - static time_t time_to_new_circuit = 0; const or_options_t *options = get_options(); /* launch a new circ for any pending streams that need one */ @@ -978,14 +977,34 @@ circuit_build_needed_circs(time_t now) /* make sure any hidden services have enough intro points */ rend_services_introduce(); - if (time_to_new_circuit < now) { + circuit_expire_old_circs_as_needed(now); + + if (!options->DisablePredictedCircuits) + circuit_predict_and_launch_new(); +} + +/** + * Called once a second either directly or from + * circuit_build_needed_circs(). As appropriate (once per NewCircuitPeriod) + * resets failure counts and expires old circuits. + */ +void +circuit_expire_old_circs_as_needed(time_t now) +{ + static time_t time_to_expire_and_reset = 0; + + if (time_to_expire_and_reset < now) { circuit_reset_failure_count(1); - time_to_new_circuit = now + options->NewCircuitPeriod; + time_to_expire_and_reset = now + get_options()->NewCircuitPeriod; if (proxy_mode(get_options())) addressmap_clean(now); circuit_expire_old_circuits_clientside(); #if 0 /* disable for now, until predict-and-launch-new can cull leftovers */ + + /* If we ever re-enable, this has to move into + * circuit_build_needed_circs */ + circ = circuit_get_youngest_clean_open(CIRCUIT_PURPOSE_C_GENERAL); if (get_options()->RunTesting && circ && @@ -995,8 +1014,6 @@ circuit_build_needed_circs(time_t now) } #endif } - if (!options->DisablePredictedCircuits) - circuit_predict_and_launch_new(); } /** If the stream conn is a member of any of the linked diff --git a/src/or/circuituse.h b/src/or/circuituse.h index 11e5a64..1559a40 100644 --- a/src/or/circuituse.h +++ b/src/or/circuituse.h @@ -21,6 +21,7 @@ int circuit_conforms_to_options(const origin_circuit_t *circ, const or_options_t *options); #endif void circuit_build_needed_circs(time_t now); +void circuit_expire_old_circs_as_needed(time_t now); void circuit_detach_stream(circuit_t *circ, edge_connection_t *conn); void circuit_expire_old_circuits_serverside(time_t now); diff --git a/src/or/main.c b/src/or/main.c index bd23141..495a71f 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -1485,8 +1485,11 @@ run_scheduled_events(time_t now) * and we make a new circ if there are no clean circuits. */ have_dir_info = router_have_minimum_dir_info(); - if (have_dir_info && !net_is_disabled()) + if (have_dir_info && !net_is_disabled()) { circuit_build_needed_circs(now); + } else { + circuit_expire_old_circs_as_needed(now); + } /* every 10 seconds, but not at the same second as other such events */ if (now % 10 == 5)

1 0