tor-commits November 2013

tor-commits@lists.torproject.org

17 participants
1171 discussions

[tor/release-0.2.4] start to migrate recent changes
by arma＠torproject.org 15 Nov '13

15 Nov '13

commit f503f3043673e6eb550c722b74e57b49d4443ae7 Author: Roger Dingledine <arma(a)torproject.org> Date: Fri Nov 15 17:08:45 2013 -0500 start to migrate recent changes --- ChangeLog | 24 ++++++++++++++++++++++++ changes/bug6055 | 6 ------ changes/bug9645a | 5 ----- changes/bug9731 | 3 --- changes/bug9780 | 8 -------- 5 files changed, 24 insertions(+), 22 deletions(-) diff --git a/ChangeLog b/ChangeLog index 2647723..59311cb 100644 --- a/ChangeLog +++ b/ChangeLog @@ -2,10 +2,34 @@ Changes in version 0.2.4.18-rc - 2013-10-?? Tor 0.2.4.18-rc is the fourth release candidate for the Tor 0.2.4.x series. [...] + o Major enhancements: + - Re-enable TLS 1.1 and 1.2 when built with OpenSSL 1.0.1e or later. + (OpenSSL before 1.0.1 didn't have TLS 1.1 or 1.2. OpenSSL from 1.0.1 + through 1.0.1d had bugs that prevented renegotiation from working + with TLS 1.1 or 1.2, so we disabled them to solve bug 6033.) Fix for + issue #6055. + o Major bugfixes: - Stop trying to bootstrap all our directory information from only our first guard. Discovered while fixing bug 9946; bugfix on 0.2.4.8-alpha. + - No longer stop reading or writing on cpuworker connections when + our rate limiting buckets go empty. Now we should handle circuit + handshake requests more promptly. Resolves bug 9731. + + o Minor bugfixes: + - If we are unable to save a microdescriptor to the journal, do not + drop it from memory and then reattempt downloading it. Fixes bug + 9645; bugfix on 0.2.2.6-alpha. + + o Minor bugfixes (performance, fingerprinting): + - Our default TLS ecdhe groups were backwards: we meant to be using + P224 for relays (for performance win) and P256 for bridges (since + it is more common in the wild). Instead we had it backwards. After + reconsideration, we decided that the default should be P256 on all + hosts, since its security is probably better, and since P224 is + reportedly used quite little in the wild. Found by "skruffy" on + IRC. Fix for bug 9780; bugfix on 0.2.4.8-alpha. o Minor bugfixes (on 0.2.4.x): - Correctly log long IPv6 exit policies, instead of truncating them diff --git a/changes/bug6055 b/changes/bug6055 deleted file mode 100644 index 0073007..0000000 --- a/changes/bug6055 +++ /dev/null @@ -1,6 +0,0 @@ - o Major enhancements: - - Re-enable TLS 1.1 and 1.2 when built with OpenSSL 1.0.1e or later. - (OpenSSL before 1.0.1 didn't have TLS 1.1 or 1.2. OpenSSL from 1.0.1 - through 1.0.1d had bugs that prevented renegotiation from working - with TLS 1.1 or 1.2, so we disabled them to solve bug 6033.) Fix for - issue #6055. diff --git a/changes/bug9645a b/changes/bug9645a deleted file mode 100644 index 2daba65..0000000 --- a/changes/bug9645a +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes: - - If we are unable to save a microdescriptor to the journal, do not - drop it from memory and then reattempt downloading it. Fixes bug - 9645; bugfix on 0.2.2.6-alpha. - diff --git a/changes/bug9731 b/changes/bug9731 deleted file mode 100644 index 828496a..0000000 --- a/changes/bug9731 +++ /dev/null @@ -1,3 +0,0 @@ - o Major bugfixes: - - Do not apply connection_consider_empty_read/write_buckets to - cpuworker connections. diff --git a/changes/bug9780 b/changes/bug9780 deleted file mode 100644 index 3cb51bd..0000000 --- a/changes/bug9780 +++ /dev/null @@ -1,8 +0,0 @@ - o Minor bugfixes (performance, fingerprinting): - - Our default TLS ecdhe groups were backwards: we meant to be using - P224 for relays (for performance win) and P256 for bridges (since - it is more common in the wild). Instead we had it backwards. After - reconsideration, we decided that the default should be P256 on all - hosts, since its security is probably better, and since P224 is - reportedly used quite little in the wild. Found by "skruffy" on - IRC. Fix for bug 9780; bugfix on 0.2.4.8-alpha.

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.4'
by nickm＠torproject.org 15 Nov '13

15 Nov '13

commit 7a2b30fe16eacc040b3dd11f8c39c410628c2f43 Merge: 4aa9aff 59f50c8 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Nov 15 15:35:00 2013 -0500 Merge remote-tracking branch 'origin/maint-0.2.4' Conflicts: src/or/relay.c Conflict changes were easy; compilation fixes required were using using TOR_SIMPLEQ_FIRST to get head of cell queue. changes/bug9093 | 7 ++++++ src/common/util.c | 12 +++++++++++ src/common/util.h | 1 + src/or/circuitlist.c | 58 +++++++++++++++++++++++++++++++++++++++++--------- src/or/or.h | 5 +++++ src/or/relay.c | 7 +++++- 6 files changed, 79 insertions(+), 11 deletions(-) diff --cc src/or/circuitlist.c index 19f4618,b0e24a5..c31bc49 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@@ -1620,10 -1525,41 +1620,43 @@@ n_cells_in_circ_queues(const circuit_t return n; } - /** helper to sort a list of circuit_q by total queue lengths, in descending - * order. */ + /** + * Return the age of the oldest cell queued on c, in milliseconds. + * Return 0 if there are no cells queued on c. Requires that now be + * the current time in milliseconds since the epoch, truncated. + * + * This function will return incorrect results if the oldest cell queued on + * the circuit is older than 2**32 msec (about 49 days) old. + */ + static uint32_t + circuit_max_queued_cell_age(const circuit_t *c, uint32_t now) + { + uint32_t age = 0; - if (c->n_chan_cells.head) - age = now - c->n_chan_cells.head->inserted_time; ++ packed_cell_t *cell; ++ ++ if (NULL != (cell = TOR_SIMPLEQ_FIRST(&c->n_chan_cells.head))) ++ age = now - cell->inserted_time; + + if (! CIRCUIT_IS_ORIGIN(c)) { + const or_circuit_t *orcirc = TO_OR_CIRCUIT((circuit_t*)c); - if (orcirc->p_chan_cells.head) { - uint32_t age2 = now - orcirc->p_chan_cells.head->inserted_time; ++ if (NULL != (cell = TOR_SIMPLEQ_FIRST(&orcirc->p_chan_cells.head))) { ++ uint32_t age2 = now - cell->inserted_time; + if (age2 > age) + return age2; + } + } + return age; + } + + /** Temporary variable for circuits_compare_by_oldest_queued_cell_ This is a + * kludge to work around the fact that qsort doesn't provide a way for + * comparison functions to take an extra argument. */ + static uint32_t circcomp_now_tmp; + + /** Helper to sort a list of circuit_t by age of oldest cell, in descending + * order. Requires that circcomp_now_tmp is set correctly. */ static int - circuits_compare_by_queue_len_(const void **a_, const void **b_) + circuits_compare_by_oldest_queued_cell_(const void **a_, const void **b_) { const circuit_t *a = *a_; const circuit_t *b = *b_; @@@ -1667,12 -1604,16 +1701,16 @@@ circuits_handle_oom(size_t current_allo /* This algorithm itself assumes that you've got enough memory slack * to actually run it. */ - for (circ = global_circuitlist; circ; circ = circ->next) + TOR_LIST_FOREACH(circ, &global_circuitlist, head) smartlist_add(circlist, circ); + /* Set circcomp_now_tmp so that the sort can work. */ + tor_gettimeofday_cached(&now); + circcomp_now_tmp = (uint32_t)tv_to_msec(&now); + /* This is O(n log n); there are faster algorithms we could use instead. * Let's hope this doesn't happen enough to be in the critical path. */ - smartlist_sort(circlist, circuits_compare_by_queue_len_); + smartlist_sort(circlist, circuits_compare_by_oldest_queued_cell_); /* Okay, now the worst circuits are at the front of the list. Let's mark * them, and reclaim their storage aggressively. */ diff --cc src/or/or.h index a313248,5318b0f..6b18f13 --- a/src/or/or.h +++ b/src/or/or.h @@@ -1108,20 -1073,17 +1108,25 @@@ typedef struct var_cell_t uint8_t payload[FLEXIBLE_ARRAY_MEMBER]; } var_cell_t; +/** A parsed Extended ORPort message. */ +typedef struct ext_or_cmd_t { + uint16_t cmd; /** Command type */ + uint16_t len; /** Body length */ + char body[FLEXIBLE_ARRAY_MEMBER]; /** Message body */ +} ext_or_cmd_t; + /** A cell as packed for writing to the network. */ typedef struct packed_cell_t { - struct packed_cell_t *next; /**< Next cell queued on this circuit. */ + /** Next cell queued on this circuit. */ + TOR_SIMPLEQ_ENTRY(packed_cell_t) next; char body[CELL_MAX_NETWORK_SIZE]; /**< Cell as packed for network. */ + uint32_t inserted_time; /**< Time (in milliseconds since epoch, with high + * bits truncated) when this cell was inserted. */ } packed_cell_t; + /* XXXX This next structure may be obsoleted by inserted_time in + * packed_cell_t */ + /** Number of cells added to a circuit queue including their insertion * time on 10 millisecond detail; used for buffer statistics. */ typedef struct insertion_time_elem_t { diff --cc src/or/relay.c index 3600707,63119cb..0c9267a --- a/src/or/relay.c +++ b/src/or/relay.c @@@ -2149,75 -2133,30 +2149,80 @@@ packed_cell_copy(const cell_t *cell, in void cell_queue_append(cell_queue_t *queue, packed_cell_t *cell) { - if (queue->tail) { - tor_assert(!queue->tail->next); - queue->tail->next = cell; + TOR_SIMPLEQ_INSERT_TAIL(&queue->head, cell, next); + ++queue->n; +} + +/** Append command of type command in direction to queue for + * CELL_STATS event. */ +static void +cell_command_queue_append(cell_queue_t *queue, uint8_t command) +{ + insertion_command_queue_t *ic_queue = queue->insertion_commands; + if (!ic_pool) + ic_pool = mp_pool_new(sizeof(insertion_command_elem_t), 1024); + if (!ic_queue) { + ic_queue = tor_malloc_zero(sizeof(insertion_command_queue_t)); + queue->insertion_commands = ic_queue; + } + if (ic_queue->last && ic_queue->last->command == command) { + ic_queue->last->counter++; } else { - queue->head = cell; + insertion_command_elem_t *elem = mp_pool_get(ic_pool); + elem->next = NULL; + elem->command = command; + elem->counter = 1; + if (ic_queue->last) { + ic_queue->last->next = elem; + ic_queue->last = elem; + } else { + ic_queue->first = ic_queue->last = elem; + } } - queue->tail = cell; - cell->next = NULL; - ++queue->n; } -/** Append a newly allocated copy of cell to the end of queue */ +/** Retrieve oldest command from queue and write it to + * command for CELL_STATS event. Return 0 for success, -1 + * otherwise. */ +static int +cell_command_queue_pop(uint8_t *command, cell_queue_t *queue) +{ + int res = -1; + insertion_command_queue_t *ic_queue = queue->insertion_commands; + if (ic_queue && ic_queue->first) { + insertion_command_elem_t *ic_elem = ic_queue->first; + ic_elem->counter--; + if (ic_elem->counter < 1) { + ic_queue->first = ic_elem->next; + if (ic_elem == ic_queue->last) + ic_queue->last = NULL; + mp_pool_release(ic_elem); + } + *command = ic_elem->command; + res = 0; + } + return res; +} + +/** Append a newly allocated copy of cell to the end of the + * exitward (or app-ward) queue of circ. If + * use_stats is true, record statistics about the cell. + */ void -cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell, - int wide_circ_ids) +cell_queue_append_packed_copy(circuit_t *circ, cell_queue_t *queue, + int exitward, const cell_t *cell, + int wide_circ_ids, int use_stats) { + struct timeval now; packed_cell_t *copy = packed_cell_copy(cell, wide_circ_ids); + tor_gettimeofday_cached(&now); + copy->inserted_time = (uint32_t)tv_to_msec(&now); + /* Remember the time when this cell was put in the queue. */ + /*XXXX This may be obsoleted by inserted_time */ - if (get_options()->CellStatistics) { + if ((get_options()->CellStatistics || + get_options()->TestingEnableCellStatsEvent) && use_stats) { + struct timeval now; uint32_t added; insertion_time_queue_t *it_queue = queue->insertion_times; if (!it_pool)

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.3' into maint-0.2.4
by nickm＠torproject.org 15 Nov '13

15 Nov '13

commit 59f50c80d443a7e148f85cfed493e3e703cc4386 Merge: a82b18f 9e90707 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Nov 15 15:29:24 2013 -0500 Merge remote-tracking branch 'origin/maint-0.2.3' into maint-0.2.4 Conflicts: src/or/or.h src/or/relay.c Conflicts were simple to resolve. More fixes were needed for compilation, including: reinstating the tv_to_msec function, and renaming *_conn_cells to *_chan_cells. changes/bug9093 | 7 +++++++ src/common/util.c | 12 +++++++++++ src/common/util.h | 1 + src/or/circuitlist.c | 56 +++++++++++++++++++++++++++++++++++++++++--------- src/or/or.h | 5 +++++ src/or/relay.c | 8 ++++++-- 6 files changed, 77 insertions(+), 12 deletions(-) diff --cc src/common/util.c index ae385e1,b16afa1..5eb0f9a --- a/src/common/util.c +++ b/src/common/util.c @@@ -1232,6 -1322,6 +1232,18 @@@ tv_mdiff(const struct timeval *start, c return mdiff; } ++/** ++ * Converts timeval to milliseconds. ++ */ ++int64_t ++tv_to_msec(const struct timeval *tv) ++{ ++ int64_t conv = ((int64_t)tv->tv_sec)*1000L; ++ /* Round ghetto-style */ ++ conv += ((int64_t)tv->tv_usec+500)/1000L; ++ return conv; ++} ++ /** Yield true iff y is a leap-year. */ #define IS_LEAPYEAR(y) (!(y % 4) && ((y % 100) || !(y % 400))) /** Helper: Return the number of leap-days between Jan 1, y1 and Jan 1, y2. */ diff --cc src/common/util.h index 96a02dd,8977d27..73daa6e --- a/src/common/util.h +++ b/src/common/util.h @@@ -251,8 -239,11 +251,9 @@@ void base16_encode(char *dest, size_t d int base16_decode(char *dest, size_t destlen, const char *src, size_t srclen); /* Time helpers */ -double tv_to_double(const struct timeval *tv); -int64_t tv_to_msec(const struct timeval *tv); -int64_t tv_to_usec(const struct timeval *tv); long tv_udiff(const struct timeval *start, const struct timeval *end); long tv_mdiff(const struct timeval *start, const struct timeval *end); ++int64_t tv_to_msec(const struct timeval *tv); int tor_timegm(const struct tm *tm, time_t *time_out); #define RFC1123_TIME_LEN 29 void format_rfc1123_time(char *buf, time_t t); diff --cc src/or/circuitlist.c index daeaa37,6250c11..b0e24a5 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@@ -1525,10 -1383,41 +1525,41 @@@ n_cells_in_circ_queues(const circuit_t return n; } - /** helper to sort a list of circuit_q by total queue lengths, in descending - * order. */ + /** + * Return the age of the oldest cell queued on c, in milliseconds. + * Return 0 if there are no cells queued on c. Requires that now be + * the current time in milliseconds since the epoch, truncated. + * + * This function will return incorrect results if the oldest cell queued on + * the circuit is older than 2**32 msec (about 49 days) old. + */ + static uint32_t + circuit_max_queued_cell_age(const circuit_t *c, uint32_t now) + { + uint32_t age = 0; - if (c->n_conn_cells.head) - age = now - c->n_conn_cells.head->inserted_time; ++ if (c->n_chan_cells.head) ++ age = now - c->n_chan_cells.head->inserted_time; + + if (! CIRCUIT_IS_ORIGIN(c)) { + const or_circuit_t *orcirc = TO_OR_CIRCUIT((circuit_t*)c); - if (orcirc->p_conn_cells.head) { - uint32_t age2 = now - orcirc->p_conn_cells.head->inserted_time; ++ if (orcirc->p_chan_cells.head) { ++ uint32_t age2 = now - orcirc->p_chan_cells.head->inserted_time; + if (age2 > age) + return age2; + } + } + return age; + } + + /** Temporary variable for circuits_compare_by_oldest_queued_cell_ This is a + * kludge to work around the fact that qsort doesn't provide a way for + * comparison functions to take an extra argument. */ + static uint32_t circcomp_now_tmp; + + /** Helper to sort a list of circuit_t by age of oldest cell, in descending + * order. Requires that circcomp_now_tmp is set correctly. */ static int - circuits_compare_by_queue_len_(const void **a_, const void **b_) + circuits_compare_by_oldest_queued_cell_(const void **a_, const void **b_) { const circuit_t *a = *a_; const circuit_t *b = *b_; diff --cc src/or/or.h index eff5a6d,4622391..5318b0f --- a/src/or/or.h +++ b/src/or/or.h @@@ -1076,9 -911,14 +1076,14 @@@ typedef struct var_cell_t /** A cell as packed for writing to the network. */ typedef struct packed_cell_t { struct packed_cell_t *next; /**< Next cell queued on this circuit. */ - char body[CELL_NETWORK_SIZE]; /**< Cell as packed for network. */ + char body[CELL_MAX_NETWORK_SIZE]; /**< Cell as packed for network. */ + uint32_t inserted_time; /**< Time (in milliseconds since epoch, with high + * bits truncated) when this cell was inserted. */ } packed_cell_t; + /* XXXX This next structure may be obsoleted by inserted_time in + * packed_cell_t */ + /** Number of cells added to a circuit queue including their insertion * time on 10 millisecond detail; used for buffer statistics. */ typedef struct insertion_time_elem_t { diff --cc src/or/relay.c index 29dc361,a193ad8..63119cb --- a/src/or/relay.c +++ b/src/or/relay.c @@@ -2146,13 -1902,16 +2146,17 @@@ cell_queue_append(cell_queue_t *queue, /** Append a newly allocated copy of cell to the end of queue */ void -cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell) +cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell, + int wide_circ_ids) { + struct timeval now; - packed_cell_t *copy = packed_cell_copy(cell); + packed_cell_t *copy = packed_cell_copy(cell, wide_circ_ids); + tor_gettimeofday_cached(&now); + copy->inserted_time = (uint32_t)tv_to_msec(&now); + /* Remember the time when this cell was put in the queue. */ + /*XXXX This may be obsoleted by inserted_time */ if (get_options()->CellStatistics) { - struct timeval now; uint32_t added; insertion_time_queue_t *it_queue = queue->insertion_times; if (!it_pool)

1 0

[tor/master] Merge branch 'bug9093_023' into maint-0.2.3
by nickm＠torproject.org 15 Nov '13

15 Nov '13

commit 9e907076025ccd91abfad7fc70c09ba4c9228f82 Merge: 82d8944 1b8ceb8 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Nov 15 15:23:51 2013 -0500 Merge branch 'bug9093_023' into maint-0.2.3 changes/bug9093 | 7 +++++++ src/or/circuitlist.c | 56 +++++++++++++++++++++++++++++++++++++++++--------- src/or/or.h | 5 +++++ src/or/relay.c | 8 ++++++-- 4 files changed, 64 insertions(+), 12 deletions(-)

1 0

[tor/master] Improved circuit queue out-of-memory handler
by nickm＠torproject.org 15 Nov '13

15 Nov '13

commit 1b8ceb83c951f1cdea6b71a615a10d33b8adf2b3 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Nov 7 12:15:30 2013 -0500 Improved circuit queue out-of-memory handler Previously, when we ran low on memory, we'd close whichever circuits had the most queued cells. Now, we close those that have the *oldest* queued cells, on the theory that those are most responsible for us running low on memory, and that those are the least likely to actually drain on their own if we wait a little longer. Based on analysis from a forthcoming paper by Jansen, Tschorsch, Johnson, and Scheuermann. Fixes bug 9093. --- changes/bug9093 | 7 +++++++ src/or/circuitlist.c | 56 +++++++++++++++++++++++++++++++++++++++++--------- src/or/or.h | 5 +++++ src/or/relay.c | 8 ++++++-- 4 files changed, 64 insertions(+), 12 deletions(-) diff --git a/changes/bug9093 b/changes/bug9093 new file mode 100644 index 0000000..06b6cb9 --- /dev/null +++ b/changes/bug9093 @@ -0,0 +1,7 @@ + o Minor features: + - Improve the circuit queue out-of-memory handler. Previously, when + we ran low on memory, we'd close whichever circuits had the most + queued cells. Now, we close those that have the *oldest* queued + cells, on the theory that those are most responsible for us + running low on memory. Based on analysis from a forthcoming paper + by Jansen, Tschorsch, Johnson, and Scheuermann. Fixes bug 9093. \ No newline at end of file diff --git a/src/or/circuitlist.c b/src/or/circuitlist.c index d9ea4d1..6250c11 100644 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@ -1383,25 +1383,56 @@ n_cells_in_circ_queues(const circuit_t *c) return n; } -/** helper to sort a list of circuit_q by total queue lengths, in descending - * order. */ +/** + * Return the age of the oldest cell queued on c, in milliseconds. + * Return 0 if there are no cells queued on c. Requires that now be + * the current time in milliseconds since the epoch, truncated. + * + * This function will return incorrect results if the oldest cell queued on + * the circuit is older than 2**32 msec (about 49 days) old. + */ +static uint32_t +circuit_max_queued_cell_age(const circuit_t *c, uint32_t now) +{ + uint32_t age = 0; + if (c->n_conn_cells.head) + age = now - c->n_conn_cells.head->inserted_time; + + if (! CIRCUIT_IS_ORIGIN(c)) { + const or_circuit_t *orcirc = TO_OR_CIRCUIT((circuit_t*)c); + if (orcirc->p_conn_cells.head) { + uint32_t age2 = now - orcirc->p_conn_cells.head->inserted_time; + if (age2 > age) + return age2; + } + } + return age; +} + +/** Temporary variable for circuits_compare_by_oldest_queued_cell_ This is a + * kludge to work around the fact that qsort doesn't provide a way for + * comparison functions to take an extra argument. */ +static uint32_t circcomp_now_tmp; + +/** Helper to sort a list of circuit_t by age of oldest cell, in descending + * order. Requires that circcomp_now_tmp is set correctly. */ static int -circuits_compare_by_queue_len_(const void **a_, const void **b_) +circuits_compare_by_oldest_queued_cell_(const void **a_, const void **b_) { const circuit_t *a = *a_; const circuit_t *b = *b_; - size_t a_n = n_cells_in_circ_queues(a); - size_t b_n = n_cells_in_circ_queues(b); + uint32_t age_a = circuit_max_queued_cell_age(a, circcomp_now_tmp); + uint32_t age_b = circuit_max_queued_cell_age(b, circcomp_now_tmp); - if (a_n < b_n) + if (age_a < age_b) return 1; - else if (a_n == b_n) + else if (age_a == age_b) return 0; else return -1; } -#define FRACTION_OF_CIRCS_TO_RETAIN_ON_OOM 0.90 +#define FRACTION_OF_CELLS_TO_RETAIN_ON_OOM 0.90 /** We're out of memory for cells, having allocated current_allocation * bytes' worth. Kill the 'worst' circuits until we're under @@ -1414,13 +1445,14 @@ circuits_handle_oom(size_t current_allocation) circuit_t *circ; size_t n_cells_removed=0, n_cells_to_remove; int n_circuits_killed=0; + struct timeval now; log_notice(LD_GENERAL, "We're low on memory. Killing circuits with " "over-long queues. (This behavior is controlled by " "MaxMemInCellQueues.)"); { size_t mem_target = (size_t)(get_options()->MaxMemInCellQueues * - FRACTION_OF_CIRCS_TO_RETAIN_ON_OOM); + FRACTION_OF_CELLS_TO_RETAIN_ON_OOM); size_t mem_to_recover; if (current_allocation <= mem_target) return; @@ -1433,9 +1465,13 @@ circuits_handle_oom(size_t current_allocation) for (circ = global_circuitlist; circ; circ = circ->next) smartlist_add(circlist, circ); + /* Set circcomp_now_tmp so that the sort can work. */ + tor_gettimeofday_cached(&now); + circcomp_now_tmp = (uint32_t)tv_to_msec(&now); + /* This is O(n log n); there are faster algorithms we could use instead. * Let's hope this doesn't happen enough to be in the critical path. */ - smartlist_sort(circlist, circuits_compare_by_queue_len_); + smartlist_sort(circlist, circuits_compare_by_oldest_queued_cell_); /* Okay, now the worst circuits are at the front of the list. Let's mark * them, and reclaim their storage aggressively. */ diff --git a/src/or/or.h b/src/or/or.h index b8f334e..4622391 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -912,8 +912,13 @@ typedef struct var_cell_t { typedef struct packed_cell_t { struct packed_cell_t *next; /**< Next cell queued on this circuit. */ char body[CELL_NETWORK_SIZE]; /**< Cell as packed for network. */ + uint32_t inserted_time; /**< Time (in milliseconds since epoch, with high + * bits truncated) when this cell was inserted. */ } packed_cell_t; +/* XXXX This next structure may be obsoleted by inserted_time in + * packed_cell_t */ + /** Number of cells added to a circuit queue including their insertion * time on 10 millisecond detail; used for buffer statistics. */ typedef struct insertion_time_elem_t { diff --git a/src/or/relay.c b/src/or/relay.c index fda9e89..a193ad8 100644 --- a/src/or/relay.c +++ b/src/or/relay.c @@ -1904,15 +1904,19 @@ cell_queue_append(cell_queue_t *queue, packed_cell_t *cell) void cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell) { + struct timeval now; packed_cell_t *copy = packed_cell_copy(cell); + tor_gettimeofday_cached(&now); + copy->inserted_time = (uint32_t)tv_to_msec(&now); + /* Remember the time when this cell was put in the queue. */ + /*XXXX This may be obsoleted by inserted_time */ if (get_options()->CellStatistics) { - struct timeval now; uint32_t added; insertion_time_queue_t *it_queue = queue->insertion_times; if (!it_pool) it_pool = mp_pool_new(sizeof(insertion_time_elem_t), 1024); - tor_gettimeofday_cached(&now); + #define SECONDS_IN_A_DAY 86400L added = (uint32_t)(((now.tv_sec % SECONDS_IN_A_DAY) * 100L) + ((uint32_t)now.tv_usec / (uint32_t)10000L));

1 0

[tor/maint-0.2.4] Merge remote-tracking branch 'origin/maint-0.2.3' into maint-0.2.4
by nickm＠torproject.org 15 Nov '13

15 Nov '13

1 0

[tor/maint-0.2.4] Merge branch 'bug9093_023' into maint-0.2.3
by nickm＠torproject.org 15 Nov '13

15 Nov '13

1 0

[tor/maint-0.2.4] Improved circuit queue out-of-memory handler
by nickm＠torproject.org 15 Nov '13

15 Nov '13

1 0

[tor/maint-0.2.3] Improved circuit queue out-of-memory handler
by nickm＠torproject.org 15 Nov '13

15 Nov '13

1 0

[tor/maint-0.2.3] Merge branch 'bug9093_023' into maint-0.2.3
by nickm＠torproject.org 15 Nov '13

15 Nov '13

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits November 2013