December 2017 - tor-commits - lists.torproject.org

[tor/maint-0.3.2] Increment version to 0.3.2.8-rc
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit 1a777996651bcc63e3d91391c27c5d6ec46c8015 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 21 12:42:00 2017 -0500 Increment version to 0.3.2.8-rc --- configure.ac | 2 +- contrib/win32build/tor-mingw.nsi.in | 2 +- src/win32/orconfig.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/configure.ac b/configure.ac index 608d6864b..f8c888445 100644 --- a/configure.ac +++ b/configure.ac @@ -4,7 +4,7 @@ dnl Copyright (c) 2007-2017, The Tor Project, Inc. dnl See LICENSE for licensing information AC_PREREQ([2.63]) -AC_INIT([tor],[0.3.2.7-rc-dev]) +AC_INIT([tor],[0.3.2.8-rc]) AC_CONFIG_SRCDIR([src/or/main.c]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/contrib/win32build/tor-mingw.nsi.in b/contrib/win32build/tor-mingw.nsi.in index e9099b108..40459a951 100644 --- a/contrib/win32build/tor-mingw.nsi.in +++ b/contrib/win32build/tor-mingw.nsi.in @@ -8,7 +8,7 @@ !include "LogicLib.nsh" !include "FileFunc.nsh" !insertmacro GetParameters -!define VERSION "0.3.2.7-rc-dev" +!define VERSION "0.3.2.8-rc" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" diff --git a/src/win32/orconfig.h b/src/win32/orconfig.h index 7def010cd..674c324f8 100644 --- a/src/win32/orconfig.h +++ b/src/win32/orconfig.h @@ -218,7 +218,7 @@ #define USING_TWOS_COMPLEMENT /* Version number of package */ -#define VERSION "0.3.2.7-rc-dev" +#define VERSION "0.3.2.8-rc"

1 0

[tor/release-0.3.2] Increment version to 0.3.2.8-rc
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit 1a777996651bcc63e3d91391c27c5d6ec46c8015 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 21 12:42:00 2017 -0500 Increment version to 0.3.2.8-rc --- configure.ac | 2 +- contrib/win32build/tor-mingw.nsi.in | 2 +- src/win32/orconfig.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/configure.ac b/configure.ac index 608d6864b..f8c888445 100644 --- a/configure.ac +++ b/configure.ac @@ -4,7 +4,7 @@ dnl Copyright (c) 2007-2017, The Tor Project, Inc. dnl See LICENSE for licensing information AC_PREREQ([2.63]) -AC_INIT([tor],[0.3.2.7-rc-dev]) +AC_INIT([tor],[0.3.2.8-rc]) AC_CONFIG_SRCDIR([src/or/main.c]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/contrib/win32build/tor-mingw.nsi.in b/contrib/win32build/tor-mingw.nsi.in index e9099b108..40459a951 100644 --- a/contrib/win32build/tor-mingw.nsi.in +++ b/contrib/win32build/tor-mingw.nsi.in @@ -8,7 +8,7 @@ !include "LogicLib.nsh" !include "FileFunc.nsh" !insertmacro GetParameters -!define VERSION "0.3.2.7-rc-dev" +!define VERSION "0.3.2.8-rc" !define INSTALLER "tor-${VERSION}-win32.exe" !define WEBSITE "https://www.torproject.org/" !define LICENSE "LICENSE" diff --git a/src/win32/orconfig.h b/src/win32/orconfig.h index 7def010cd..674c324f8 100644 --- a/src/win32/orconfig.h +++ b/src/win32/orconfig.h @@ -218,7 +218,7 @@ #define USING_TWOS_COMPLEMENT /* Version number of package */ -#define VERSION "0.3.2.7-rc-dev" +#define VERSION "0.3.2.8-rc"

1 0

[tor/master] Merge branch 'maint-0.3.2'
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit eb00840ab88f07820397e00cdc2a62b87a0f52cd Merge: 7b9e79095 1a7779966 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 21 12:42:11 2017 -0500 Merge branch 'maint-0.3.2' "ours" merge to avoid version bump.

1 0

[tor/release-0.3.2] start an 0.3.2.8 changelog
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit 9be4b91760d5a5b5a15abe69d0015c3016c79355 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 21 12:40:11 2017 -0500 start an 0.3.2.8 changelog --- ChangeLog | 44 ++++++++++++++++++++++++++++++++++++++++++++ changes/bug24665 | 6 ------ changes/bug24666 | 7 ------- changes/bug24671 | 6 ------ changes/geoip-2017-12-06 | 4 ---- changes/ticket24425 | 4 ---- 6 files changed, 44 insertions(+), 27 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7729d2435..073822d4b 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,47 @@ +Changes in version 0.3.2.8-rc - 2017-12-21 + Tor 0.3.2.8-rc fixes a pair of bugs in the KIST and KISTLite schedulers + that had led servers under heavy load to overload their outgoing + connections. All relay operators running earlier 0.3.2.x versions should + upgrade. This version also includes a mitigation for over-full DESTROY + queues leading to out-of-memory conditions: if it works, we will soon + backport it to earlier release series. + + This is the second release candidate in the 0.3.2 series. If we find no + new bugs or regression here, then the first stable 0.3.2 release will + be nearly identical to this. + + o Major bugfixes (KIST, scheduler): + - The KIST scheduler did not correctly account for data already enqueued + in each connection's send socket buffer, particularly in cases when the + TCP/IP congestion window was reduced between scheduler calls. This + situation lead to excessive per-connection buffering in the kernel, and + a potential memory DoS. Fixes bug 24665; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (scheduler, KIST): + - Use a sane write limit for KISTLite when writing onto a connection + buffer instead of using INT_MAX and shoving as much as it can. Because + the OOM handler cleans up circuit queues, we are better off at keeping + them in that queue instead of the connection's buffer. Fixes bug 24671; + bugfix on 0.3.2.1-alpha. + + o Minor features (geoip): + - Update geoip and geoip6 to the December 6 2017 Maxmind GeoLite2 + Country database. + + o Minor bugfixes (hidden service v3): + - Bump hsdir_spread_store parameter from 3 to 4 in order to increase the + probability of reaching a service for a client missing microdescriptors. + Fixes bug 24425; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (memory usage): + - When queuing DESTROY cells on a channel, only queue the + circuit-id and reason fields: not the entire 514-byte + cell. This fix should help mitigate any bugs or attacks that + fill up these queues, and free more RAM for other uses. Fixes + bug 24666; bugfix on 0.2.5.1-alpha. + + + Changes in version 0.3.2.7-rc - 2017-12-14 Tor 0.3.2.7-rc fixes various bugs in earlier versions of Tor, including some that could affect reliability or correctness. diff --git a/changes/bug24665 b/changes/bug24665 deleted file mode 100644 index f950d9dd0..000000000 --- a/changes/bug24665 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes (KIST, scheduler): - - The KIST scheduler did not correctly account for data already enqueued - in each connection's send socket buffer, particularly in cases when the - TCP/IP congestion window was reduced between scheduler calls. This - situation lead to excessive per-connection buffering in the kernel, and - a potential memory DoS. Fixes bug 24665; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug24666 b/changes/bug24666 deleted file mode 100644 index 830775f5f..000000000 --- a/changes/bug24666 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (memory usage): - - - When queuing DESTROY cells on a channel, only queue the - circuit-id and reason fields: not the entire 514-byte - cell. This fix should help mitigate any bugs or attacks that - fill up these queues, and free more RAM for other uses. Fixes - bug 24666; bugfix on 0.2.5.1-alpha. diff --git a/changes/bug24671 b/changes/bug24671 deleted file mode 100644 index 34d09e704..000000000 --- a/changes/bug24671 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (scheduler, KIST): - - Use a sane write limit for KISTLite when writing onto a connection - buffer instead of using INT_MAX and shoving as much as it can. Because - the OOM handler cleans up circuit queues, we are better off at keeping - them in that queue instead of the connection's buffer. Fixes bug 24671; - bugfix on 0.3.2.1-alpha. diff --git a/changes/geoip-2017-12-06 b/changes/geoip-2017-12-06 deleted file mode 100644 index ae4fb1149..000000000 --- a/changes/geoip-2017-12-06 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the December 6 2017 Maxmind GeoLite2 - Country database. - diff --git a/changes/ticket24425 b/changes/ticket24425 deleted file mode 100644 index aa6f082bc..000000000 --- a/changes/ticket24425 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (hidden service v3): - - Bump hsdir_spread_store parameter from 3 to 4 in order to increase the - probability of reaching a service for a client missing microdescriptors. - Fixes bug 24425; bugfix on 0.3.2.1-alpha.

1 0

[tor/release-0.3.2] reflow the changelog
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit afacaa02a54dcdf0751529dbd8420b3e09009550 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 21 12:40:32 2017 -0500 reflow the changelog --- ChangeLog | 60 ++++++++++++++++++++++++++++++------------------------------ 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/ChangeLog b/ChangeLog index 073822d4b..77d763b80 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,45 +1,45 @@ Changes in version 0.3.2.8-rc - 2017-12-21 - Tor 0.3.2.8-rc fixes a pair of bugs in the KIST and KISTLite schedulers - that had led servers under heavy load to overload their outgoing - connections. All relay operators running earlier 0.3.2.x versions should - upgrade. This version also includes a mitigation for over-full DESTROY - queues leading to out-of-memory conditions: if it works, we will soon - backport it to earlier release series. - - This is the second release candidate in the 0.3.2 series. If we find no - new bugs or regression here, then the first stable 0.3.2 release will - be nearly identical to this. + Tor 0.3.2.8-rc fixes a pair of bugs in the KIST and KISTLite + schedulers that had led servers under heavy load to overload their + outgoing connections. All relay operators running earlier 0.3.2.x + versions should upgrade. This version also includes a mitigation for + over-full DESTROY queues leading to out-of-memory conditions: if it + works, we will soon backport it to earlier release series. - o Major bugfixes (KIST, scheduler): - - The KIST scheduler did not correctly account for data already enqueued - in each connection's send socket buffer, particularly in cases when the - TCP/IP congestion window was reduced between scheduler calls. This - situation lead to excessive per-connection buffering in the kernel, and - a potential memory DoS. Fixes bug 24665; bugfix on 0.3.2.1-alpha. + This is the second release candidate in the 0.3.2 series. If we find + no new bugs or regression here, then the first stable 0.3.2 release + will be nearly identical to this. - o Minor bugfixes (scheduler, KIST): - - Use a sane write limit for KISTLite when writing onto a connection - buffer instead of using INT_MAX and shoving as much as it can. Because - the OOM handler cleans up circuit queues, we are better off at keeping - them in that queue instead of the connection's buffer. Fixes bug 24671; - bugfix on 0.3.2.1-alpha. + o Major bugfixes (KIST, scheduler): + - The KIST scheduler did not correctly account for data already + enqueued in each connection's send socket buffer, particularly in + cases when the TCP/IP congestion window was reduced between + scheduler calls. This situation lead to excessive per-connection + buffering in the kernel, and a potential memory DoS. Fixes bug + 24665; bugfix on 0.3.2.1-alpha. o Minor features (geoip): - Update geoip and geoip6 to the December 6 2017 Maxmind GeoLite2 Country database. o Minor bugfixes (hidden service v3): - - Bump hsdir_spread_store parameter from 3 to 4 in order to increase the - probability of reaching a service for a client missing microdescriptors. - Fixes bug 24425; bugfix on 0.3.2.1-alpha. + - Bump hsdir_spread_store parameter from 3 to 4 in order to increase + the probability of reaching a service for a client missing + microdescriptors. Fixes bug 24425; bugfix on 0.3.2.1-alpha. o Minor bugfixes (memory usage): - - When queuing DESTROY cells on a channel, only queue the - circuit-id and reason fields: not the entire 514-byte - cell. This fix should help mitigate any bugs or attacks that - fill up these queues, and free more RAM for other uses. Fixes - bug 24666; bugfix on 0.2.5.1-alpha. + - When queuing DESTROY cells on a channel, only queue the circuit-id + and reason fields: not the entire 514-byte cell. This fix should + help mitigate any bugs or attacks that fill up these queues, and + free more RAM for other uses. Fixes bug 24666; bugfix + on 0.2.5.1-alpha. + o Minor bugfixes (scheduler, KIST): + - Use a sane write limit for KISTLite when writing onto a connection + buffer instead of using INT_MAX and shoving as much as it can. + Because the OOM handler cleans up circuit queues, we are better + off at keeping them in that queue instead of the connection's + buffer. Fixes bug 24671; bugfix on 0.3.2.1-alpha. Changes in version 0.3.2.7-rc - 2017-12-14

1 0

[tor/maint-0.3.2] sched: Use lower layer cell limit with KISTLite
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit fdfa4a5a140a2d82ebd92e980a577cbcecd2180b Author: David Goulet <dgoulet(a)torproject.org> Date: Wed Dec 20 14:14:02 2017 -0500 sched: Use lower layer cell limit with KISTLite Instead of using INT_MAX as a write limit for KISTLite, use the lower layer limit which is using the specialized num_cells_writeable() of the channel that will down the line check the connection's outbuf and limit it to 32KB (OR_CONN_HIGHWATER). That way we don't take the chance of bloating the connection's outbuf and we keep the cells in the circuit queue which our OOM handler can take care of, not the outbuf. Finally, this commit adds a log_debug() in the update socket information function of KIST so we can get the socket information in debug. Fixes #24671 Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- changes/bug24671 | 6 ++++++ src/or/scheduler_kist.c | 22 ++++++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/changes/bug24671 b/changes/bug24671 new file mode 100644 index 000000000..34d09e704 --- /dev/null +++ b/changes/bug24671 @@ -0,0 +1,6 @@ + o Minor bugfixes (scheduler, KIST): + - Use a sane write limit for KISTLite when writing onto a connection + buffer instead of using INT_MAX and shoving as much as it can. Because + the OOM handler cleans up circuit queues, we are better off at keeping + them in that queue instead of the connection's buffer. Fixes bug 24671; + bugfix on 0.3.2.1-alpha. diff --git a/src/or/scheduler_kist.c b/src/or/scheduler_kist.c index 9acd89b37..b7a4ee021 100644 --- a/src/or/scheduler_kist.c +++ b/src/or/scheduler_kist.c @@ -298,13 +298,18 @@ update_socket_info_impl, (socket_table_ent_t *ent)) fallback: /* If all of a sudden we don't have kist support, we just zero out all the - * variables for this socket since we don't know what they should be. - * We also effectively allow the socket write as much as it wants to the - * kernel, effectively returning it to vanilla scheduler behavior. Writes - * are still limited by the lower layers of Tor: socket blocking, full - * outbuf, etc. */ + * variables for this socket since we don't know what they should be. We + * also allow the socket to write as much as it can from the estimated + * number of cells the lower layer can accept, effectively returning it to + * Vanilla scheduler behavior. */ ent->cwnd = ent->unacked = ent->mss = ent->notsent = 0; - ent->limit = INT_MAX; + /* This function calls the specialized channel object (currently channeltls) + * and ask how many cells it can write on the outbuf which we then multiply + * by the size of the cells for this channel. The cast is because this + * function requires a non-const channel object, meh. */ + ent->limit = channel_num_cells_writeable((channel_t *) ent->chan) * + (get_cell_network_size(ent->chan->wide_circ_ids) + + TLS_PER_CELL_OVERHEAD); } /* Given a socket that isn't in the table, add it. @@ -398,6 +403,11 @@ update_socket_info(socket_table_t *table, const channel_t *chan) return; // Whelp. Entry didn't exist for some reason so nothing to do. } update_socket_info_impl(ent); + log_debug(LD_SCHED, "chan=%" PRIu64 " updated socket info, limit: %" PRIu64 + ", cwnd: %" PRIu32 ", unacked: %" PRIu32 + ", notsent: %" PRIu32 ", mss: %" PRIu32, + ent->chan->global_identifier, ent->limit, ent->cwnd, ent->unacked, + ent->notsent, ent->mss); } /* Increment the channel's socket written value by the number of bytes. */

1 0

[tor/master] Merge remote-tracking branch 'dgoulet/bug24671_032_01' into maint-0.3.2
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit 6cd567d7974e6bf55874f3a6d907feb050144095 Merge: 84adb9fcc fdfa4a5a1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 21 11:13:33 2017 -0500 Merge remote-tracking branch 'dgoulet/bug24671_032_01' into maint-0.3.2 changes/bug24671 | 6 ++++++ src/or/scheduler_kist.c | 22 ++++++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-)

1 0

[tor/master] sched: Use lower layer cell limit with KISTLite
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit fdfa4a5a140a2d82ebd92e980a577cbcecd2180b Author: David Goulet <dgoulet(a)torproject.org> Date: Wed Dec 20 14:14:02 2017 -0500 sched: Use lower layer cell limit with KISTLite Instead of using INT_MAX as a write limit for KISTLite, use the lower layer limit which is using the specialized num_cells_writeable() of the channel that will down the line check the connection's outbuf and limit it to 32KB (OR_CONN_HIGHWATER). That way we don't take the chance of bloating the connection's outbuf and we keep the cells in the circuit queue which our OOM handler can take care of, not the outbuf. Finally, this commit adds a log_debug() in the update socket information function of KIST so we can get the socket information in debug. Fixes #24671 Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- changes/bug24671 | 6 ++++++ src/or/scheduler_kist.c | 22 ++++++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/changes/bug24671 b/changes/bug24671 new file mode 100644 index 000000000..34d09e704 --- /dev/null +++ b/changes/bug24671 @@ -0,0 +1,6 @@ + o Minor bugfixes (scheduler, KIST): + - Use a sane write limit for KISTLite when writing onto a connection + buffer instead of using INT_MAX and shoving as much as it can. Because + the OOM handler cleans up circuit queues, we are better off at keeping + them in that queue instead of the connection's buffer. Fixes bug 24671; + bugfix on 0.3.2.1-alpha. diff --git a/src/or/scheduler_kist.c b/src/or/scheduler_kist.c index 9acd89b37..b7a4ee021 100644 --- a/src/or/scheduler_kist.c +++ b/src/or/scheduler_kist.c @@ -298,13 +298,18 @@ update_socket_info_impl, (socket_table_ent_t *ent)) fallback: /* If all of a sudden we don't have kist support, we just zero out all the - * variables for this socket since we don't know what they should be. - * We also effectively allow the socket write as much as it wants to the - * kernel, effectively returning it to vanilla scheduler behavior. Writes - * are still limited by the lower layers of Tor: socket blocking, full - * outbuf, etc. */ + * variables for this socket since we don't know what they should be. We + * also allow the socket to write as much as it can from the estimated + * number of cells the lower layer can accept, effectively returning it to + * Vanilla scheduler behavior. */ ent->cwnd = ent->unacked = ent->mss = ent->notsent = 0; - ent->limit = INT_MAX; + /* This function calls the specialized channel object (currently channeltls) + * and ask how many cells it can write on the outbuf which we then multiply + * by the size of the cells for this channel. The cast is because this + * function requires a non-const channel object, meh. */ + ent->limit = channel_num_cells_writeable((channel_t *) ent->chan) * + (get_cell_network_size(ent->chan->wide_circ_ids) + + TLS_PER_CELL_OVERHEAD); } /* Given a socket that isn't in the table, add it. @@ -398,6 +403,11 @@ update_socket_info(socket_table_t *table, const channel_t *chan) return; // Whelp. Entry didn't exist for some reason so nothing to do. } update_socket_info_impl(ent); + log_debug(LD_SCHED, "chan=%" PRIu64 " updated socket info, limit: %" PRIu64 + ", cwnd: %" PRIu32 ", unacked: %" PRIu32 + ", notsent: %" PRIu32 ", mss: %" PRIu32, + ent->chan->global_identifier, ent->limit, ent->cwnd, ent->unacked, + ent->notsent, ent->mss); } /* Increment the channel's socket written value by the number of bytes. */

1 0

[tor/maint-0.3.2] Merge remote-tracking branch 'dgoulet/bug24671_032_01' into maint-0.3.2
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit 6cd567d7974e6bf55874f3a6d907feb050144095 Merge: 84adb9fcc fdfa4a5a1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 21 11:13:33 2017 -0500 Merge remote-tracking branch 'dgoulet/bug24671_032_01' into maint-0.3.2 changes/bug24671 | 6 ++++++ src/or/scheduler_kist.c | 22 ++++++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-)

1 0

[tor/master] Merge branch 'maint-0.3.2'
by nickm＠torproject.org 21 Dec '17

21 Dec '17

commit 2b8a06a2ef23350616df755a32b0e5b87d463314 Merge: 2f0d57db5 6cd567d79 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Dec 21 11:16:00 2017 -0500 Merge branch 'maint-0.3.2' changes/bug24671 | 6 ++++++ src/or/scheduler_kist.c | 22 ++++++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-)

1 0