MaxMemInQueues defends against 375000 circuits in 9 secs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi everybody Another circuit storm, right ? - ---- MaxMemInQueues 2 GB - ---- 2017-09-26_17-55-00: PID JID USERNAME THR PRI NICE SIZE RES STATE C TIME CPU COMMAND 84847 2 256 6 20 0 484M 427M uwait 0 907:02 11.77% tor 2017-09-26_18-59-00: PID JID USERNAME THR PRI NICE SIZE RES STATE C TIME CPU COMMAND 84847 2 256 6 20 0 2437M 2380M uwait 0 917:03 18.26% tor - ---- On Tor 0.3.2.1-alpha 290274dbb5428bc5 Sep 26 18:24:37.000 [notice] Heartbeat: Tor's uptime is 4 days 18:59 hours, with 13714 circuits open. I've sent 2459.48 GB and received 2422.58 GB. Sep 26 18:24:37.000 [notice] Circuit handshake stats since last time: 4635/4635 TAP, 113271/113271 NTor. Sep 26 18:24:37.000 [notice] Since startup, we have initiated 0 v1 connections, 0 v2 connections, 3 v3 connections, and 214559 v4 connections; and received 886 v1 connections, 24624 v2 connections, 60722 v3 connections, and 354411 v4 connections. Sep 26 18:59:28.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Sep 26 18:59:28.000 [notice] Removed 106528 bytes by killing 14408 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. Sep 26 18:59:28.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Sep 26 18:59:28.000 [notice] Removed 528 bytes by killing 7 circuits; 0 circuits remain alive. Also killed 0 non-linked directory connections. ... ************************************************************************ **************** 2500 x removed some bytes by killing between single and 300 circuits (constantly increasing sawtoothy, average 150 circuits) average 150 circuits x 2500 = 375000 circuits avoided in 9 seconds ************************************************************************ **************** ... Sep 26 18:59:37.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Sep 26 18:59:37.000 [notice] Removed 4624 bytes by killing 115 circuits; 0 circuits remain alive. Also killed 1 non-linked directory connections. [] https:// lists.torproject.org/pipermail/tor-relays/2017-July/012624.html - -- Cheers, Felix -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJZyqCOAAoJEF1W24InZUQd/eAQAIK7IDNbnR+E1uTpvVwimUCJ jrSPr5sCXu8t4ly0Tci/tg7mCGTd9xKJvJgUHLei7pd6UmdDJvwajrX08tE/iBC/ uJPLEr69OBBqDICf7E+Y4oQGRz4ICY/4IGeb7lU/My6TzT00gHcs0/0tuww1actm P06uobAeRx+56GCg9hGopLavSOIHsR0bFC3k5kqin79qx40XwNuN6xkFCDni/fHW EO9GazD2iE4zJ3+1lIJsIzaA7fgu+Ack2GcLpaPKMNMwX3DagG/A619HsddOoBfH EFV2ZOhnR9XWytgIfPX4e/gErTHvWPshm0qaO4PzxwAlAoSKygvz497O1HZ3AlM+ fcn+C/fkfCcU2PYVPXKV3vDzupiZ17OJGfRNtAy1EqFLCK6q7xQAp1e3JuRJZBDO 3E/18FJM7JXCUz5StxwB+oAu9LGib50vXbtAOtuLwfX4OC3Nl0Fc/gw0ueYwWIPu QvsU0WNelwaUo3wWs47fJsMsJqjpt0Uywh8yvK809xvqogUFiQBF2ynIp5QzfaJ0 or/7axRHXglkxVaF+q84GFG8LAaspurGauQQFtzUdzl/ljWC963PyrGiOr0kCj/R OIc3TDSDYopFTllXeeVu2bysW7PrzL3vu5rcOHbbVAnUoRukA4Hh6qJFQuTk+Ndb tT8G8jk1QJjRqH/IaTLg =r7/i -----END PGP SIGNATURE-----
On 27 Sep 2017, at 04:46, Felix <zwiebel@quantentunnel.de> wrote:
Signed PGP part Hi everybody
Another circuit storm, right ?
----
MaxMemInQueues 2 GB
----
2017-09-26_17-55-00: PID JID USERNAME THR PRI NICE SIZE RES STATE C TIME CPU COMMAND 84847 2 256 6 20 0 484M 427M uwait 0 907:02 11.77% tor
2017-09-26_18-59-00: PID JID USERNAME THR PRI NICE SIZE RES STATE C TIME CPU COMMAND 84847 2 256 6 20 0 2437M 2380M uwait 0 917:03 18.26% tor
----
On Tor 0.3.2.1-alpha 290274dbb5428bc5
Sep 26 18:24:37.000 [notice] Heartbeat: Tor's uptime is 4 days 18:59 hours, with 13714 circuits open. I've sent 2459.48 GB and received 2422.58 GB. Sep 26 18:24:37.000 [notice] Circuit handshake stats since last time: 4635/4635 TAP, 113271/113271 NTor. Sep 26 18:24:37.000 [notice] Since startup, we have initiated 0 v1 connections, 0 v2 connections, 3 v3 connections, and 214559 v4 connections; and received 886 v1 connections, 24624 v2 connections, 60722 v3 connections, and 354411 v4 connections. Sep 26 18:59:28.000 [notice] We're low on memory. Killing circuits with over-long queues. (This behavior is controlled by MaxMemInQueues.) Sep 26 18:59:28.000 [notice] Removed 106528 bytes by killing 14408 circuits; 0 circuits remain alive
If you have 0 circuits remaining, it looks like something else is taking up all the RAM. (It might not be circuits.) T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n ------------------------------------------------------------------------
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Am 27-Sep-17 um 08:19 schrieb teor:
On 27 Sep 2017, at 04:46, Felix <zwiebel@quantentunnel.de> wrote:
Sep 26 18:59:28.000 [notice] Removed 106528 bytes by killing 14408 circuits; 0 circuits remain alive
If you have 0 circuits remaining, it looks like something else is taking up all the RAM. (It might not be circuits.)
Thanks T. I had to restart tor because it was idling. That's why I changed the title. No more log entries, no tor cpu business. I looked into the log with open eyes and found an error message at the end (which I overlooked because I thought it was again this trac 23551 message): Sep 26 18:59:37.000 [err] tor_assertion_failed_: Bug: src/common/buffers.c:651: buf_flush_to_socket: Assertion *buf_flushlen <= buf->datalen failed; aborting. Sep 26 18:59:37.000 [err] Bug: Assertion *buf_flushlen <= buf->datalen failed in buf_flush_to_socket at src/common/buffers.c:651. Stack trace: 0x11a2bf8 <log_backtrace+0x48> at /usr/local/bin/tor 0x11bd6f7 <tor_assertion_failed_+0x97> at /usr/local/bin/tor 0x11a421a <buf_flush_to_socket+0x22a> at /usr/local/bin/tor 0x10774eb <tor_main+0x24bb> at /usr/local/bin/tor 0x801b37d3e <event_base_assert_ok_nolock_+0xbce> at /usr/local/lib/libevent-2.1.so.6 0x801b33cfe <event_base_loop+0x50e> at /usr/local/lib/libevent-2.1.so.6 0x1072d15 <do_main_loop+0x575> at /usr/local/bin/tor 0x1075119 <tor_main+0xe9> at /usr/local/bin/tor 0x1070b69 <main+0x19> at /usr/local/bin/tor 0x1070a61 <_start+0x1a1> at /usr/local/bin/tor Sorry for the confusion. - ---- Sep 26 18:59:37.000 [err] tor_assertion_failed_: Bug: src/common/buffers.c:651: buf_flush_to_socket: Assertion *buf_flushlen <= buf->datalen failed; aborting. (on Tor 0.3.2.1-alpha 290274dbb5428bc5 ) Sep 26 18:59:37.000 [err] Bug: Assertion *buf_flushlen <= buf->datalen failed in buf_flush_to_socket at src/common/buffers.c:651. Stack trace: (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x11a2bf8 <log_backtrace+0x48> at /usr/local/bin/tor (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x11bd6f7 <tor_assertion_failed_+0x97> at /usr/local/bin/tor (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x11a421a <buf_flush_to_socket+0x22a> at /usr/local/bin/tor (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x10774eb <tor_main+0x24bb> at /usr/local/bin/tor (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x801b37d3e <event_base_assert_ok_nolock_+0xbce> at /usr/local/lib/libevent-2.1.so.6 (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x801b33cfe <event_base_loop+0x50e> at /usr/local/lib/libevent-2.1.so.6 (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x1072d15 <do_main_loop+0x575> at /usr/local/bin/tor (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x1075119 <tor_main+0xe9> at /usr/local/bin/tor (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x1070b69 <main+0x19> at /usr/local/bin/tor (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 18:59:37.000 [err] Bug: 0x1070a61 <_start+0x1a1> at /usr/local/bin/tor (on Tor 0.3.2.1-alpha 290274dbb5428bc5) Sep 26 21:25:31.000 [notice] Tor 0.3.2.1-alpha (git-290274dbb5428bc5) opening log file. - ---- - -- Cheers, Felix -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJZzWWkAAoJEF1W24InZUQdfJ4QAO8GECYSo0MEiRThZdcX5n+R /BqpndmZm9tBKOTIBrIHlsfW+V6ZG1FN8jPHsBWyFu1IzQcv7XO61tqwXK6dDRz8 kSJqGZeoIXacnQu9Pc+TZ8DRXsWB9EThcMd6vxlf/Mny53CoaZC6QR3hECu8U5Pt CPZ+jST54HEKMMcYQ79LAMmtH6ySlu2Ma6LpMvxJGu0mG4ZgjqFyLo/CBOJ0AoZS qXc42IK/+iQiZuPcUiceYWdzTWjpOjd7VlgeX0+CqAO0OP4m21tuOFWewFMqb8Nf u4A+9m4R3uyPq2h53PCjyxlagKcF5dG5Mm7g17rf16MKDh6WECcUAco74R151J6+ AevXEdkHntEXnmMbk1MHRsmlw8bFRrOCE6I8ZfJB/ubNo01Cxe+N77IWSTvIE7Aj pxplPgfu4+ay9rksVWm/rFRkY/saaNWj8wgle9jkblHZoHuzv2Kf+azvYNdKfcwI wlMUq1miKRPxMG5eu2MBrCpPodtrc3USxzEwQIOKSBxAtKw+gnugqS90Pyn2pGrn Tqsglrm7NqivulHxPH/k0lJycWT1ABvjfu0CeCNw3xWETpXN4izEIMHSxciRpdiu S4i7ZMLIJ3Hy19Vmw8pmxjWMqRM4VEYMpxHsfzMJL5OyFrsI1tmOQi/RahzaZXfV 19nt/thdR5leEMMLIJlN =iHTK -----END PGP SIGNATURE-----
On Thu, Sep 28, 2017 at 11:12:05PM +0200, Felix wrote:
Sep 26 18:59:37.000 [err] tor_assertion_failed_: Bug: src/common/buffers.c:651: buf_flush_to_socket: Assertion *buf_flushlen <= buf->datalen failed; aborting.
Neat! Can you open a ticket on https://bugs.torproject.org/ ? With as many details as you can. Thanks, --Roger
participants (3)
-
Felix -
Roger Dingledine -
teor