Hi list
I have upgraded recently my relays (totorbe*) to CentOS8 + tor 0.4.2.7 and they all seemed to run properly This morning, I just noticed a weird behavior on one of them (no idea since when this is going on)
When starting tor, it goes to the point of IP identification and then starts shutting down ("Interrupt: we have stopped accepting new connections...") It attempts then to restart immediately and loops start --> interrupt --> start etc...
Any idea where too start investigating??
[root@totorbex tor]# cat notices.log Apr 06 10:32:47.000 [notice] Starting with guard context "default" Apr 06 10:32:47.000 [notice] Signaled readiness to systemd Apr 06 10:32:47.000 [notice] Interrupt: we have stopped accepting new connections, and will shut down in 30 seconds. Interrupt again to exit now. Apr 06 10:32:47.000 [notice] Guessed our IP address as 82.212.170.79 (source: 128.31.0.34). Apr 06 10:32:48.000 [notice] Delaying directory fetches: We are hibernating or shutting down. Apr 06 10:33:17.000 [notice] Clean shutdown finished. Exiting. Apr 06 10:33:18.000 [notice] Tor 0.4.2.7 opening log file. Apr 06 10:33:18.719 [notice] Tor 0.4.2.7 running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1c, Zlib 1.2.11, Liblzma 5.2.4, and Libzstd 1.4.4. Apr 06 10:33:18.719 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning Apr 06 10:33:18.719 [notice] Read configuration file "/usr/share/tor/defaults-torrc". Apr 06 10:33:18.719 [notice] Read configuration file "/etc/tor/torrc". Apr 06 10:33:18.723 [notice] Based on detected system memory, MaxMemInQueues is set to 5791 MB. You can override this by setting MaxMemInQueues by hand. Apr 06 10:33:18.724 [notice] Opening Socks listener on 127.0.0.1:9050 Apr 06 10:33:18.725 [notice] Opened Socks listener on 127.0.0.1:9050 Apr 06 10:33:18.725 [notice] Opening Control listener on 127.0.0.1:9051 Apr 06 10:33:18.725 [notice] Opened Control listener on 127.0.0.1:9051 Apr 06 10:33:18.725 [notice] Opening OR listener on 0.0.0.0:9001 Apr 06 10:33:18.725 [notice] Opened OR listener on 0.0.0.0:9001 Apr 06 10:33:18.725 [notice] Opening Directory listener on 0.0.0.0:9030 Apr 06 10:33:18.725 [notice] Opened Directory listener on 0.0.0.0:9030 Apr 06 10:33:18.000 [notice] Not disabling debugger attaching for unprivileged users. Apr 06 10:33:20.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip. Apr 06 10:33:20.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6. Apr 06 10:33:20.000 [notice] Configured to measure statistics. Look for the *-stats files that will first be written to the data directory in 24 hours from now. Apr 06 10:33:21.000 [notice] Your Tor server's identity key fingerprint is 'TotorBEx E514A3818BC87E87B0B82B6494672F69CED5350D' Apr 06 10:33:21.000 [notice] Bootstrapped 0% (starting): Starting Apr 06 10:34:00.000 [notice] Starting with guard context "default" Apr 06 10:34:00.000 [notice] Signaled readiness to systemd Apr 06 10:34:00.000 [notice] Interrupt: we have stopped accepting new connections, and will shut down in 30 seconds. Interrupt again to exit now. Apr 06 10:34:00.000 [notice] Guessed our IP address as 82.212.170.79 (source: 128.31.0.34). Apr 06 10:34:01.000 [notice] Delaying directory fetches: We are hibernating or shutting down. Apr 06 10:34:30.000 [notice] Clean shutdown finished. Exiting. Apr 06 10:34:31.000 [notice] Tor 0.4.2.7 opening log file. Apr 06 10:34:31.721 [notice] Tor 0.4.2.7 running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1c, Zlib 1.2.11, Liblzma 5.2.4, and Libzstd 1.4.4. Apr 06 10:34:31.721 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning Apr 06 10:34:31.721 [notice] Read configuration file "/usr/share/tor/defaults-torrc". Apr 06 10:34:31.721 [notice] Read configuration file "/etc/tor/torrc". Apr 06 10:34:31.725 [notice] Based on detected system memory, MaxMemInQueues is set to 5791 MB. You can override this by setting MaxMemInQueues by hand. Apr 06 10:34:31.726 [notice] Opening Socks listener on 127.0.0.1:9050 Apr 06 10:34:31.726 [notice] Opened Socks listener on 127.0.0.1:9050 Apr 06 10:34:31.726 [notice] Opening Control listener on 127.0.0.1:9051 Apr 06 10:34:31.727 [notice] Opened Control listener on 127.0.0.1:9051 Apr 06 10:34:31.727 [notice] Opening OR listener on 0.0.0.0:9001 Apr 06 10:34:31.727 [notice] Opened OR listener on 0.0.0.0:9001 Apr 06 10:34:31.727 [notice] Opening Directory listener on 0.0.0.0:9030 Apr 06 10:34:31.727 [notice] Opened Directory listener on 0.0.0.0:9030 Apr 06 10:34:31.000 [notice] Not disabling debugger attaching for unprivileged users. Apr 06 10:34:33.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip. Apr 06 10:34:33.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6. Apr 06 10:34:33.000 [notice] Configured to measure statistics. Look for the *-stats files that will first be written to the data directory in 24 hours from now. Apr 06 10:34:34.000 [notice] Your Tor server's identity key fingerprint is 'TotorBEx E514A3818BC87E87B0B82B6494672F69CED5350D' Apr 06 10:34:34.000 [notice] Bootstrapped 0% (starting): Starting Apr 06 10:35:13.000 [notice] Starting with guard context "default" Apr 06 10:35:13.000 [notice] Signaled readiness to systemd Apr 06 10:35:13.000 [notice] Interrupt: we have stopped accepting new connections, and will shut down in 30 seconds. Interrupt again to exit now.
On Mon, Apr 06, 2020 at 11:01:09AM +0200, Totor be wrote:
I have upgraded recently my relays (totorbe*) to CentOS8 + tor 0.4.2.7 and they all seemed to run properly This morning, I just noticed a weird behavior on one of them (no idea since when this is going on)
When starting tor, it goes to the point of IP identification and then starts shutting down ("Interrupt: we have stopped accepting new connections...") It attempts then to restart immediately and loops start --> interrupt --> start etc...
Any idea where too start investigating??
Hm!
It sounds like whatever package you have auto-starts Tor if it notices that it's not running. That's not unreasonable, but also it ought to have some error count where it only tries to restart it a certain number of times per timeframe. This is something that either the package's init script should do, or that systemd's settings should handle.
The bigger mystery for you is: why does Tor keep deciding to exit? It looks like something is sending it the equivalent of a ^C. What is doing that? This is the main thing to investigate.
Maybe you are running some external tool like nyx, or your own script based on stem, which is killing it somehow?
Maybe systemd is somehow deciding that after some timeout it needs to die? I find it interesting that you get these two lines right after each other each time:
Apr 06 10:35:13.000 [notice] Signaled readiness to systemd Apr 06 10:35:13.000 [notice] Interrupt
It's as though systemd is misinterpreting the signal from Tor, and deciding to kill it rather than be happy.
So: I suspect "bug in centos package" as a good place to investigate. Maybe there is already a ticket open in your packaging system?
Be sure also to check other more general issues like "do you have enough disk space?"
--Roger
Hi Roger,
Some quick facts - tor is enabled to auto-start when the system boots - I do have nyx on that box, but I just run it every now and then (i.e. it does not run permanently) - there is nothing else than tor on the box - disc space is not an issue: 45G free on /root, 700M free on /boot and 53G free on /home
The other things to investigate are way beyond my linux competences My best bet would be to perform a fresh install (OS+tor)
Thanks!
On Mon, Apr 6, 2020 at 11:11 AM Roger Dingledine arma@torproject.org wrote:
On Mon, Apr 06, 2020 at 11:01:09AM +0200, Totor be wrote:
I have upgraded recently my relays (totorbe*) to CentOS8 + tor 0.4.2.7
and
they all seemed to run properly This morning, I just noticed a weird behavior on one of them (no idea
since
when this is going on)
When starting tor, it goes to the point of IP identification and then starts shutting down ("Interrupt: we have stopped accepting new connections...") It attempts then to restart immediately and loops start --> interrupt --> start etc...
Any idea where too start investigating??
Hm!
It sounds like whatever package you have auto-starts Tor if it notices that it's not running. That's not unreasonable, but also it ought to have some error count where it only tries to restart it a certain number of times per timeframe. This is something that either the package's init script should do, or that systemd's settings should handle.
The bigger mystery for you is: why does Tor keep deciding to exit? It looks like something is sending it the equivalent of a ^C. What is doing that? This is the main thing to investigate.
Maybe you are running some external tool like nyx, or your own script based on stem, which is killing it somehow?
Maybe systemd is somehow deciding that after some timeout it needs to die? I find it interesting that you get these two lines right after each other each time:
Apr 06 10:35:13.000 [notice] Signaled readiness to systemd Apr 06 10:35:13.000 [notice] Interrupt
It's as though systemd is misinterpreting the signal from Tor, and deciding to kill it rather than be happy.
So: I suspect "bug in centos package" as a good place to investigate. Maybe there is already a ticket open in your packaging system?
Be sure also to check other more general issues like "do you have enough disk space?"
--Roger
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On 06.04.2020 11:01, Totor be wrote:
When starting tor, it goes to the point of IP identification and then starts shutting down ("Interrupt: we have stopped accepting new connections...") It attempts then to restart immediately and loops start --> interrupt --> start etc...
Any idea where too start investigating??
Apr 06 10:33:18.724 [notice] Opening Socks listener on 127.0.0.1:9050 Apr 06 10:33:18.725 [notice] Opened Socks listener on 127.0.0.1:9050
^^^ Has nothing to do with your problem. Why do listen on socks port?
Apr 06 10:35:13.000 [notice] Starting with guard context "default" Apr 06 10:35:13.000 [notice] Signaled readiness to systemd Apr 06 10:35:13.000 [notice] Interrupt: we have stopped accepting new connections, and will shut down in 30 seconds. Interrupt again to exit now.
Look in var/log/syslog if systemd might not find $TORPIDDIR/tor.pid
tor-relays@lists.torproject.org
-
lists＠for-privacy.net -
Roger Dingledine -
Totor be