December 2023 - metrics-alerts - lists.torproject.org

Cron <exonerator@materculae> cd /srv/exonerator.torproject.org/exonerator/ && ~exonerator/bin/run-exonerator
by root＠materculae.torproject.org 07 Dec '23

07 Dec '23

2023-12-07 08:22:05,320 WARN o.t.d.i.DescriptorIndexCollector:79 Cannot fetch index file https://collector.torproject.org/index/index.json and hence cannot determine which remote files to fetch. Aborting descriptor collection. java.net.NoRouteToHostException: No route to host (Host unreachable) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:305) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:509) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:604) at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266) at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:207) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:193) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168) at org.torproject.descriptor.index.IndexNode.fetchIndex(IndexNode.java:110) at org.torproject.descriptor.index.DescriptorIndexCollector.collectDescriptors(DescriptorIndexCollector.java:76) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.fetchDescriptors(ExoneraTorDatabaseImporter.java:201) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.main(ExoneraTorDatabaseImporter.java:65)

1 0

Cron <exonerator@materculae> cd /srv/exonerator.torproject.org/exonerator/ && ~exonerator/bin/run-exonerator
by root＠materculae.torproject.org 07 Dec '23

07 Dec '23

2023-12-07 07:22:04,881 WARN o.t.d.i.DescriptorIndexCollector:79 Cannot fetch index file https://collector.torproject.org/index/index.json and hence cannot determine which remote files to fetch. Aborting descriptor collection. java.net.NoRouteToHostException: No route to host (Host unreachable) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:305) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:509) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:604) at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266) at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:207) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:193) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168) at org.torproject.descriptor.index.IndexNode.fetchIndex(IndexNode.java:110) at org.torproject.descriptor.index.DescriptorIndexCollector.collectDescriptors(DescriptorIndexCollector.java:76) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.fetchDescriptors(ExoneraTorDatabaseImporter.java:201) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.main(ExoneraTorDatabaseImporter.java:65)

1 0

Cron <exonerator@materculae> cd /srv/exonerator.torproject.org/exonerator/ && ~exonerator/bin/run-exonerator
by root＠materculae.torproject.org 07 Dec '23

07 Dec '23

2023-12-07 06:22:05,576 WARN o.t.d.i.DescriptorIndexCollector:79 Cannot fetch index file https://collector.torproject.org/index/index.json and hence cannot determine which remote files to fetch. Aborting descriptor collection. java.net.NoRouteToHostException: No route to host (Host unreachable) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:305) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:509) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:604) at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266) at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:207) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:193) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168) at org.torproject.descriptor.index.IndexNode.fetchIndex(IndexNode.java:110) at org.torproject.descriptor.index.DescriptorIndexCollector.collectDescriptors(DescriptorIndexCollector.java:76) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.fetchDescriptors(ExoneraTorDatabaseImporter.java:201) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.main(ExoneraTorDatabaseImporter.java:65)

1 0

Cron <exonerator@materculae> cd /srv/exonerator.torproject.org/exonerator/ && ~exonerator/bin/run-exonerator
by root＠materculae.torproject.org 07 Dec '23

07 Dec '23

2023-12-07 05:22:05,032 WARN o.t.d.i.DescriptorIndexCollector:79 Cannot fetch index file https://collector.torproject.org/index/index.json and hence cannot determine which remote files to fetch. Aborting descriptor collection. java.net.NoRouteToHostException: No route to host (Host unreachable) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:305) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:509) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:604) at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266) at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:207) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:193) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168) at org.torproject.descriptor.index.IndexNode.fetchIndex(IndexNode.java:110) at org.torproject.descriptor.index.DescriptorIndexCollector.collectDescriptors(DescriptorIndexCollector.java:76) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.fetchDescriptors(ExoneraTorDatabaseImporter.java:201) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.main(ExoneraTorDatabaseImporter.java:65)

1 0

Cron <exonerator@materculae> cd /srv/exonerator.torproject.org/exonerator/ && ~exonerator/bin/run-exonerator
by root＠materculae.torproject.org 07 Dec '23

07 Dec '23

2023-12-07 04:22:05,486 WARN o.t.d.i.DescriptorIndexCollector:79 Cannot fetch index file https://collector.torproject.org/index/index.json and hence cannot determine which remote files to fetch. Aborting descriptor collection. java.net.NoRouteToHostException: No route to host (Host unreachable) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:305) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:509) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:604) at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266) at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:207) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:193) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168) at org.torproject.descriptor.index.IndexNode.fetchIndex(IndexNode.java:110) at org.torproject.descriptor.index.DescriptorIndexCollector.collectDescriptors(DescriptorIndexCollector.java:76) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.fetchDescriptors(ExoneraTorDatabaseImporter.java:201) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.main(ExoneraTorDatabaseImporter.java:65)

1 0

Cron <exonerator@materculae> cd /srv/exonerator.torproject.org/exonerator/ && ~exonerator/bin/run-exonerator
by root＠materculae.torproject.org 07 Dec '23

07 Dec '23

2023-12-07 03:22:05,292 WARN o.t.d.i.DescriptorIndexCollector:79 Cannot fetch index file https://collector.torproject.org/index/index.json and hence cannot determine which remote files to fetch. Aborting descriptor collection. java.net.NoRouteToHostException: No route to host (Host unreachable) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.base/java.net.Socket.connect(Socket.java:609) at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:305) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:509) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:604) at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266) at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:207) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:193) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168) at org.torproject.descriptor.index.IndexNode.fetchIndex(IndexNode.java:110) at org.torproject.descriptor.index.DescriptorIndexCollector.collectDescriptors(DescriptorIndexCollector.java:76) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.fetchDescriptors(ExoneraTorDatabaseImporter.java:201) at org.torproject.metrics.exonerator.ExoneraTorDatabaseImporter.main(ExoneraTorDatabaseImporter.java:65)

1 0

** PROBLEM Service Alert: onionoo-frontend-01/network service - onionoo full service check is CRITICAL **
by nagios＠hetzner-hel1-01.torproject.org 07 Dec '23

07 Dec '23

***** Icinga ***** Notification Type: PROBLEM Service: network service - onionoo full service check Host: onionoo-frontend-01 Address: 116.202.120.171 State: CRITICAL Date/Time: Thu Dec 7 03:04:09 UTC 2023 Additional Info: CRITICAL: Either timestamp is too old (22401 sec): relays_published=2023-12-07 01:00:00, bridges_published=2023-12-06 20:50:44

1 0

** PROBLEM Service Alert: onionoo-frontend-01/network service - onionoo varnish is CRITICAL **
by nagios＠hetzner-hel1-01.torproject.org 07 Dec '23

07 Dec '23

***** Icinga ***** Notification Type: PROBLEM Service: network service - onionoo varnish Host: onionoo-frontend-01 Address: 116.202.120.171 State: CRITICAL Date/Time: Thu Dec 7 03:03:49 UTC 2023 Additional Info: CRITICAL: Either timestamp is too old (22379 sec): relays_published=2023-12-07 01:00:00, bridges_published=2023-12-06 20:50:44

1 0

** PROBLEM Service Alert: onionoo-frontend-02/network service - onionoo full service check is CRITICAL **
by nagios＠hetzner-hel1-01.torproject.org 07 Dec '23

07 Dec '23

***** Icinga ***** Notification Type: PROBLEM Service: network service - onionoo full service check Host: onionoo-frontend-02 Address: 49.12.57.137 State: CRITICAL Date/Time: Thu Dec 7 03:03:49 UTC 2023 Additional Info: CRITICAL: Either timestamp is too old (22381 sec): relays_published=2023-12-07 01:00:00, bridges_published=2023-12-06 20:50:44

1 0

** PROBLEM Service Alert: onionoo-backend-02/network service - onionoo varnish is CRITICAL **
by nagios＠hetzner-hel1-01.torproject.org 07 Dec '23

07 Dec '23

***** Icinga ***** Notification Type: PROBLEM Service: network service - onionoo varnish Host: onionoo-backend-02 Address: 116.202.120.178 State: CRITICAL Date/Time: Thu Dec 7 03:03:29 UTC 2023 Additional Info: CRITICAL: Either timestamp is too old (22355 sec): relays_published=2023-12-07 01:00:00, bridges_published=2023-12-06 20:50:44

1 0