- metrics-alerts - lists.torproject.org

Cron <exonerator@materculae> cd /srv/exonerator.torproject.org/exonerator/ && ~exonerator/bin/run-exonerator
by root＠materculae.torproject.org 21 Sep '24

21 Sep '24

2024-09-21 00:22:02,119 ERROR o.t.m.e.ExoneraTorDatabaseImporter:167 Lock file /srv/exonerator.torproject.org/exonerator/exonerator-lock is less than 6 hours old. Either make sure that there are no other ExoneraTor database importers running and manually delete that file, or wait until the file is 6 hours old when it will be overwritten automatically. Exiting.

1 0

Cron <exonerator@materculae> cd /srv/exonerator.torproject.org/exonerator/ && ~exonerator/bin/run-exonerator
by root＠materculae.torproject.org 20 Sep '24

20 Sep '24

2024-09-20 23:22:02,340 ERROR o.t.m.e.ExoneraTorDatabaseImporter:167 Lock file /srv/exonerator.torproject.org/exonerator/exonerator-lock is less than 6 hours old. Either make sure that there are no other ExoneraTor database importers running and manually delete that file, or wait until the file is 6 hours old when it will be overwritten automatically. Exiting.

1 0

[FIRING:1] Relays per Flag MEDIUM -- onionoo_network
by alertmanager＠hetzner-nbg1-02.torproject.org 20 Sep '24

20 Sep '24

Total firing alerts: 1 Total resolved alerts: 0 ## Firing Alerts ----- Time: 2024-09-20 22:12:27.544 +0000 UTC Summary: More than 50 relays have joined the network with "HSDir" flag Description: More than 50 relays with "HSDir" flag have joined the network in the last 2 hours. ----- ##Resolved Alerts

1 0

[FIRING:6] NeedsReboot -- node
by alertmanager＠hetzner-nbg1-02.torproject.org 20 Sep '24

20 Sep '24

Total firing alerts: 6 Total resolved alerts: 0 ## Firing Alerts ----- Time: 2024-09-11 20:21:36.784 +0000 UTC Summary: Host Onionperf needs to reboot Description: Found pending kernel or microcode upgrades for host Onionperf ----- ----- Time: 2024-09-11 20:21:36.784 +0000 UTC Summary: Host Onionperf needs to reboot Description: Found pending kernel or microcode upgrades for host Onionperf ----- ----- Time: 2024-09-11 20:21:36.784 +0000 UTC Summary: Host Onionperf needs to reboot Description: Found pending kernel or microcode upgrades for host Onionperf ----- ----- Time: 2024-09-11 20:21:36.784 +0000 UTC Summary: Host Onionperf needs to reboot Description: Found pending kernel or microcode upgrades for host Onionperf ----- ----- Time: 2024-09-11 20:21:36.784 +0000 UTC Summary: Host Onionperf needs to reboot Description: Found pending kernel or microcode upgrades for host Onionperf ----- ----- Time: 2024-09-11 20:21:36.784 +0000 UTC Summary: Host Onionperf needs to reboot Description: Found pending kernel or microcode upgrades for host Onionperf ----- ##Resolved Alerts

1 0

[FIRING:1] Relays per Flag MEDIUM
by alertmanager＠hetzner-nbg1-02.torproject.org 20 Sep '24

20 Sep '24

Total firing alerts: 1 Total resolved alerts: 0 ## Firing Alerts ----- Time: 2024-09-20 13:12:27.544 +0000 UTC Summary: More than 50 relays have joined the network with "HSDir" flag Description: More than 50 relays with "HSDir" flag have joined the network in the last 2 hours. ----- ##Resolved Alerts

1 0

** RECOVERY Service Alert: onionoo-backend-01/network service - onionoo varnish is OK **
by nagios＠hetzner-hel1-01.torproject.org 20 Sep '24

20 Sep '24

***** Icinga ***** Notification Type: RECOVERY Service: network service - onionoo varnish Host: onionoo-backend-01 Address: 116.202.120.163 State: OK Date/Time: Fri Sept 20 07:19:30 UTC 2024 Additional Info: OK: Valid response with recent timestamps (5534 sec): relays_published=2024-09-20 06:00:00, bridges_published=2024-09-20 05:47:06

1 0

** RECOVERY Service Alert: onionoo-frontend-02/network service - onionoo varnish is OK **
by nagios＠hetzner-hel1-01.torproject.org 20 Sep '24

20 Sep '24

***** Icinga ***** Notification Type: RECOVERY Service: network service - onionoo varnish Host: onionoo-frontend-02 Address: 49.12.57.137 State: OK Date/Time: Fri Sept 20 07:19:30 UTC 2024 Additional Info: OK: Valid response with recent timestamps (5534 sec): relays_published=2024-09-20 06:00:00, bridges_published=2024-09-20 05:47:06

1 0

** RECOVERY Service Alert: onionoo-frontend-01/network service - onionoo varnish is OK **
by nagios＠hetzner-hel1-01.torproject.org 20 Sep '24

20 Sep '24

***** Icinga ***** Notification Type: RECOVERY Service: network service - onionoo varnish Host: onionoo-frontend-01 Address: 116.202.120.171 State: OK Date/Time: Fri Sept 20 07:17:30 UTC 2024 Additional Info: OK: Valid response with recent timestamps (5414 sec): relays_published=2024-09-20 06:00:00, bridges_published=2024-09-20 05:47:06

1 0

** RECOVERY Service Alert: onionoo-backend-02/network service - onionoo varnish is OK **
by nagios＠hetzner-hel1-01.torproject.org 20 Sep '24

20 Sep '24

***** Icinga ***** Notification Type: RECOVERY Service: network service - onionoo varnish Host: onionoo-backend-02 Address: 116.202.120.178 State: OK Date/Time: Fri Sept 20 07:17:30 UTC 2024 Additional Info: OK: Valid response with recent timestamps (5414 sec): relays_published=2024-09-20 06:00:00, bridges_published=2024-09-20 05:47:06

1 0

** RECOVERY Service Alert: onionoo-backend-01/network service - onionoo backend is OK **
by nagios＠hetzner-hel1-01.torproject.org 20 Sep '24

20 Sep '24

***** Icinga ***** Notification Type: RECOVERY Service: network service - onionoo backend Host: onionoo-backend-01 Address: 116.202.120.163 State: OK Date/Time: Fri Sept 20 07:15:30 UTC 2024 Additional Info: OK: Valid response with recent timestamps (5294 sec): relays_published=2024-09-20 06:00:00, bridges_published=2024-09-20 05:47:06

1 0