March 2026 - metrics-alerts - lists.torproject.org

consensus-ns missing from collector.tpo, was seen in seasalp
by tlogmon＠link.verkligendata.se 14 Mar '26

14 Mar '26

Consensus of type ns with (double) checksum 27b2dbfb38b7b5429a95ddd0af8fc4fef3fa2450c0488178a648ed66de6cab69 submitted to translog seasalp by dirauth key gabelmoo-ns not found in archive collector.tpo Found in translog at 2026-03-14T03:56Z Expected in archive no later than 2026-03-14T07:56Z

1 0

Cron <onionoo@onionoo-backend-03> /srv/onionoo.torproject.org/onionoo/monitoring/services/onionoo
by root＠onionoo-backend-03.torproject.org 14 Mar '26

14 Mar '26

Error fetching minimal summary document: <urlopen error [Errno -3] Temporary failure in name resolution>

1 0

Cron <onionoo@onionoo-backend-03> /srv/onionoo.torproject.org/onionoo/monitoring/network/onionoo
by root＠onionoo-backend-03.torproject.org 14 Mar '26

14 Mar '26

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 198, in _new_conn sock = connection.create_connection( (self._dns_host, self.port), ...<2 lines>... socket_options=self.socket_options, ) File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/socket.py", line 977, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( conn, ...<10 lines>... **response_kw, ) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) ~~~~~~~~~~~~~~~~~~~^^^^^^ File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() ~~~~~~~~~~~~^^ File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 704, in connect self.sock = sock = self._new_conn() ~~~~~~~~~~~~~~^^ File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 205, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f402734ef90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( method=request.method, ...<9 lines>... chunked=chunked, ) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2] ) File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='onionoo.torproject.org', port=443): Max retries exceeded with url: /details (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f402734ef90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/srv/onionoo.torproject.org/onionoo/monitoring/network/onionoo", line 31, in <module> details = json.loads(requests.get("https://onionoo.torproject.org/details").text) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) File "/usr/lib/python3/dist-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='onionoo.torproject.org', port=443): Max retries exceeded with url: /details (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f402734ef90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution)"))

1 0

[FIRING:1] HTTPSResponseDelayExceeded -- blackbox
by alertmanager＠prometheus2.torproject.org 14 Mar '26

14 Mar '26

Total firing alerts: 1 ## Firing Alerts ----- Time: 2026-03-14 06:15:10.651 +0000 UTC Summary: Website http://onionoo.torproject.org/ is too slow. Description: Website http://onionoo.torproject.org/ has been too slow to respond in the last hour, not achieving our availbility target (95%) in response times (1.5s), instead having a 2.031s response time. playbook: TODO -----

1 0

Cron <onionoo@onionoo-backend-03> /srv/onionoo.torproject.org/onionoo/monitoring/network/onionoo
by root＠onionoo-backend-03.torproject.org 14 Mar '26

14 Mar '26

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 198, in _new_conn sock = connection.create_connection( (self._dns_host, self.port), ...<2 lines>... socket_options=self.socket_options, ) File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/socket.py", line 977, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( conn, ...<10 lines>... **response_kw, ) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) ~~~~~~~~~~~~~~~~~~~^^^^^^ File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() ~~~~~~~~~~~~^^ File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 704, in connect self.sock = sock = self._new_conn() ~~~~~~~~~~~~~~^^ File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 205, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f4b38ebaf90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( method=request.method, ...<9 lines>... chunked=chunked, ) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2] ) File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='onionoo.torproject.org', port=443): Max retries exceeded with url: /details (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f4b38ebaf90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/srv/onionoo.torproject.org/onionoo/monitoring/network/onionoo", line 31, in <module> details = json.loads(requests.get("https://onionoo.torproject.org/details").text) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) File "/usr/lib/python3/dist-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='onionoo.torproject.org', port=443): Max retries exceeded with url: /details (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f4b38ebaf90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution)"))

1 0

Cron <onionoo@onionoo-backend-03> /srv/onionoo.torproject.org/onionoo/monitoring/services/onionoo
by root＠onionoo-backend-03.torproject.org 14 Mar '26

14 Mar '26

Error fetching minimal summary document: <urlopen error [Errno -3] Temporary failure in name resolution>

1 0

Cron <onionoo@onionoo-backend-03> /srv/onionoo.torproject.org/onionoo/monitoring/network/onionoo
by root＠onionoo-backend-03.torproject.org 14 Mar '26

14 Mar '26

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 198, in _new_conn sock = connection.create_connection( (self._dns_host, self.port), ...<2 lines>... socket_options=self.socket_options, ) File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/socket.py", line 977, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( conn, ...<10 lines>... **response_kw, ) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) ~~~~~~~~~~~~~~~~~~~^^^^^^ File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() ~~~~~~~~~~~~^^ File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 704, in connect self.sock = sock = self._new_conn() ~~~~~~~~~~~~~~^^ File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 205, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f3b67a8ef90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( method=request.method, ...<9 lines>... chunked=chunked, ) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2] ) File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='onionoo.torproject.org', port=443): Max retries exceeded with url: /details (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f3b67a8ef90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/srv/onionoo.torproject.org/onionoo/monitoring/network/onionoo", line 31, in <module> details = json.loads(requests.get("https://onionoo.torproject.org/details").text) ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) File "/usr/lib/python3/dist-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='onionoo.torproject.org', port=443): Max retries exceeded with url: /details (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f3b67a8ef90>: Failed to resolve 'onionoo.torproject.org' ([Errno -3] Temporary failure in name resolution)"))

1 0

Cron <onionoo@onionoo-backend-03> /srv/onionoo.torproject.org/onionoo/monitoring/services/onionoo
by root＠onionoo-backend-03.torproject.org 14 Mar '26

14 Mar '26

Error fetching minimal summary document: <urlopen error [Errno -3] Temporary failure in name resolution>

1 0

consensus-microdesc missing from collector.tpo, was seen in seasalp
by tlogmon＠link.verkligendata.se 14 Mar '26

14 Mar '26

Consensus of type microdesc with (double) checksum 67bafe8c45a11812a38b081cffd8d36ac8540f5ec5dab627018cb1d6343c7f63 submitted to translog seasalp by dirauth key maatuska-microdesc not found in archive collector.tpo Found in translog at 2026-03-14T02:56Z Expected in archive no later than 2026-03-14T06:56Z

1 0

consensus-ns missing from collector.tpo, was seen in seasalp
by tlogmon＠link.verkligendata.se 14 Mar '26

14 Mar '26

Consensus of type ns with (double) checksum 2b948a03d288ecaad7e8a3f2c340e16326562b28fd083a760210b1862ca3d8cf submitted to translog seasalp by dirauth key maatuska-ns not found in archive collector.tpo Found in translog at 2026-03-14T02:56Z Expected in archive no later than 2026-03-14T06:56Z

1 0