- metrics-alerts - lists.torproject.org

Cron <metrics@meronense> cd /srv/metrics.torproject.org/metrics/work/modules/logs && cat metrics.log | grep -v "Could not insert bandwidth "
by root＠meronense.torproject.org 23 Nov '21

23 Nov '21

2021-11-22 08:40:02,420 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 08:40:02,473 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 08:40:02,476 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 08:40:02,476 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 08:40:06,386 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 08:40:07,128 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 08:40:10,899 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 08:40:28,727 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 08:40:29,959 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 08:40:29,959 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 08:40:29,959 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 08:40:37,280 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 08:40:37,280 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 08:40:37,282 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 08:40:37,282 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 08:40:37,746 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 08:45:26,703 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 08:46:12,639 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 08:46:13,521 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 08:46:16,429 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 08:46:16,454 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 08:46:58,997 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 08:46:59,798 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 08:47:12,732 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 08:47:12,794 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 08:47:24,178 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 08:47:24,180 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 08:47:24,180 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 08:47:24,181 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 08:47:24,181 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 08:47:24,181 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 08:47:24,181 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 08:57:09,946 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 150891 network status entries and 137335 bandwidth history elements 2021-11-22 08:57:12,269 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 08:58:06,669 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 150891 network status entries and 137335 bandwidth history elements 2021-11-22 08:58:08,270 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 08:58:08,690 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 08:58:08,691 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 08:58:08,691 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 08:58:08,691 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 08:59:02,870 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 08:59:02,915 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 08:59:02,916 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 08:59:03,068 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 09:00:47,073 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 09:00:47,122 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 09:06:23,694 INFO o.t.m.s.hidserv.Main:82 Aggregating v2 statistics... 2021-11-22 09:08:02,717 INFO o.t.m.s.hidserv.Main:98 Aggregating v3 statistics... 2021-11-22 09:08:02,951 INFO o.t.m.s.hidserv.Main:108 Terminating. 2021-11-22 09:08:02,952 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 09:08:02,954 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.clients.Main module. 2021-11-22 09:08:02,971 INFO o.t.m.s.clients.Main:36 Starting clients module. 2021-11-22 09:08:02,972 INFO o.t.m.s.clients.Main:38 Connecting to database. 2021-11-22 09:08:03,222 INFO o.t.m.s.clients.Main:41 Reading relay descriptors and importing relevant parts into the database. 2021-11-22 09:09:30,378 INFO o.t.m.s.clients.Main:45 Reading bridge descriptors and importing relevant parts into the database. 2021-11-22 09:10:07,372 INFO o.t.m.s.clients.Main:49 Processing newly imported data. 2021-11-22 10:19:27,740 INFO o.t.m.s.clients.Main:53 Querying aggregated statistics from the database. 2021-11-22 10:19:49,856 INFO o.t.m.s.clients.Main:59 Disconnecting from database. 2021-11-22 10:19:49,889 INFO o.t.m.s.clients.Main:62 Running detector. 2021-11-22 10:20:25,828 INFO o.t.m.s.clients.Main:65 Terminating clients module. 2021-11-22 10:20:25,846 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.clients.Main module. 2021-11-22 10:20:25,847 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.servers.Main module. 2021-11-22 10:20:25,865 INFO o.t.m.s.servers.Main:43 Starting servers module. 2021-11-22 10:20:25,865 INFO o.t.m.s.servers.Main:45 Reading descriptors and inserting relevant parts into the database. 2021-11-22 10:46:15,034 INFO o.t.m.s.servers.Main:77 Aggregating database entries. 2021-11-22 11:48:39,634 INFO o.t.m.s.servers.Main:80 Committing all updated parts in the database. 2021-11-22 11:48:40,099 INFO o.t.m.s.servers.Main:90 Querying aggregated statistics from the database. 2021-11-22 12:05:12,459 INFO o.t.m.s.servers.Main:105 Terminating servers module. 2021-11-22 12:05:12,666 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.servers.Main module. 2021-11-22 12:05:12,667 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.webstats.Main module. 2021-11-22 12:05:12,708 INFO o.t.m.s.w.Main:75 Starting webstats module. 2021-11-22 12:05:12,708 INFO o.t.m.s.w.Main:92 Connecting to database. 2021-11-22 12:05:13,082 INFO o.t.m.s.w.Main:95 Successfully connected to database. 2021-11-22 12:05:13,083 INFO o.t.m.s.w.Main:101 Querying previously imported log files. 2021-11-22 12:05:15,131 INFO o.t.m.s.w.Main:113 Found 284694 previously imported log files. 2021-11-22 12:25:52,295 INFO o.t.m.s.w.Main:268 Querying statistics from database. 2021-11-22 12:47:57,595 INFO o.t.m.s.w.Main:298 Writing 1417810 lines to /srv/metrics.torproject.org/metrics/work/modules/webstats/stats/webstats.csv. 2021-11-22 12:47:58,408 INFO o.t.m.s.w.Main:305 Disconnecting from database. 2021-11-22 12:47:58,410 INFO o.t.m.s.w.Main:87 Terminated webstats module. 2021-11-22 12:47:58,410 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.webstats.Main module. 2021-11-22 12:47:58,410 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.totalcw.Main module. 2021-11-22 12:47:58,412 INFO o.t.m.s.totalcw.Main:38 Starting totalcw module. 2021-11-22 12:47:58,412 INFO o.t.m.s.totalcw.Main:40 Reading consensuses and votes and inserting relevant parts into the database. 2021-11-22 12:51:28,183 INFO o.t.m.s.totalcw.Main:64 Committing all updated parts in the database. 2021-11-22 12:51:28,246 INFO o.t.m.s.totalcw.Main:74 Querying aggregated statistics from the database. 2021-11-22 12:51:38,353 INFO o.t.m.s.totalcw.Main:77 Writing aggregated statistics to /srv/metrics.torproject.org/metrics/work/modules/totalcw/stats/totalcw.csv. 2021-11-22 12:51:38,553 INFO o.t.m.s.totalcw.Main:82 Terminating totalcw module. 2021-11-22 12:51:38,554 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.totalcw.Main module. 2021-11-22 12:51:38,554 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bridgedb.Main module. 2021-11-22 12:51:38,977 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bridgedb.Main module. 2021-11-22 12:51:38,977 INFO o.t.m.s.main.Main:82 Making module data available. 2021-11-22 12:51:47,361 INFO o.t.m.s.main.Main:126 Successfully copied 22 files to stats output directory: [connbidirect2.csv, onionperf-failures.csv, buildtimes.csv, latencies.csv, torperf-1.1.csv, onionperf-throughput.csv, onionperf-including-partials.csv, bandwidth.csv, advbwdist.csv, v3hidserv.csv, hidserv.csv, clients.csv, userstats-combined.csv, platforms.csv, relayflags.csv, versions.csv, advbw.csv, networksize.csv, ipv6servers.csv, webstats.csv, totalcw.csv, bridgedb-stats.csv] 2021-11-22 12:51:47,366 INFO o.t.m.s.main.Main:130 Completed metrics update run. 2021-11-22 15:21:22,435 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 15:21:22,458 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 15:21:22,460 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 15:21:22,460 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 15:21:22,501 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 15:21:23,108 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 15:21:24,538 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 15:21:29,553 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 15:21:29,644 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 15:21:29,644 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 15:21:29,644 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 15:21:33,390 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 15:21:33,390 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 15:21:33,391 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 15:21:33,690 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 15:21:33,941 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 15:21:44,197 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 15:22:32,742 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 15:22:33,356 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 15:22:37,612 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 15:22:37,615 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 15:23:59,456 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 15:23:59,463 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 15:24:11,591 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 15:24:11,665 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 15:24:23,114 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 15:24:23,118 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 15:24:23,118 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 15:24:23,118 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 15:24:23,118 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 15:24:23,119 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 15:24:23,119 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 15:29:20,611 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 44045 network status entries and 30850 bandwidth history elements 2021-11-22 15:29:25,224 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 15:30:25,037 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 44045 network status entries and 30850 bandwidth history elements 2021-11-22 15:30:25,916 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 15:30:26,293 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 15:30:26,293 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 15:30:26,293 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 15:30:26,294 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 15:31:41,138 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 15:31:41,139 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 15:31:41,139 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 15:31:41,240 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 15:33:17,514 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 15:33:17,642 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 15:37:30,549 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 15:37:30,574 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 15:37:30,577 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 15:37:30,578 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 15:37:32,053 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 15:37:32,907 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 15:37:33,058 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 15:37:33,060 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 15:37:33,064 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 15:37:33,064 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 15:37:33,064 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 15:37:34,932 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 15:37:34,932 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 15:37:34,934 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 15:37:34,934 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 15:37:35,879 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 15:37:48,888 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 15:38:32,458 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 15:38:32,560 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 15:38:36,776 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 15:38:36,833 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 15:39:31,250 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 15:39:31,871 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 15:39:43,212 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 15:39:43,214 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 15:39:54,502 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 15:39:54,505 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 15:39:54,506 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 15:39:54,506 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 15:39:54,506 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 15:39:54,506 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 15:39:54,506 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 15:39:54,674 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 0 network status entries and 0 bandwidth history elements 2021-11-22 15:39:54,675 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 15:40:22,541 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 0 network status entries and 0 bandwidth history elements 2021-11-22 15:40:22,589 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 15:40:22,782 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 15:40:22,782 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 15:40:22,782 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 15:40:22,782 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 15:41:07,158 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 15:41:07,158 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 15:41:07,158 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 15:41:07,264 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 15:42:28,580 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 15:42:28,815 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 15:47:20,105 INFO o.t.m.s.hidserv.Main:82 Aggregating v2 statistics... 2021-11-22 15:48:09,834 INFO o.t.m.s.hidserv.Main:98 Aggregating v3 statistics... 2021-11-22 15:48:10,043 INFO o.t.m.s.hidserv.Main:108 Terminating. 2021-11-22 15:48:10,045 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 15:48:10,045 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.clients.Main module. 2021-11-22 15:48:10,072 INFO o.t.m.s.clients.Main:36 Starting clients module. 2021-11-22 15:48:10,073 INFO o.t.m.s.clients.Main:38 Connecting to database. 2021-11-22 15:48:10,329 INFO o.t.m.s.clients.Main:41 Reading relay descriptors and importing relevant parts into the database. 2021-11-22 15:48:34,936 INFO o.t.m.s.clients.Main:45 Reading bridge descriptors and importing relevant parts into the database. 2021-11-22 15:48:43,601 INFO o.t.m.s.clients.Main:49 Processing newly imported data. 2021-11-22 16:05:35,513 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 16:05:35,571 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 16:05:35,574 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 16:05:35,614 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 16:05:38,664 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 16:05:39,942 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 16:05:40,148 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 16:05:40,150 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 16:05:40,181 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 16:05:40,181 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 16:05:40,181 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 16:05:44,606 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 16:05:44,607 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 16:05:44,608 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 16:05:44,608 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 16:05:45,049 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 16:05:55,789 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 16:06:31,639 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 16:06:31,880 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 16:06:49,147 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 16:06:49,150 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 16:07:48,883 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 16:07:48,904 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 16:08:00,750 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 16:08:00,753 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 16:08:12,548 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 16:08:12,551 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 16:08:12,551 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 16:08:12,551 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 16:08:12,551 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 16:08:12,551 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 16:08:12,552 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 16:08:25,795 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 0 network status entries and 1964 bandwidth history elements 2021-11-22 16:08:30,634 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 16:08:43,202 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 0 network status entries and 1964 bandwidth history elements 2021-11-22 16:08:43,231 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 16:08:43,416 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 16:08:43,416 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 16:08:43,416 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 16:08:43,416 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 16:09:27,670 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 16:09:27,705 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 16:09:27,705 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 16:09:27,763 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 16:10:56,887 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 16:10:56,953 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 16:14:47,208 INFO o.t.m.s.hidserv.Main:82 Aggregating v2 statistics... 2021-11-22 16:15:43,512 INFO o.t.m.s.hidserv.Main:98 Aggregating v3 statistics... 2021-11-22 16:15:43,897 INFO o.t.m.s.hidserv.Main:108 Terminating. 2021-11-22 16:15:43,897 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 16:15:43,898 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.clients.Main module. 2021-11-22 16:15:43,910 INFO o.t.m.s.clients.Main:36 Starting clients module. 2021-11-22 16:15:43,910 INFO o.t.m.s.clients.Main:38 Connecting to database. 2021-11-22 16:15:44,273 INFO o.t.m.s.clients.Main:41 Reading relay descriptors and importing relevant parts into the database. 2021-11-22 16:15:44,968 INFO o.t.m.s.clients.Main:45 Reading bridge descriptors and importing relevant parts into the database. 2021-11-22 16:15:45,159 INFO o.t.m.s.clients.Main:49 Processing newly imported data. 2021-11-22 17:22:20,708 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 17:22:21,031 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 17:22:21,034 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 17:22:21,034 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 17:22:22,451 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 17:22:24,521 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 17:22:25,583 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 17:22:26,447 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 17:22:27,267 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 17:22:27,267 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 17:22:27,267 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 17:22:33,432 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 17:22:33,444 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 17:22:33,446 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 17:22:33,446 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 17:22:33,653 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 17:22:45,602 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 17:26:37,438 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 17:26:37,977 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 17:27:32,551 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 17:27:32,746 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 17:35:16,854 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 17:35:17,000 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 17:36:22,624 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 17:36:22,766 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 17:37:08,585 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 17:37:08,599 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 17:37:08,604 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 17:37:08,605 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 17:37:08,605 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 17:37:08,605 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 17:37:08,605 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 17:37:56,597 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 12532 network status entries and 5749 bandwidth history elements 2021-11-22 17:37:59,799 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 17:38:13,258 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 12532 network status entries and 5749 bandwidth history elements 2021-11-22 17:38:13,609 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 17:38:14,067 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 17:38:14,067 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 17:38:14,067 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 17:38:14,068 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 17:39:03,556 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 17:39:03,557 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 17:39:03,558 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 17:39:03,586 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 17:44:00,218 INFO o.t.m.s.clients.Main:53 Querying aggregated statistics from the database.

1 0

Cron <collector@colchicifolium> cd /srv/collector.torproject.org/collector/logs/ && cat metrics.log | grep -E "WARN|ERROR|FATAL" | grep -v "Overwriting existing descriptor file"
by root＠colchicifolium.torproject.org 23 Nov '21

23 Nov '21

2021-11-23 00:07:34,702 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (29.118167462542278).

1 0

Cron <onionoo@onionoo-backend-01> /srv/onionoo.torproject.org/onionoo/bin/service/onionoo
by root＠onionoo-backend-01.torproject.org 23 Nov '21

23 Nov '21

Traceback (most recent call last): File "/srv/onionoo.torproject.org/onionoo/bin/service/onionoo", line 23, in <module> list_of_errors = Counter([(x.split(' ')[2], x.split(' ')[3].split(':')[0]) for x in str(out).split('\\n') if x != '"']) File "/srv/onionoo.torproject.org/onionoo/bin/service/onionoo", line 23, in <listcomp> list_of_errors = Counter([(x.split(' ')[2], x.split(' ')[3].split(':')[0]) for x in str(out).split('\\n') if x != '"']) IndexError: list index out of range During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/srv/onionoo.torproject.org/onionoo/bin/service/onionoo", line 29, in <module> g_warnings.labels(host, "Problem running exporter - {}".format(e)).set(1) File "/usr/lib/python3/dist-packages/prometheus_client/metrics.py", line 146, in labels raise ValueError('Incorrect label count') ValueError: Incorrect label count

1 0

Cron <metrics@meronense> cd /srv/metrics.torproject.org/metrics/work/modules/logs && cat metrics.log | grep -v "Could not insert bandwidth "
by root＠meronense.torproject.org 22 Nov '21

22 Nov '21

2021-11-22 08:40:02,420 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 08:40:02,473 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 08:40:02,476 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 08:40:02,476 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 08:40:06,386 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 08:40:07,128 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 08:40:10,899 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 08:40:28,727 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 08:40:29,959 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 08:40:29,959 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 08:40:29,959 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 08:40:37,280 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 08:40:37,280 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 08:40:37,282 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 08:40:37,282 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 08:40:37,746 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 08:45:26,703 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 08:46:12,639 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 08:46:13,521 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 08:46:16,429 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 08:46:16,454 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 08:46:58,997 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 08:46:59,798 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 08:47:12,732 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 08:47:12,794 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 08:47:24,178 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 08:47:24,180 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 08:47:24,180 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 08:47:24,181 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 08:47:24,181 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 08:47:24,181 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 08:47:24,181 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 08:57:09,946 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 150891 network status entries and 137335 bandwidth history elements 2021-11-22 08:57:12,269 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 08:58:06,669 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 150891 network status entries and 137335 bandwidth history elements 2021-11-22 08:58:08,270 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 08:58:08,690 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 08:58:08,691 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 08:58:08,691 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 08:58:08,691 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 08:59:02,870 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 08:59:02,915 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 08:59:02,916 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 08:59:03,068 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 09:00:47,073 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 09:00:47,122 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 09:06:23,694 INFO o.t.m.s.hidserv.Main:82 Aggregating v2 statistics... 2021-11-22 09:08:02,717 INFO o.t.m.s.hidserv.Main:98 Aggregating v3 statistics... 2021-11-22 09:08:02,951 INFO o.t.m.s.hidserv.Main:108 Terminating. 2021-11-22 09:08:02,952 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 09:08:02,954 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.clients.Main module. 2021-11-22 09:08:02,971 INFO o.t.m.s.clients.Main:36 Starting clients module. 2021-11-22 09:08:02,972 INFO o.t.m.s.clients.Main:38 Connecting to database. 2021-11-22 09:08:03,222 INFO o.t.m.s.clients.Main:41 Reading relay descriptors and importing relevant parts into the database. 2021-11-22 09:09:30,378 INFO o.t.m.s.clients.Main:45 Reading bridge descriptors and importing relevant parts into the database. 2021-11-22 09:10:07,372 INFO o.t.m.s.clients.Main:49 Processing newly imported data. 2021-11-22 10:19:27,740 INFO o.t.m.s.clients.Main:53 Querying aggregated statistics from the database. 2021-11-22 10:19:49,856 INFO o.t.m.s.clients.Main:59 Disconnecting from database. 2021-11-22 10:19:49,889 INFO o.t.m.s.clients.Main:62 Running detector. 2021-11-22 10:20:25,828 INFO o.t.m.s.clients.Main:65 Terminating clients module. 2021-11-22 10:20:25,846 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.clients.Main module. 2021-11-22 10:20:25,847 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.servers.Main module. 2021-11-22 10:20:25,865 INFO o.t.m.s.servers.Main:43 Starting servers module. 2021-11-22 10:20:25,865 INFO o.t.m.s.servers.Main:45 Reading descriptors and inserting relevant parts into the database. 2021-11-22 10:46:15,034 INFO o.t.m.s.servers.Main:77 Aggregating database entries. 2021-11-22 11:48:39,634 INFO o.t.m.s.servers.Main:80 Committing all updated parts in the database. 2021-11-22 11:48:40,099 INFO o.t.m.s.servers.Main:90 Querying aggregated statistics from the database. 2021-11-22 12:05:12,459 INFO o.t.m.s.servers.Main:105 Terminating servers module. 2021-11-22 12:05:12,666 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.servers.Main module. 2021-11-22 12:05:12,667 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.webstats.Main module. 2021-11-22 12:05:12,708 INFO o.t.m.s.w.Main:75 Starting webstats module. 2021-11-22 12:05:12,708 INFO o.t.m.s.w.Main:92 Connecting to database. 2021-11-22 12:05:13,082 INFO o.t.m.s.w.Main:95 Successfully connected to database. 2021-11-22 12:05:13,083 INFO o.t.m.s.w.Main:101 Querying previously imported log files. 2021-11-22 12:05:15,131 INFO o.t.m.s.w.Main:113 Found 284694 previously imported log files. 2021-11-22 12:25:52,295 INFO o.t.m.s.w.Main:268 Querying statistics from database. 2021-11-22 12:47:57,595 INFO o.t.m.s.w.Main:298 Writing 1417810 lines to /srv/metrics.torproject.org/metrics/work/modules/webstats/stats/webstats.csv. 2021-11-22 12:47:58,408 INFO o.t.m.s.w.Main:305 Disconnecting from database. 2021-11-22 12:47:58,410 INFO o.t.m.s.w.Main:87 Terminated webstats module. 2021-11-22 12:47:58,410 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.webstats.Main module. 2021-11-22 12:47:58,410 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.totalcw.Main module. 2021-11-22 12:47:58,412 INFO o.t.m.s.totalcw.Main:38 Starting totalcw module. 2021-11-22 12:47:58,412 INFO o.t.m.s.totalcw.Main:40 Reading consensuses and votes and inserting relevant parts into the database. 2021-11-22 12:51:28,183 INFO o.t.m.s.totalcw.Main:64 Committing all updated parts in the database. 2021-11-22 12:51:28,246 INFO o.t.m.s.totalcw.Main:74 Querying aggregated statistics from the database. 2021-11-22 12:51:38,353 INFO o.t.m.s.totalcw.Main:77 Writing aggregated statistics to /srv/metrics.torproject.org/metrics/work/modules/totalcw/stats/totalcw.csv. 2021-11-22 12:51:38,553 INFO o.t.m.s.totalcw.Main:82 Terminating totalcw module. 2021-11-22 12:51:38,554 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.totalcw.Main module. 2021-11-22 12:51:38,554 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bridgedb.Main module. 2021-11-22 12:51:38,977 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bridgedb.Main module. 2021-11-22 12:51:38,977 INFO o.t.m.s.main.Main:82 Making module data available. 2021-11-22 12:51:47,361 INFO o.t.m.s.main.Main:126 Successfully copied 22 files to stats output directory: [connbidirect2.csv, onionperf-failures.csv, buildtimes.csv, latencies.csv, torperf-1.1.csv, onionperf-throughput.csv, onionperf-including-partials.csv, bandwidth.csv, advbwdist.csv, v3hidserv.csv, hidserv.csv, clients.csv, userstats-combined.csv, platforms.csv, relayflags.csv, versions.csv, advbw.csv, networksize.csv, ipv6servers.csv, webstats.csv, totalcw.csv, bridgedb-stats.csv] 2021-11-22 12:51:47,366 INFO o.t.m.s.main.Main:130 Completed metrics update run. 2021-11-22 15:21:22,435 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 15:21:22,458 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 15:21:22,460 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 15:21:22,460 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 15:21:22,501 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 15:21:23,108 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 15:21:24,538 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 15:21:29,553 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 15:21:29,644 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 15:21:29,644 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 15:21:29,644 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 15:21:33,390 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 15:21:33,390 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 15:21:33,391 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 15:21:33,690 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 15:21:33,941 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 15:21:44,197 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 15:22:32,742 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 15:22:33,356 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 15:22:37,612 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 15:22:37,615 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 15:23:59,456 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 15:23:59,463 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 15:24:11,591 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 15:24:11,665 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 15:24:23,114 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 15:24:23,118 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 15:24:23,118 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 15:24:23,118 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 15:24:23,118 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 15:24:23,119 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 15:24:23,119 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 15:29:20,611 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 44045 network status entries and 30850 bandwidth history elements 2021-11-22 15:29:25,224 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 15:30:25,037 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 44045 network status entries and 30850 bandwidth history elements 2021-11-22 15:30:25,916 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 15:30:26,293 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 15:30:26,293 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 15:30:26,293 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 15:30:26,294 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 15:31:41,138 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 15:31:41,139 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 15:31:41,139 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 15:31:41,240 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 15:33:17,514 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 15:33:17,642 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 15:37:30,549 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 15:37:30,574 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 15:37:30,577 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 15:37:30,578 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 15:37:32,053 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 15:37:32,907 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 15:37:33,058 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 15:37:33,060 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 15:37:33,064 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 15:37:33,064 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 15:37:33,064 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 15:37:34,932 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 15:37:34,932 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 15:37:34,934 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 15:37:34,934 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 15:37:35,879 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 15:37:48,888 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 15:38:32,458 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 15:38:32,560 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 15:38:36,776 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 15:38:36,833 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 15:39:31,250 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 15:39:31,871 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 15:39:43,212 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 15:39:43,214 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 15:39:54,502 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 15:39:54,505 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 15:39:54,506 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 15:39:54,506 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 15:39:54,506 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 15:39:54,506 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 15:39:54,506 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 15:39:54,674 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 0 network status entries and 0 bandwidth history elements 2021-11-22 15:39:54,675 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 15:40:22,541 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 0 network status entries and 0 bandwidth history elements 2021-11-22 15:40:22,589 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 15:40:22,782 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 15:40:22,782 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 15:40:22,782 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 15:40:22,782 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 15:41:07,158 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 15:41:07,158 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 15:41:07,158 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 15:41:07,264 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 15:42:28,580 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 15:42:28,815 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 15:47:20,105 INFO o.t.m.s.hidserv.Main:82 Aggregating v2 statistics... 2021-11-22 15:48:09,834 INFO o.t.m.s.hidserv.Main:98 Aggregating v3 statistics... 2021-11-22 15:48:10,043 INFO o.t.m.s.hidserv.Main:108 Terminating. 2021-11-22 15:48:10,045 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 15:48:10,045 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.clients.Main module. 2021-11-22 15:48:10,072 INFO o.t.m.s.clients.Main:36 Starting clients module. 2021-11-22 15:48:10,073 INFO o.t.m.s.clients.Main:38 Connecting to database. 2021-11-22 15:48:10,329 INFO o.t.m.s.clients.Main:41 Reading relay descriptors and importing relevant parts into the database. 2021-11-22 15:48:34,936 INFO o.t.m.s.clients.Main:45 Reading bridge descriptors and importing relevant parts into the database. 2021-11-22 15:48:43,601 INFO o.t.m.s.clients.Main:49 Processing newly imported data. 2021-11-22 16:05:35,513 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 16:05:35,571 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 16:05:35,574 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 16:05:35,614 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 16:05:38,664 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 16:05:39,942 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 16:05:40,148 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 16:05:40,150 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 16:05:40,181 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 16:05:40,181 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 16:05:40,181 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 16:05:44,606 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 16:05:44,607 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 16:05:44,608 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 16:05:44,608 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 16:05:45,049 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 16:05:55,789 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 16:06:31,639 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 16:06:31,880 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 16:06:49,147 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 16:06:49,150 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 16:07:48,883 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 16:07:48,904 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 16:08:00,750 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 16:08:00,753 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 16:08:12,548 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 16:08:12,551 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 16:08:12,551 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 16:08:12,551 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 16:08:12,551 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 16:08:12,551 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 16:08:12,552 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 16:08:25,795 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 0 network status entries and 1964 bandwidth history elements 2021-11-22 16:08:30,634 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 16:08:43,202 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 0 network status entries and 1964 bandwidth history elements 2021-11-22 16:08:43,231 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 16:08:43,416 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 16:08:43,416 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 16:08:43,416 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 16:08:43,416 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 16:09:27,670 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 16:09:27,705 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 16:09:27,705 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 16:09:27,763 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 16:10:56,887 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 16:10:56,953 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 16:14:47,208 INFO o.t.m.s.hidserv.Main:82 Aggregating v2 statistics... 2021-11-22 16:15:43,512 INFO o.t.m.s.hidserv.Main:98 Aggregating v3 statistics... 2021-11-22 16:15:43,897 INFO o.t.m.s.hidserv.Main:108 Terminating. 2021-11-22 16:15:43,897 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 16:15:43,898 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.clients.Main module. 2021-11-22 16:15:43,910 INFO o.t.m.s.clients.Main:36 Starting clients module. 2021-11-22 16:15:43,910 INFO o.t.m.s.clients.Main:38 Connecting to database. 2021-11-22 16:15:44,273 INFO o.t.m.s.clients.Main:41 Reading relay descriptors and importing relevant parts into the database. 2021-11-22 16:15:44,968 INFO o.t.m.s.clients.Main:45 Reading bridge descriptors and importing relevant parts into the database. 2021-11-22 16:15:45,159 INFO o.t.m.s.clients.Main:49 Processing newly imported data. 2021-11-22 17:22:20,708 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 17:22:21,031 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 17:22:21,034 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 17:22:21,034 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 17:22:22,451 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 17:22:24,521 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 17:22:25,583 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 17:22:26,447 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 17:22:27,267 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 17:22:27,267 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 17:22:27,267 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 17:22:33,432 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 17:22:33,444 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 17:22:33,446 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 17:22:33,446 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 17:22:33,653 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 17:22:45,602 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 17:26:37,438 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 17:26:37,977 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 17:27:32,551 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 17:27:32,746 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 17:35:16,854 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 17:35:17,000 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 17:36:22,624 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 17:36:22,766 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 17:37:08,585 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 17:37:08,599 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 17:37:08,604 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 17:37:08,605 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 17:37:08,605 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 17:37:08,605 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 17:37:08,605 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 17:37:56,597 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 12532 network status entries and 5749 bandwidth history elements 2021-11-22 17:37:59,799 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 17:38:13,258 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 12532 network status entries and 5749 bandwidth history elements 2021-11-22 17:38:13,609 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 17:38:14,067 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 17:38:14,067 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 17:38:14,067 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 17:38:14,068 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 17:39:03,556 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 17:39:03,557 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 17:39:03,558 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 17:39:03,586 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 17:44:00,218 INFO o.t.m.s.clients.Main:53 Querying aggregated statistics from the database.

1 0

Cron <collector@colchicifolium> cd /srv/collector.torproject.org/collector/logs/ && cat metrics.log | grep -E "WARN|ERROR|FATAL" | grep -v "Overwriting existing descriptor file"
by root＠colchicifolium.torproject.org 22 Nov '21

22 Nov '21

2021-11-22 00:08:04,136 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (14.587198273283384). 2021-11-22 00:37:40,517 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (12.907462238906465). 2021-11-22 01:07:24,780 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (24.72445352188792). 2021-11-22 01:37:05,902 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (13.88107443670385). 2021-11-22 02:07:13,203 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.164231288529113). 2021-11-22 02:37:12,451 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.751352221616898). 2021-11-22 03:07:12,216 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (19.345599438098702). 2021-11-22 03:37:19,995 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (6.449838585798688). 2021-11-22 04:07:13,549 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (18.329712786921036). 2021-11-22 04:37:01,940 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.893314494780442). 2021-11-22 05:07:03,980 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.774450587468424). 2021-11-22 05:36:59,696 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.645816731665365). 2021-11-22 06:07:03,023 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.33444985163934). 2021-11-22 06:37:02,349 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (6.536175454896977). 2021-11-22 07:07:30,704 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.135293049740758). 2021-11-22 07:36:54,221 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.791270442540856). 2021-11-22 08:07:06,333 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (19.688255404232965). 2021-11-22 08:37:10,184 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (19.23868510150478). 2021-11-22 09:07:36,819 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (28.980679040977055). 2021-11-22 09:37:12,838 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (27.226004619578713). 2021-11-22 10:07:23,833 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (39.06472173112308). 2021-11-22 10:37:36,242 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (22.086299169982865). 2021-11-22 11:07:31,834 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (34.01663258205433). 2021-11-22 11:37:22,029 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (28.38917369911307). 2021-11-22 12:07:58,396 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (33.32354779922783). 2021-11-22 12:38:12,785 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (30.026187977174594). 2021-11-22 13:08:06,618 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (34.33920259206711). 2021-11-22 13:38:06,499 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (27.886554053194896). 2021-11-22 14:07:42,670 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (36.73372216523811). 2021-11-22 14:37:19,720 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (28.567892501794624). 2021-11-22 15:07:23,032 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (39.99468344267531). 2021-11-22 15:37:20,385 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (36.22777788917238). 2021-11-22 16:07:29,926 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (43.05898895490801). 2021-11-22 16:38:13,762 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (35.156863128053025). 2021-11-22 17:07:43,300 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (44.17168478114722). 2021-11-22 17:37:25,280 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (42.97783862730118). 2021-11-22 18:07:45,781 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (46.953663004794784).

1 0

Cron <metrics@meronense> cd /srv/metrics.torproject.org/metrics/ && ./start-web.sh
by root＠meronense.torproject.org 22 Nov '21

22 Nov '21

2021-11-12 06:10:44,148 ERROR o.t.m.e.ExoneraTorServlet:319 Backend query failed. java.io.IOException: Server returned HTTP response code: 500 for URL: https://exonerator.torproject.org/query.json?ip=111.47.226.23&timestamp=202… at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) at java.base/java.net.URL.openStream(URL.java:1165) at org.torproject.metrics.exonerator.ExoneraTorServlet.queryBackend(ExoneraTorServlet.java:314) at org.torproject.metrics.exonerator.ExoneraTorServlet.doGet(ExoneraTorServlet.java:100) at org.torproject.metrics.web.ExoneraTorWrapperServlet.doGet(ExoneraTorWrapperServlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.base/java.lang.Thread.run(Thread.java:829) 2021-11-12 06:11:39,080 ERROR o.t.m.e.ExoneraTorServlet:319 Backend query failed. java.io.IOException: Server returned HTTP response code: 500 for URL: https://exonerator.torproject.org/query.json?ip=125.238.203.64&timestamp=20… at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) at java.base/java.net.URL.openStream(URL.java:1165) at org.torproject.metrics.exonerator.ExoneraTorServlet.queryBackend(ExoneraTorServlet.java:314) at org.torproject.metrics.exonerator.ExoneraTorServlet.doGet(ExoneraTorServlet.java:100) at org.torproject.metrics.web.ExoneraTorWrapperServlet.doGet(ExoneraTorWrapperServlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.base/java.lang.Thread.run(Thread.java:829) 2021-11-12 06:11:39,426 ERROR o.t.m.e.ExoneraTorServlet:319 Backend query failed. java.io.IOException: Server returned HTTP response code: 500 for URL: https://exonerator.torproject.org/query.json?ip=106.195.44.236&timestamp=20… at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) at java.base/java.net.URL.openStream(URL.java:1165) at org.torproject.metrics.exonerator.ExoneraTorServlet.queryBackend(ExoneraTorServlet.java:314) at org.torproject.metrics.exonerator.ExoneraTorServlet.doGet(ExoneraTorServlet.java:100) at org.torproject.metrics.web.ExoneraTorWrapperServlet.doGet(ExoneraTorWrapperServlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.base/java.lang.Thread.run(Thread.java:829) 2021-11-12 06:11:39,484 ERROR o.t.m.e.ExoneraTorServlet:319 Backend query failed. java.io.IOException: Server returned HTTP response code: 500 for URL: https://exonerator.torproject.org/query.json?ip=116.90.79.180&timestamp=202… at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) at java.base/java.net.URL.openStream(URL.java:1165) at org.torproject.metrics.exonerator.ExoneraTorServlet.queryBackend(ExoneraTorServlet.java:314) at org.torproject.metrics.exonerator.ExoneraTorServlet.doGet(ExoneraTorServlet.java:100) at org.torproject.metrics.web.ExoneraTorWrapperServlet.doGet(ExoneraTorWrapperServlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.base/java.lang.Thread.run(Thread.java:829) 2021-11-12 06:11:39,887 ERROR o.t.m.e.ExoneraTorServlet:319 Backend query failed. java.io.IOException: Server returned HTTP response code: 500 for URL: https://exonerator.torproject.org/query.json?ip=180.235.107.5&timestamp=202… at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) at java.base/java.net.URL.openStream(URL.java:1165) at org.torproject.metrics.exonerator.ExoneraTorServlet.queryBackend(ExoneraTorServlet.java:314) at org.torproject.metrics.exonerator.ExoneraTorServlet.doGet(ExoneraTorServlet.java:100) at org.torproject.metrics.web.ExoneraTorWrapperServlet.doGet(ExoneraTorWrapperServlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.base/java.lang.Thread.run(Thread.java:829) 2021-11-12 06:11:40,315 ERROR o.t.m.e.ExoneraTorServlet:319 Backend query failed. java.io.IOException: Server returned HTTP response code: 500 for URL: https://exonerator.torproject.org/query.json?ip=118.149.72.248&timestamp=20… at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) at java.base/java.net.URL.openStream(URL.java:1165) at org.torproject.metrics.exonerator.ExoneraTorServlet.queryBackend(ExoneraTorServlet.java:314) at org.torproject.metrics.exonerator.ExoneraTorServlet.doGet(ExoneraTorServlet.java:100) at org.torproject.metrics.web.ExoneraTorWrapperServlet.doGet(ExoneraTorWrapperServlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.base/java.lang.Thread.run(Thread.java:829) 2021-11-12 06:11:41,109 ERROR o.t.m.e.ExoneraTorServlet:319 Backend query failed. java.io.IOException: Server returned HTTP response code: 500 for URL: https://exonerator.torproject.org/query.json?ip=2407:7000:a234:4900:45e0:7b… at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) at java.base/java.net.URL.openStream(URL.java:1165) at org.torproject.metrics.exonerator.ExoneraTorServlet.queryBackend(ExoneraTorServlet.java:314) at org.torproject.metrics.exonerator.ExoneraTorServlet.doGet(ExoneraTorServlet.java:100) at org.torproject.metrics.web.ExoneraTorWrapperServlet.doGet(ExoneraTorWrapperServlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.base/java.lang.Thread.run(Thread.java:829) 2021-11-12 06:11:41,512 ERROR o.t.m.e.ExoneraTorServlet:319 Backend query failed. java.io.IOException: Server returned HTTP response code: 500 for URL: https://exonerator.torproject.org/query.json?ip=2404:4408:639b:9500:915a:da… at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) at java.base/java.net.URL.openStream(URL.java:1165) at org.torproject.metrics.exonerator.ExoneraTorServlet.queryBackend(ExoneraTorServlet.java:314) at org.torproject.metrics.exonerator.ExoneraTorServlet.doGet(ExoneraTorServlet.java:100) at org.torproject.metrics.web.ExoneraTorWrapperServlet.doGet(ExoneraTorWrapperServlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.base/java.lang.Thread.run(Thread.java:829)

1 0

Cron <metrics@meronense> cd /srv/metrics.torproject.org/metrics/ && ./run-web.sh
by root＠meronense.torproject.org 22 Nov '21

22 Nov '21

2021-11-22 08:40:10,899 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file.

1 0

Cron <metrics@meronense> cd /srv/metrics.torproject.org/metrics/work/modules/logs && cat metrics.log | grep -v "Could not insert bandwidth "
by root＠meronense.torproject.org 22 Nov '21

22 Nov '21

2021-11-22 08:40:02,420 INFO o.t.m.s.main.Main:32 Starting metrics update run. 2021-11-22 08:40:02,473 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 08:40:02,476 INFO o.t.d.i.DescriptorIndexCollector:47 Starting descriptor collection. 2021-11-22 08:40:02,476 INFO o.t.d.i.DescriptorIndexCollector:63 Indexing local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 08:40:06,386 INFO o.t.d.i.DescriptorIndexCollector:75 Fetching remote index file https://collector.torproject.org/index/index.json. 2021-11-22 08:40:07,128 INFO o.t.d.i.DescriptorIndexCollector:84 Fetching remote files from https://collector.torproject.org. 2021-11-22 08:40:10,899 WARN o.t.d.i.DescriptorIndexCollector:132 Fetched remote file 2021-11-21-04-00-00-vote-0232AF901C31A04EE9848595AF9BB7620D4C5B2E-9DAFAF1CE0ABC369CCD86FB1D7E38C33CCFAB1D6 from https://collector.torproject.org has a size of 4669122 bytes which is different from the expected 1273856 bytes. Not storing this file. 2021-11-22 08:40:28,727 INFO o.t.d.i.DescriptorIndexCollector:90 Deleting extraneous files from local directory /srv/metrics.torproject.org/metrics/work/shared/in. 2021-11-22 08:40:29,959 INFO o.t.d.i.DescriptorIndexCollector:95 Finished descriptor collection. 2021-11-22 08:40:29,959 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.collectdescs.Main module. 2021-11-22 08:40:29,959 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 08:40:37,280 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.connbidirect.Main module. 2021-11-22 08:40:37,280 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 08:40:37,282 INFO o.t.m.s.o.Main:46 Starting onionperf module. 2021-11-22 08:40:37,282 INFO o.t.m.s.o.Main:67 Connecting to database. 2021-11-22 08:40:37,746 INFO o.t.m.s.o.Main:70 Successfully connected to database. 2021-11-22 08:45:26,703 INFO o.t.m.s.o.Main:261 Querying statistics including partials from database. 2021-11-22 08:46:12,639 INFO o.t.m.s.o.Main:402 Writing 59089 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-including-partials.csv. 2021-11-22 08:46:13,521 INFO o.t.m.s.o.Main:286 Querying timeout and failure statistics from database. 2021-11-22 08:46:16,429 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-failures.csv. 2021-11-22 08:46:16,454 INFO o.t.m.s.o.Main:309 Querying buildtime statistics from database. 2021-11-22 08:46:58,997 INFO o.t.m.s.o.Main:402 Writing 47674 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/buildtimes.csv. 2021-11-22 08:46:59,798 INFO o.t.m.s.o.Main:332 Querying latency statistics from database. 2021-11-22 08:47:12,732 INFO o.t.m.s.o.Main:402 Writing 18734 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/latencies.csv. 2021-11-22 08:47:12,794 INFO o.t.m.s.o.Main:357 Querying throughput statistics from database. 2021-11-22 08:47:24,178 INFO o.t.m.s.o.Main:402 Writing 19701 lines to /srv/metrics.torproject.org/metrics/work/modules/onionperf/stats/onionperf-throughput.csv. 2021-11-22 08:47:24,180 INFO o.t.m.s.o.Main:409 Disconnecting from database. 2021-11-22 08:47:24,180 INFO o.t.m.s.o.Main:62 Terminated onionperf module. 2021-11-22 08:47:24,181 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.onionperf.Main module. 2021-11-22 08:47:24,181 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 08:47:24,181 INFO o.t.m.s.bwhist.Main:37 Starting bwhist module. 2021-11-22 08:47:24,181 INFO o.t.m.s.bwhist.Main:39 Reading descriptors and inserting relevant parts into the database. 2021-11-22 08:57:09,946 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 150891 network status entries and 137335 bandwidth history elements 2021-11-22 08:57:12,269 INFO o.t.m.s.bwhist.Main:50 Aggregating database entries. 2021-11-22 08:58:06,669 INFO o.t.m.s.b.RelayDescriptorDatabaseImporter:543 Finished importing relay descriptors: 150891 network status entries and 137335 bandwidth history elements 2021-11-22 08:58:08,270 INFO o.t.m.s.bwhist.Main:53 Querying aggregated statistics from the database. 2021-11-22 08:58:08,690 INFO o.t.m.s.bwhist.Main:57 Closing database connection. 2021-11-22 08:58:08,691 INFO o.t.m.s.bwhist.Main:60 Terminating bwhist module. 2021-11-22 08:58:08,691 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.bwhist.Main module. 2021-11-22 08:58:08,691 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 08:59:02,870 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.advbwdist.Main module. 2021-11-22 08:59:02,915 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 08:59:02,916 INFO o.t.m.s.hidserv.Main:37 Initializing parser and reading parse history... 2021-11-22 08:59:03,068 INFO o.t.m.s.hidserv.Main:52 Parsing descriptors... 2021-11-22 09:00:47,073 INFO o.t.m.s.hidserv.Main:59 Writing parse history... 2021-11-22 09:00:47,122 INFO o.t.m.s.hidserv.Main:66 Extrapolating statistics... 2021-11-22 09:06:23,694 INFO o.t.m.s.hidserv.Main:82 Aggregating v2 statistics... 2021-11-22 09:08:02,717 INFO o.t.m.s.hidserv.Main:98 Aggregating v3 statistics... 2021-11-22 09:08:02,951 INFO o.t.m.s.hidserv.Main:108 Terminating. 2021-11-22 09:08:02,952 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.hidserv.Main module. 2021-11-22 09:08:02,954 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.clients.Main module. 2021-11-22 09:08:02,971 INFO o.t.m.s.clients.Main:36 Starting clients module. 2021-11-22 09:08:02,972 INFO o.t.m.s.clients.Main:38 Connecting to database. 2021-11-22 09:08:03,222 INFO o.t.m.s.clients.Main:41 Reading relay descriptors and importing relevant parts into the database. 2021-11-22 09:09:30,378 INFO o.t.m.s.clients.Main:45 Reading bridge descriptors and importing relevant parts into the database. 2021-11-22 09:10:07,372 INFO o.t.m.s.clients.Main:49 Processing newly imported data. 2021-11-22 10:19:27,740 INFO o.t.m.s.clients.Main:53 Querying aggregated statistics from the database. 2021-11-22 10:19:49,856 INFO o.t.m.s.clients.Main:59 Disconnecting from database. 2021-11-22 10:19:49,889 INFO o.t.m.s.clients.Main:62 Running detector. 2021-11-22 10:20:25,828 INFO o.t.m.s.clients.Main:65 Terminating clients module. 2021-11-22 10:20:25,846 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.clients.Main module. 2021-11-22 10:20:25,847 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.servers.Main module. 2021-11-22 10:20:25,865 INFO o.t.m.s.servers.Main:43 Starting servers module. 2021-11-22 10:20:25,865 INFO o.t.m.s.servers.Main:45 Reading descriptors and inserting relevant parts into the database. 2021-11-22 10:46:15,034 INFO o.t.m.s.servers.Main:77 Aggregating database entries. 2021-11-22 11:48:39,634 INFO o.t.m.s.servers.Main:80 Committing all updated parts in the database. 2021-11-22 11:48:40,099 INFO o.t.m.s.servers.Main:90 Querying aggregated statistics from the database. 2021-11-22 12:05:12,459 INFO o.t.m.s.servers.Main:105 Terminating servers module. 2021-11-22 12:05:12,666 INFO o.t.m.s.main.Main:73 Completed org.torproject.metrics.stats.servers.Main module. 2021-11-22 12:05:12,667 INFO o.t.m.s.main.Main:70 Starting org.torproject.metrics.stats.webstats.Main module. 2021-11-22 12:05:12,708 INFO o.t.m.s.w.Main:75 Starting webstats module. 2021-11-22 12:05:12,708 INFO o.t.m.s.w.Main:92 Connecting to database. 2021-11-22 12:05:13,082 INFO o.t.m.s.w.Main:95 Successfully connected to database. 2021-11-22 12:05:13,083 INFO o.t.m.s.w.Main:101 Querying previously imported log files. 2021-11-22 12:05:15,131 INFO o.t.m.s.w.Main:113 Found 284694 previously imported log files.

1 0

Cron <collector@colchicifolium> cd /srv/collector.torproject.org/collector/logs/ && cat metrics.log | grep -E "WARN|ERROR|FATAL" | grep -v "Overwriting existing descriptor file"
by root＠colchicifolium.torproject.org 22 Nov '21

22 Nov '21

2021-11-22 00:08:04,136 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (14.587198273283384). 2021-11-22 00:37:40,517 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (12.907462238906465). 2021-11-22 01:07:24,780 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (24.72445352188792). 2021-11-22 01:37:05,902 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (13.88107443670385). 2021-11-22 02:07:13,203 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.164231288529113). 2021-11-22 02:37:12,451 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.751352221616898). 2021-11-22 03:07:12,216 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (19.345599438098702). 2021-11-22 03:37:19,995 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (6.449838585798688). 2021-11-22 04:07:13,549 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (18.329712786921036). 2021-11-22 04:37:01,940 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.893314494780442). 2021-11-22 05:07:03,980 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.774450587468424). 2021-11-22 05:36:59,696 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.645816731665365). 2021-11-22 06:07:03,023 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.33444985163934). 2021-11-22 06:37:02,349 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (6.536175454896977). 2021-11-22 07:07:30,704 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.135293049740758). 2021-11-22 07:36:54,221 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.791270442540856). 2021-11-22 08:07:06,333 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (19.688255404232965). 2021-11-22 08:37:10,184 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (19.23868510150478). 2021-11-22 09:07:36,819 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (28.980679040977055). 2021-11-22 09:37:12,838 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (27.226004619578713). 2021-11-22 10:07:23,833 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (39.06472173112308). 2021-11-22 10:37:36,242 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (22.086299169982865). 2021-11-22 11:07:31,834 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (34.01663258205433). 2021-11-22 11:37:22,029 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (28.38917369911307). 2021-11-22 12:07:58,396 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (33.32354779922783).

1 0

Cron <collector@colchicifolium> cd /srv/collector.torproject.org/collector/logs/ && cat metrics.log | grep -E "WARN|ERROR|FATAL" | grep -v "Overwriting existing descriptor file"
by root＠colchicifolium.torproject.org 22 Nov '21

22 Nov '21

2021-11-22 00:08:04,136 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (14.587198273283384). 2021-11-22 00:37:40,517 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (12.907462238906465). 2021-11-22 01:07:24,780 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (24.72445352188792). 2021-11-22 01:37:05,902 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (13.88107443670385). 2021-11-22 02:07:13,203 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.164231288529113). 2021-11-22 02:37:12,451 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.751352221616898). 2021-11-22 03:07:12,216 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (19.345599438098702). 2021-11-22 03:37:19,995 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (6.449838585798688). 2021-11-22 04:07:13,549 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (18.329712786921036). 2021-11-22 04:37:01,940 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.893314494780442). 2021-11-22 05:07:03,980 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.774450587468424). 2021-11-22 05:36:59,696 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (7.645816731665365). 2021-11-22 06:07:03,023 WARN o.t.m.c.r.ReferenceChecker:314 Missing too many referenced descriptors (8.33444985163934).

1 0