Hi David, On 6/30/26 6:06 PM, David Fifield via network-health wrote:
For the past 3 months, collector-rs has been making snowflake-stats tarballs in an incorrect format. The recently committed fix for this problem implies that the incorrect format was the result of the Snowflake broker changing the format of its metrics. But this is not the case: Snowflake broker metrics have always (since 2019) contained multiple days of metrics. This was by design, and Java Collector was always able to parse that format. Since the recent fix, collector-rs treats snowflake-stats as a special case ("skip_seed_for_archive"), but Java Collector didn't treat snowflake-stats as exceptional—it parsed snowflake-stats in the same way as other types that have multiple days of data in one file, like bridgedb-metrics and bridgestrap-stats, using a method called parseOneOrMoreDescriptors.
[...] thanks for reporting this and apologies for the misunderstanding on snowflake-stats format. It was me trying to solve an issue fast in production (after some months with my mind on other projects) to be able to continue re-designing and re-factoring [1] a minimal version [2] that was just storing all the descriptors as they were fetched without parsing (not really mentioned here [3], but we agreed that a MVP could just do this). IMO, it's better to re-design a new software (and ensure it has all the needed functionality, tests, etc.) before getting it in production. In this case, it was put into production early, because Java CollecTor was already having other issues and we ended up with a production code for collector-rs we need to maintain, while still working on having a better collector-rs. I'm currently working on #35 and changing the base structs (File and Filegen) into different structs for different descriptors/documents and having models for both single and "multi" descriptor/documents content. This's going to be a major (in the internal API) change and "skip_seed_for_archive" will disappear. Meanwhile, i can create a mr to correct the wrong comment on snowflake-stats. Would this temporal fix work for you? I'd ask if at least !26 solves #48, but #70, the issue to deploy this in production, is not solved yet. juga [1] eg. https://gitlab.torproject.org/tpo/network-health/metrics/collector-rs/-/work..., https://gitlab.torproject.org/tpo/network-health/metrics/collector-rs/-/work... [2] https://gitlab.torproject.org/tpo/network-health/metrics/collector-rs/-/comm... [3] https://juga.pages.torproject.net/collector_docs/collector_new/analysis.html...