[tor-project] Bandwidth scanner report for the last six months

[juga]

Hi all!

these are some of the issues or tasks we have worked on during the last 
six months:

We improved and run a tool [0] to find sbws bugs and differences between 
bandwidth files generated by the bandwidth authorities.
One of the issues that it helped us to find is why sbws was reporting 
less relays in the bandwidth file compared to Torflow [1].
It was just that Torflow uses data from the last 28 days when generating 
the bandwidth file, while sbws was only using data from the last 5 days.
We also found that there're a bunch of relays that are up during the 
first days of the month but not the last ones.

In sbws, some circuits were failing to be created because the relay to 
be measured or its helper wasn't in the consensus anymore.
We changed that [2] to only measure relays in the consensus.
The relays that are not currently in the consensus but were in the last 
28 days and were measured, are included in the bandwidth file.

We fixed a lot of bugs [3] to make sbws report relays' weights in a more 
similar way to Torflow, to be able to deploy it to all bandwidth 
authorities.

We also continued to improve test coverage and documentation [4].

We created an script to calculate bandwidth usage retrieving descriptors 
and parsing them [5] and another script to monitor relays' overload [6].

Finally, sbws version 1.2.0 [7] has been released and bastet has 
deployed it.

The main challenge has been tracking down discrepancies and bugs between 
sbws and Torflow.

Thanks,
juga.

[0] https://gitlab.torproject.org/juga/bwauthealth
[1] https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40017
[2] https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40037
[3] https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40016,
https://gitlab.torproject.org/tpo/core/torspec/-/issues/35,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40020,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40023,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40027,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40029,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40038,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40040,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40041,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40043,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40049,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40051,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40054,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40059,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40060,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40069,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40076,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40080
[4] https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/33149,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/33599,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40055,
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40056
https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40070
[5] 
https://gitlab.torproject.org/tpo/network-health/helper-scripts/-/issues/8
[6] https://gitlab.torproject.org/tpo/network-health/team/-/issues/35
[7] https://gitlab.torproject.org/tpo/network-health/sbws/-/releases#v1.2.0

-- 
they/them

http://zkaan2xfbuxia2wpf7ofnkbz6r5zdbbvxbunvp5g2iebopbfc4iqmbad.onion/vks/v1/by-fingerprint/2DA81D01455C3A0032198850F305447AF806D46B