[tor-bugs] #28563 [Core Tor/sbws]: Work out how sbws can report excluded relays in the bandwidth file

[Tor Bug Tracker & Wiki]

#28563: Work out how sbws can report excluded relays in the bandwidth file
---------------------------+-------------------------------------
 Reporter:  teor           |          Owner:  (none)
     Type:  defect         |         Status:  new
 Priority:  Medium         |      Milestone:  sbws 1.0 (MVP must)
Component:  Core Tor/sbws  |        Version:
 Severity:  Normal         |     Resolution:
 Keywords:  tor-bwauth     |  Actual Points:
Parent ID:  #28547         |         Points:
 Reviewer:                 |        Sponsor:
---------------------------+-------------------------------------

Comment (by juga):

 I see some inconvenients to do this:
 - once we figure out why are being relays excluded, we might not want to
 keep the same format.
 - we need to wait until longclaw update to the code that publish the files
 - it'd add like around 1000 extra relays with some extra data, though this
 might not be a problem.
 - the delay that implies creating the spec before

 I was thinking either on something temporal:
 1. produce a different file, with the relays excluded and useful data*
 2. implement other script to dump the data to a DB. It sounds kind of
 crazy, but it might not be much work and from that it's easier to make
 queries
 While currently it's only me accessing to the original data, i can publish
 the results of that.

 *Currently relays excluded, can be because:
 - circuits timeout, this is already in the raw results file
 - when scaling, doesn't find 2 measurements that are at least 1 day away
 and 5 days recent
 - something else we don't know yet

 What about if i try one of the two other approaches before?. Otherwise i'm
 fine with this.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28563#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online