[tor-dev] Brief state of sbws thoughts

[teor]

Hi,

> On 20 Jul 2018, at 01:16, juga <juga at riseup.net> wrote:
> 
> Matt Traudt:
>> Teor, Juga
>> 
>> There's a lot of things fighting for my attention right now, so you
>> might have noticed I've slowed way down on attending to sbws
>> tickets/PRs/etc. I think time will free up in the next few days.
>> 
>> I think sbws is in a very good place code-wise right now. I don't think
>> much more **has** to be done to the code. Even though I enjoy adding
>> things like the state file (GHPR#236 [2]), I don't think that was a good
>> use of my time.
>> 
>> It looks like there's a lot of check boxes Juga has made regarding
>> making a Debian package[0]. Those should get checked. These are important.
>> 
>> However, I think the absolute most important thing for us to be spending
>> our time on right now is deciding what "good" results are and verifying
>> sbws produces "good" results.

You’re right -  we need to know if we can switch to sbws, and we can’t use
sbws unless it has reasonable results.

If the results aren’t reasonable, we might need to:
* do further processing on the sbws results (like scaling)
* change the sbws measurement design

The good news is that sbws ranks are approximately the same as torflow ranks.
So the measurement design is probably ok.

But torflow weights are larger (max 100,000) than sbws weights (max 4000),
so we will need to scale the sbws results.

torflow results are also steeper than sbws results: the ratio between high
and low ranked relays is 1000:1 in torflow, but 10:1 in sbws.

If we want to, we can make sbws match torflow by defining a scaling algorithm
that scales large relays more than small relays. But we could also decide that
the flatter sbws curve is better for the network, because high-weight relays
are overloaded.

Let’s do a few more experiments before we decide.

>> To accomplish this, I think one of the two suggestions I made in a
>> comment on GH#182 [1] (quoted here) is what we should be doing.
>> 
>> 1. Run torflow and sbws side-by-side (but not at the same time) to
>> remove more variables. This has the added benefit of us having access to
>> the raw scanner results from torflow before it does whatever magic
>> scaling it does. OR
> 
> In that ticket you also mentioned that someone that already runs torflow
> should also run sbws.
> I said i can run both, and still the case if needed.

Ok, so juga can run sbws and torflow at different times on the same machine.

> On 20 Jul 2018, at 01:34, Tom Ritter <tom at ritter.vg> wrote:
> 
> I'm happy and prepared to run sbws and torflow side by side. I'm a
> little less swamped than I was a month ago.  I don't need a debian
> package; I'd rather run it from a git clone.
> 
> I think the only things I can't do are
> a) give you access to the box directly (but I can make whatever
> files/logs/raw results that you want available to you over HTTP)
> b) stop running torflow. (Unless we're ready to switch a live bwauth
> over to sbws.)

And tom can run sbws and torflow at the same time on the same machine.

I think we should run both comparisons, wait a week so they are in a stable
state, and then check the results for a few weeks.

T

>> [0]: https://trac.torproject.org/projects/tor/ticket/26848
>> [1]:
>> https://github.com/pastly/simple-bw-scanner/issues/182#issuecomment-404250053
>> [2]: https://github.com/pastly/simple-bw-scanner/pull/236


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20180720/f3b00e63/attachment-0001.sig>