Notes from 12 April 2018 Simple Bandwidth Scanner Meeting
See below for the pad notes. Next meeting is scheduled 19 April 2018 at 2200 UTC in #tor-meeting. (This one was held in #tor-dev, but we should use meetbot). ------------- Simple Bandwidth Scanner meeting 12 April 2018 #### Updates/Status messages #### pastly: What's on my plate? <- doesn't have to be all in your plate :P - Test coverage getting closer to 100% - Immediate future: switch to standard python logging module, which is quite good - Improving documentation - Checking results against torflow - Monitor CPU of sbws client/server - +1 on considering asyncio - See how chutney generates random strings - Run testnet authority - Reach out to current auths about running sbws/torflow and adding me as an auth juga: - open/close PRs/issues about things to improve in doc, refactor code, etc..., but not changing functionality - re. doc: - thought to update sbws spec (or create other) to doc differences with Torflow, not sure it's useful - i'd document further some of the classes/functions (as measure_relay) - code doc vs spec (see below) - find box to run other sbws, bwauth also in testnet? ## Topic: what is still missing for milestone 1? (aka 1st release, v1.0.0) - could we create all tickets needed to achive it? - maybe previous list is enough? Missing: - A consensus parameter stating the scaling factor - sbws config option to set fallback if no consensus param - `sbws generate` code to use the consensus param - https://stem.torproject.org/api/descriptor/networkstatus.html#stem.descripto... - Correlation coefficient on comparision graphs ## Topic: comparing to torflow tah - Can we make the test sbws deployment a little bigger? - What else needs to be compared? teor: actually running it in a voting network, to check the feedback loop (if any) the scaling - Conclusions after comparing? - what we could think to change/improve after comparing? Graphs pastly can explain: - sbws vs moria, sorted by sbws: https://share.riseup.net/#-W_zqcv-08AX4SnOgTatUw - sorted by moria: https://share.riseup.net/#URXp6NccZHEhOPFJQcfO4w teor: the correlation seems good here If we're going to use these charts to compare, please compare two existing bwauths See: https://share.riseup.net/#lPGcIrgHp3ftnvTHUKqOKg (but ignore the sbws-scaled line, it's wrong wrong wrong) ## Topic: convincing people to run sbws juga: maybe something to do when 1st sbws release? pastly: yes, probalby. unless we need to convince testnet people <- ah, right i was thinking on the Tor net ## Topic: status of open sourcing sbws - No real update. Time is still passing. ## Topic: specifications torflow/BwAuthority: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/R..., https://ohmygodel.com/publications/peerflow-popets2017.pdf has a section that also makes a nice summary sbws: https://github.com/pastly/simple-bw-scanner/blob/master/docs/source/specific... (ask Pastly for access) bwscanner: no spec, but reading https://github.com/TheTorProject/bwscanner/blob/develop/bwscanner/circuit.py... it looks like a Torflow clone <- almost :) We need a spec for the v3bw file that tor reads (in torspec/dir-spec.txt) We need a spec for bwauth migration, including acceptance criteria for new bwauth implementations Scanners should have their own detailed design documents
I'm happy to run a sbws alongside my torflow. It will let us compare bw numbers apples to apples too. My only difficulty is being unable to spend significant time to diagnose why it doesn't work, if it doesn't work. If it's at the point I should give it a shot, point me at some instructions :) -tom On Thu, Apr 12, 2018, 7:38 PM Matt Traudt <pastly@torproject.org> wrote:
See below for the pad notes. Next meeting is scheduled 19 April 2018 at 2200 UTC in #tor-meeting. (This one was held in #tor-dev, but we should use meetbot).
-------------
Simple Bandwidth Scanner meeting 12 April 2018
#### Updates/Status messages ####
pastly: What's on my plate? <- doesn't have to be all in your plate :P - Test coverage getting closer to 100% - Immediate future: switch to standard python logging module, which is quite good - Improving documentation - Checking results against torflow - Monitor CPU of sbws client/server - +1 on considering asyncio - See how chutney generates random strings - Run testnet authority - Reach out to current auths about running sbws/torflow and adding me as an auth
juga: - open/close PRs/issues about things to improve in doc, refactor code, etc..., but not changing functionality - re. doc: - thought to update sbws spec (or create other) to doc differences with Torflow, not sure it's useful - i'd document further some of the classes/functions (as measure_relay) - code doc vs spec (see below) - find box to run other sbws, bwauth also in testnet?
## Topic: what is still missing for milestone 1? (aka 1st release, v1.0.0) - could we create all tickets needed to achive it? - maybe previous list is enough? Missing: - A consensus parameter stating the scaling factor - sbws config option to set fallback if no consensus param - `sbws generate` code to use the consensus param
-
https://stem.torproject.org/api/descriptor/networkstatus.html#stem.descripto...
- Correlation coefficient on comparision graphs
## Topic: comparing to torflow tah - Can we make the test sbws deployment a little bigger? - What else needs to be compared?
teor: actually running it in a voting network, to check the feedback loop (if any) the scaling
- Conclusions after comparing? - what we could think to change/improve after comparing?
Graphs pastly can explain: - sbws vs moria, sorted by sbws: https://share.riseup.net/#-W_zqcv-08AX4SnOgTatUw - sorted by moria: https://share.riseup.net/#URXp6NccZHEhOPFJQcfO4w
teor: the correlation seems good here If we're going to use these charts to compare, please compare two existing bwauths See: https://share.riseup.net/#lPGcIrgHp3ftnvTHUKqOKg (but ignore the sbws-scaled line, it's wrong wrong wrong)
## Topic: convincing people to run sbws juga: maybe something to do when 1st sbws release? pastly: yes, probalby. unless we need to convince testnet people <- ah, right i was thinking on the Tor net
## Topic: status of open sourcing sbws - No real update. Time is still passing.
## Topic: specifications
torflow/BwAuthority:
https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/R... , https://ohmygodel.com/publications/peerflow-popets2017.pdf has a section that also makes a nice summary sbws:
https://github.com/pastly/simple-bw-scanner/blob/master/docs/source/specific... (ask Pastly for access) bwscanner: no spec, but reading
https://github.com/TheTorProject/bwscanner/blob/develop/bwscanner/circuit.py... it looks like a Torflow clone <- almost :)
We need a spec for the v3bw file that tor reads (in torspec/dir-spec.txt) We need a spec for bwauth migration, including acceptance criteria for new bwauth implementations Scanners should have their own detailed design documents
_______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Thanks Tom! There's a few more things that I think we need to figure out before we take you up on this offer. I'll keep you in mind though, because I think having sbws and torflow running side by side is something that should happen very soon. Stay tuned! And thanks for your help obtaining torflow data recently. Matt On 4/12/18 21:50, Tom Ritter wrote:
I'm happy to run a sbws alongside my torflow. It will let us compare bw numbers apples to apples too. My only difficulty is being unable to spend significant time to diagnose why it doesn't work, if it doesn't work.
If it's at the point I should give it a shot, point me at some instructions :)
-tom
https://github.com/pastly/simple-bw-scanner/blob/master/docs/source/specific... (ask Pastly for access)
Hi Matt, why is this repo read restricted? I was idly curious to see the code of sbws and was surprised it's effectively closed source.
On 4/13/18 12:26, Damian Johnson wrote:
https://github.com/pastly/simple-bw-scanner/blob/master/docs/source/specific... (ask Pastly for access)
Hi Matt, why is this repo read restricted? I was idly curious to see the code of sbws and was surprised it's effectively closed source.
Good question! While I didn't necessarily work on sbws while at my place of work, I couldn't rationalize that it is unrelated to my day job. Thus I need to get permission from my employer in order to release sbws. I've already submitted that paperwork and expect to get it back in about 2 weeks. It is licensed under CC0 and is therefore in the public domain (or will be... depending on how you want to interpret the situation). Fun side note: little-t-tor itself had to go through the same process. Sorry for the inconvenience. Matt
Good question!
While I didn't necessarily work on sbws while at my place of work, I couldn't rationalize that it is unrelated to my day job. Thus I need to get permission from my employer in order to release sbws.
I've already submitted that paperwork and expect to get it back in about 2 weeks.
It is licensed under CC0 and is therefore in the public domain (or will be... depending on how you want to interpret the situation).
Fun side note: little-t-tor itself had to go through the same process.
Sorry for the inconvenience.
Ahhhh! Mystery solved. Makes complete sense - thanks Matt!
participants (3)
-
Damian Johnson -
Matt Traudt -
Tom Ritter