[metrics-team] [network-team] Back from Montreal, anything I should do first?

[Karsten Loesing]

On 2017-10-16 19:19, Damian Johnson wrote:
>> I just returned from the Montreal meeting today and have a ton of things
>> on my todo list.
> 
> Hi Karsten!

Hi Damian!

I'm replying to you and both lists here. However, metrics-team@ is a
public list whereas network-team@ is not. FYI.

> After our discussion I continued to mull things over but
> unfortunately by the time I mentally got my ducks in a row you were
> flying out. Oh well - I suppose it's a good sign when the hallway
> track is crammed full of so many interesting conversations that you
> can't conclude them all.
> 
> The reason I brought up languages as the elephant in the room is
> because it silos our efforts. The reason I don't attend the metrics
> meetings is because there aren't many points where we can collaborate
> - you and iwakeh improve the java codebases (metrics-lib, exonerator,
> oninoo, collector, etc) whereas Philipp, Isis, and I help progress the
> python side (stem, bridgedb, doctor, exitmap, etc). I would *love* to
> work directly with you and in past meetings we've discussed crossover
> efforts though really the language barrier has resulted in two
> distinct stacks that don't talk to much. Your work on metrics-lib
> doesn't benefit our python stacks, my work on stem can't benefit you,
> and every descriptor change gets implemented in two different
> libraries.
> 
> Really a pity since I'd love for our work to help each other but c'est
> la vie. The two of us have discussed this at dev meets for half a
> decade now without much progress so guess the duplication of efforts
> will remain for the foreseeable future. If you have any thoughts on
> how to collaborate more please don't hesitate to let me know - this is
> a chasm I'd love to bridge. :P

I'd love to work more closely with you and other Python folks. And I
don't rule out that it might become a reality at some point in the
future. But I think it's safe to assume that this future won't be in the
next 12 months which is our current planning horizon.

> That aside, at the dev meeting I floated the idea of pyoninoo because
> after years of work Nyx is *finally* nearing release so I'll be coming
> up for air soon to look for another project. Present ideas I have
> are...
> 
> 1. Look into Python/Rust bindings like rust-cpython. We actually have
> *three* descriptor parsers: metrics-lib (java), stem (python), and tor
> itself (C). That last is about to be reimplemented in Rust. Maybe I
> should write it? Research needed, but if the bindings are good enough
> maybe Stem and Tor can share a parser.

This is more for the network team, I guess.

> 2. Pyonionoo. This might dovetail nicely with my day job depending on
> how things go at work as a way of learning Boto.

As I said in Montreal, it would be best to focus on the database parts
of such a re-implementation. I'd be curious.

> 3. Take over Atlas maintenance. For a long time Atlas has looked like
> a fun project to me and could be a neat way of improving my
> javascript-fu. This is why I cringed when you mentioned merging Atlas
> with the metrics site. Metrics is primarily a service for researchers
> whereas Atlas is a site for relay operators. I do see the overlap you
> described but personally I think these sites should remain separate.

Well, even after merging Atlas into Metrics, it will stay a JavaScript
application, at least for the foreseeable future. And it's likely not
even necessary to take over Atlas maintenance, because irl takes care of
it. You could easily take a bunch of tickets there and run with them,
without long-term commitment.

> 4. ORPort communication via Stem. Tim has a fantastic proof of concept
> for making a one-hop circuit in Python, which could finally let Stem
> retrieve descriptors from a relay's ORPort rather than the DirPort
> (letting us deprecate the later).

Sounds great to me! I'd be happy to contribute to that effort of
deprecating the DirPort by adapting Tim's or your proof of concept to
Java and using it in CollecTor.

> 5. Web dashboard for relay operators. My original plan was to follow
> up Nyx with a web version where relay operators could browse localhost
> to see bandwidth graphs, event logs, etc. But honestly with so many
> other interesting projects I'll probably defer this.

You should check with relay operators how much they'd care about this
vs. other efforts. Maybe also check with irl who generally has a good
sense of relay operators' needs from receiving lots of questions and
other feedback from Atlas users.

> Some of these have overlap with Metrics and others with the Network
> team so including both teams here so they're aware of my prospective
> plans. Lots of fun directions to go. :P

As indicated above, happy to keep this discussion on both lists. But if
others prefer, we could as well move it to one of the lists or even to
tor-dev at .

> Cheers! -Damian

All the best,
Karsten

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 528 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/metrics-team/attachments/20171016/a3641c10/attachment.sig>