[metrics-team] Back from Montreal, anything I should do first?

Damian Johnson atagar at torproject.org
Mon Oct 16 17:19:20 UTC 2017


> I just returned from the Montreal meeting today and have a ton of things
> on my todo list.

Hi Karsten! After our discussion I continued to mull things over but
unfortunately by the time I mentally got my ducks in a row you were
flying out. Oh well - I suppose it's a good sign when the hallway
track is crammed full of so many interesting conversations that you
can't conclude them all.

The reason I brought up languages as the elephant in the room is
because it silos our efforts. The reason I don't attend the metrics
meetings is because there aren't many points where we can collaborate
- you and iwakeh improve the java codebases (metrics-lib, exonerator,
oninoo, collector, etc) whereas Philipp, Isis, and I help progress the
python side (stem, bridgedb, doctor, exitmap, etc). I would *love* to
work directly with you and in past meetings we've discussed crossover
efforts though really the language barrier has resulted in two
distinct stacks that don't talk to much. Your work on metrics-lib
doesn't benefit our python stacks, my work on stem can't benefit you,
and every descriptor change gets implemented in two different
libraries.

Really a pity since I'd love for our work to help each other but c'est
la vie. The two of us have discussed this at dev meets for half a
decade now without much progress so guess the duplication of efforts
will remain for the foreseeable future. If you have any thoughts on
how to collaborate more please don't hesitate to let me know - this is
a chasm I'd love to bridge. :P

That aside, at the dev meeting I floated the idea of pyoninoo because
after years of work Nyx is *finally* nearing release so I'll be coming
up for air soon to look for another project. Present ideas I have
are...

1. Look into Python/Rust bindings like rust-cpython. We actually have
*three* descriptor parsers: metrics-lib (java), stem (python), and tor
itself (C). That last is about to be reimplemented in Rust. Maybe I
should write it? Research needed, but if the bindings are good enough
maybe Stem and Tor can share a parser.

2. Pyonionoo. This might dovetail nicely with my day job depending on
how things go at work as a way of learning Boto.

3. Take over Atlas maintenance. For a long time Atlas has looked like
a fun project to me and could be a neat way of improving my
javascript-fu. This is why I cringed when you mentioned merging Atlas
with the metrics site. Metrics is primarily a service for researchers
whereas Atlas is a site for relay operators. I do see the overlap you
described but personally I think these sites should remain separate.

4. ORPort communication via Stem. Tim has a fantastic proof of concept
for making a one-hop circuit in Python, which could finally let Stem
retrieve descriptors from a relay's ORPort rather than the DirPort
(letting us deprecate the later).

5. Web dashboard for relay operators. My original plan was to follow
up Nyx with a web version where relay operators could browse localhost
to see bandwidth graphs, event logs, etc. But honestly with so many
other interesting projects I'll probably defer this.

Some of these have overlap with Metrics and others with the Network
team so including both teams here so they're aware of my prospective
plans. Lots of fun directions to go. :P

Cheers! -Damian


More information about the metrics-team mailing list