[metrics-team] Metrics team priorities

teor teor2345 at gmail.com
Wed Jan 17 22:36:23 UTC 2018


Hi Karsten, iwakeh,

Thank you for your hard work, and the hard work of your many volunteers
(I've mainly worked with irl, tjr, and ana, but thanks to others as well.)

> On 17 Jan 2018, at 21:27, Karsten Loesing <karsten at torproject.org> wrote:
> 
> Hello everyone,
> 
> I'm writing this message, because I feel like the metrics team is trying
> to achieve too many things at the moment.
> 
> Let me give you some context: A few months ago we received a grant to
> write some long-needed documentation on how we're processing data, which
> started in October 2017 and runs until September 2018.

I saw the draft of this, but haven't had time to comment yet.

> At the same time we're trying to implement new features requested by
> other Tor folks, including adding sanitized web server logs to
> CollecTor, showing metrics timeline events underneath graphs, adding
> graphs on IPv6 servers to Tor Metrics, adding new Onionoo features to
> make it more useful for keeping the Tor network healthy, and so on.

I thought of adding a list of tasks to the pad. But I'm not sure which tasks
are done. And I'm not sure which tasks are future tasks.

I know I've asked for (or contributed to):
* IPv6 graphs
* 1 & 3 month Relay Search bandwidth graphs missing on newer Tor
  versions
* authority votes in Onionoo (for bandwidth votes on the Relay Search map)
* Tor versions not updating for a while
* map changes
* fallback directory mirror flags (and other synthetic flags)
* PrivCount in Tor

Let me know if I have asked for anything else.

Here is my take on each of these:

* The IPv6 graphs are enough for us right now. We don't need anything
  else. Tor won't produce new IPv6 stats until at least 0.3.4 (late 2018).

* Bandwidth graphs confuse relay operators, but that's ok.
  This change is in Tor 0.3.2.9 and all the next backport releases.
  Maybe we can delete the 1 & 3 month graphs as a quick fix?
  Then we can bring them back when the Onionoo bug is fixed.

* We want bandwidth votes so we can map bandwidth authority bias.
  But this is waiting on other tasks. We need to talk to bwauth operators
  to change servers. There is no code ready for any new scanners.
  When we do scanner or server changes, we can ask irl for a once-off
  analysis. Hourly maps can wait until we actually have new code.

* We also want to be able to take test bandwidth authorities and map
  their output. I don't think test data belongs in Onionoo. We can get a
  once-off analysis for this when we need it.

* Thanks for the analysis on the tor versions. There is nothing major
  wrong here. We will let operators know that it takes a day or two.

* Thanks for the map changes. I think we are ok here. We can fine tune
  map projections another time.

* Thanks for the flag changes, on Relay Search and Consensus Health.
  Don't waste time implementing this in metrics-lib yet. We need to do
  one more change to the fallback format. Then we will have authorities
  and fallbacks in separate lists in a similar format. We are targeting 0.3.4
  for this change (late 2018). The other flags are fine with the current
  workarounds.

* I will start working on PrivCount in Tor in February. It won't be working
  on relays until 0.3.4 (late 2018) or 0.3.5 (early 2019). We will focus on
  new statistics. We will ask you before removing old statistics. We want
  to move bandwidth and other guard discovery statistics to PrivCount.
  Let's talk about this later in 2018.

> I also have a couple requests in my inbox where folks need more data or
> new features to do their job better. All reasonable requests, but they
> pile up here.
> 
> Oh, and of course we're doing stuff like fixing bugs and keeping
> services running.
> 
> However, I believe we're currently understaffed for doing all these
> things. We currently only have iwakeh and me as paid developers, plus
> we'll be adding irl as paid developer, but not before July 2018.

Wow, yes, this is understaffed for what we expect.
I appreciate all your hard work on all the things we ask you to do.

> The part where this becomes most apparent is that we haven't started
> working on the grant mentioned above, even though we had planned to do
> this in January. It's not too late to fulfill our commitment there, but
> at some point it will be too late if we keep doing things like now. This
> is bad, and it's the main reason why I'm writing this message.
> 
> I'm not sure how exactly to resolve this.
> 
> One idea is to drop most new things now and prioritize the things that
> we promised in the past. This would be the paid deliverables plus a few
> other tasks. Basically, what we wrote in our roadmap a few months ago.
> However, I'm a bit worried that people will suffer from not having the
> tools that they need, or that they'll build workarounds that will be
> difficult to maintain in the long run.

From my perspective:
* the current workarounds or systems are built, and they work for us.
* we will want to compare tjr's test bandwidth authority on a map, but test
  data doesn't belong in Onionoo in my opinion.
* we will want to compare bandwidth authority votes if we change bandwidth
  servers. If we change servers soon, we can ask for a once-off analysis. If we
  change them later, maybe Onionoo will be ready by then.
* any other features or fixes can wait.

> Another idea is to add more people now. Obviously, this needs to be done
> with great care, because new people require attention, too. But I could
> imagine that we'd find tasks for new people that need doing. Maybe those
> people would work on existing tasks that we already committed, and we'd
> use the newly available time to work on new tasks that help other Tor folks.

I would support this as well.
But I don't know enough details to have an opinion on it.

> Maybe there are other ideas? I'd like to collect some feedback from you
> and then discuss this more at tomorrow's team meeting (Thursday, 14:30
> UTC in #tor-meeting).

I'll be asleep at this time.

Are there things that people from other teams can do to help?
Could you borrow someone from another team for some tasks?

We can also improve our communication with metrics.
When someone asks for a task, they could tell you:
* when they need it done by
* how important it is
* what will happen if it's not done
* what else needs to happen (in other teams) for it to work

I will have more time for other Tor tasks from February.
Some of those tasks could be metrics tasks.
I just need information about where the code or specs are.

T


More information about the metrics-team mailing list