[tor-scaling] Summary of 5/31 meeting; next steps

[Mike Perry]

Mike Perry:
> So because of this, the most important goal I see for Mozilla All Hands
> is to ensure that everybody is happy with the metrics we're collecting
> and plan to add, and understands how we're going to use/analyze them,
> and why. This requires consideration of our immediate live tuning tasks,
> review of how the Tor network has behaved in reaction to load spikes and
> cycles in the past, and making some educated guesses as to what future
> research will require to measure success. It also requires review of
> these metrics from the metrics team, to ensure that every metric has at
> least the baseline recorded Tor network data to derive it, before we
> undertake any experiments.
> 
> This baseline in particular is incredibly important to establish early
> and get right, so we have as much historical data for comparison as
> possible.

> To this end, I think we should have at least one but possibly three
> meetings at Mozilla all hands, covering the following:
>  1. Metrics team review of our planned metrics, to learn what we need to
>     start recording right away vs what can be derived, and at what dev
>     cost (and what we need to put into funding proposals to do).
>  2. Researcher review of our planned metrics, to ensure that potential
>     future Tor design changes will be properly measured by them.
>  3. Mozilla review of our planned metrics, and clarification about what
>     Mozilla wants them for.
> 
> Longer-term goals for Tor performance and scalability will naturally be
> viewed through the lens of these metrics, and we should have a meeting
> or two that covers what the future looks like as we use these metrics to
> make decisions about the Tor network.

In a parallel thread about Mozilla all hands planning, Roger pointed out
that we could use a session on enumerating on low hanging fruit in terms
of Tor development items that would improve performance, since so far
we've only been examining what we can tune right now without development
effort. Asn mentioned this in conversation also. This is a great idea.

As we do this brainstorming, we should make note if any of these low
hanging fruit items require different metrics than what we have for the
performance tuning experiments. I will also add these items to the
"Develop" column of the kanban.

One particular area where we're weak is on measuring performance
improvements that are specific to very accurate Tor Browser user+client
models (like predictive circuit building). We should decide if that is
the sort of thing we should even model with our metrics, or if we can
rely on Mozilla's metrics for any of those evaluations.

> Most importantly to Mozilla, at least one long-term meeting should cover
> what how we expect our metrics to change as various combinations of new
> users and additional capacity are added to the network. I expect this to
> be a Mozilla-focused meeting, but Tor staff will need to be present to
> field questions about what we're comfortable with in terms of how users
> are added and how capacity is managed.
> 
> Another one of these long-term meetings should cover what the current
> research horizon is, and how we expect various research ideas to improve
> performance and/or scalability, and what that looks like as far as
> changes to our metrics. I expect this to be mostly of interest to Rob,
> other researchers, and Mozilla. I expect most Tor Project Inc staff to
> not be that excited about this, except for curious interest. I expect to
> have many followup meetings about this at Tor all hands/PETS.

FYI: At Mozilla All Hands, the bottleneck for our meeting scheduling is
any coordination with Mozilla. We have 1-2 metrics+performance meeting
slots scheduled and available with Mozilla folks.

Therefore, I think that we should break the above down into smaller
preliminary meetings with subsets of the Tor folks to brainstorm at
first, and then recap those results in our our Mozilla-side meetings,
dealing with metrics review first, and then covering future-looking R&D
horizon and user+capacity addition.

(We'll be hashing the Mozilla All Hands meeting details out off-list
from now on, but I wanted to also record this here for people who are
not attending Mozilla all hands.)

-- 
Mike Perry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Digital signature
URL: <http://lists.torproject.org/pipermail/tor-scaling/attachments/20190531/234cf330/attachment.sig>