[tor-scaling] Summary of 5/31 meeting; next steps

[Mike Perry]

Notes are on the pad (feel free to update; beware any rando on the
Internet also can do so): https://nc.riseup.net/s/AEnQ4CRH2kH3fLe

High level summary: Our meeting today more or less followed the agenda:
I recappped the short and long term plans, we discussed experiments and
metrics (particularly focusing on what metrics would be required to
re-tune EWMA -- http://www.cypherpunks.ca/~iang/pubs/ewma-ccs.pdf), and
we got an overview of the capabilities, compute cost, and development
plans for the shadow and netmirage Tor network emulators.


I got some feedback that it's still a little unclear why we spent the
time discussing what we did, and some requests to plan what we want to
get out of face to face meetings at the Mozilla All Hands, Tor All
Hands, and PETS.

So first off: It is extremely important early-on to dial in versions of
the user-facing metrics that Mozilla requested as essential to a Firefox
Tor deployment. The high level metrics of avg+worstcase throughput,
latency, and network capacity are obviously very important for user
experience, and we want to ensure that we provide them in some form.

But, deciding how we record and provide these metrics, and how we
analyze them, is very Tor specific, and can have a surprising amount of
wrinkles. We also want to ensure that future performance research uses
an accurate as possible model of the Tor network, and has results that
can be reproduced and ultimately deployed with as little additional
development and verification costs by Tor Project Inc as possible.

So because of this, the most important goal I see for Mozilla All Hands
is to ensure that everybody is happy with the metrics we're collecting
and plan to add, and understands how we're going to use/analyze them,
and why. This requires consideration of our immediate live tuning tasks,
review of how the Tor network has behaved in reaction to load spikes and
cycles in the past, and making some educated guesses as to what future
research will require to measure success. It also requires review of
these metrics from the metrics team, to ensure that every metric has at
least the baseline recorded Tor network data to derive it, before we
undertake any experiments.

This baseline in particular is incredibly important to establish early
and get right, so we have as much historical data for comparison as
possible.

To this end, I think we should have at least one but possibly three
meetings at Mozilla all hands, covering the following:
 1. Metrics team review of our planned metrics, to learn what we need to
    start recording right away vs what can be derived, and at what dev
    cost (and what we need to put into funding proposals to do).
 2. Researcher review of our planned metrics, to ensure that potential
    future Tor design changes will be properly measured by them.
 3. Mozilla review of our planned metrics, and clarification about what
    Mozilla wants them for.

Longer-term goals for Tor performance and scalability will naturally be
viewed through the lens of these metrics, and we should have a meeting
or two that covers what the future looks like as we use these metrics to
make decisions about the Tor network.

Most importantly to Mozilla, at least one long-term meeting should cover
what how we expect our metrics to change as various combinations of new
users and additional capacity are added to the network. I expect this to
be a Mozilla-focused meeting, but Tor staff will need to be present to
field questions about what we're comfortable with in terms of how users
are added and how capacity is managed.

Another one of these long-term meetings should cover what the current
research horizon is, and how we expect various research ideas to improve
performance and/or scalability, and what that looks like as far as
changes to our metrics. I expect this to be mostly of interest to Rob,
other researchers, and Mozilla. I expect most Tor Project Inc staff to
not be that excited about this, except for curious interest. I expect to
have many followup meetings about this at Tor all hands/PETS.


In the meantime, with input from folks on this list and on the wiki
page, I would like to add the EWMA re-tuning experiment, fill out the
KIST tuning experiment, and flesh out the metrics section to highlight
metrics that need new data collection. (I will start separate threads
for this on-list as I run into questions -- I have several already).

As for how we use our time at Tor All Hands and PETS, let's wait until
Whistler to flesh that out (though we should grab a timeslot or two on
the Tor All Hands Agenda wiki ASAP -- if anyone *can't* make a
particular day there, please speak up).

For the next voice call, we have tentatively selected June 28 at 1600
UTC, but we can vary that if a stakeholder has a specific need to change
it. Let me know via private mail if so.

Also please do propose additions/changes to the above goals/meeting
topics on-list, and feel free to ask any questions or offer suggestions.


-- 
Mike Perry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-scaling/attachments/20190531/72678235/attachment.sig>