[ooni-dev] M-Lab docs
soltesz at opentechinstitute.org
Tue Sep 3 18:34:19 UTC 2013
Also requested was a description of typical package deployment and
experiment management on M-Lab.
Typical package management follows these basic steps:
1) build rpm package from source repos in http://github.com/m-lab-tools/
2) copy and sign rpm package to M-Lab yum repository
Now per-machine, the slice (i.e. experiment VM) is instantiated:
3) upon slice vm creation on an M-lab server, the first-phase of
initialization includes bootstrapping the yum configuration of the vm's
filesystem (i.e. /etc/yum.conf and /etc/yum.slice.d/slice.repo list).
These import the public signing key, and point to CentOS mirrors and the
M-Lab slice package repository.
4) the second phase of slice initialization then tries to install the
i.e. yum install mlab_ooni
5) on success, the service starts. on failure, stop.
6) M-Lab uses external monitoring to identify failed services and inform
a directory service like mlab-ns of available servers.
Since the rpm packages are effectively public, they do not include any
private information. Some experiments have conditional deployment. i.e.
if site==nuq01, then run an additional service. As well, the initialize
scripts for a package could generate per-machine key pairs.
The scenario I remember us talking about in Berlin involved M-Lab
harvesting the public part of the generated slice keys, or the onion
urls from the ooni slice, and publishing them in some way, through
mlab-ns, or other service like the ooni bouncer.
Does that sound workable for oonib?
On 08/30/2013 04:50 PM, Meredith Whittaker wrote:
> Hi all,
> As an action from today's call, here's a collection of most pertinent
> M-lab docs. If you're looking for something that doesn't seem to be
> here, or if you have questions, don't hesitate to ask Stephen and me.
> *Documents: *
> * Platform architecture doc
> The intended audience is hosting partners (who donate
> * Policies and procedures for approval
> <http://measurementlab.net/sites/default/files/SubmissionguidelinesforM-Labexperiments.pdf> of
> new experiments. Note that I wrote this, and that it may not be
> specific enough, but I can answer any questions. Note that OONI has
> been approved as meeting these criteria per the Steering Committee
> * Server map <http://measurementlab.net/mlab_sites>. Get a look at the
> global M-Lab footprint.
> * List of existing M-Lab
> <http://measurementlab.net/measurement-lab-tools> measurement tools.
> Code repos and documentation for individual tools should be linked
> from here. If they're not, let me know and I'll add this.
> * Access existing M-Lab data <http://measurementlab.net/data>,
> tarballs and BigQuery.
> * Blog post describing the process to visualize data using BigQuery
> <http://dmadev.com/2012/11/19/visualizing-m-lab-data-with-bigquery/>, a
> good introduction both to the tool and the data structure. If you'd
> like whitelisted access to M-Lab data on BigQuery, let me know and
> I'll make it happen.
> * Founding document
> which will give you background on M-Lab's founding motivations. Many
> of the facts quoted here have changed (e.g. number of servers), but
> as history this is instructive.
> * Existing visualizations <http://measurementlab.net/visualization>,
> which are fun and may be instructive.
> Meredith Whittaker
> Program Manager, Google Research
> Google NYC
More information about the ooni-dev