Hello Tor!
Here are some updates related to what the OONI team has been up to in April 2017.
# OONI Monthly Report: April 2017
The OONI team made steady progress in April 2017.
We released the stable version of ooniprobe's distribution for Raspberry Pis ("Lepidopter"), as well as the beta version of our data processing pipeline. We created proof of concepts for the orchestrator backend and the client component, and we improved the monitoring of our infrastructure. We also made progress towards new ooniprobe mobile and desktop releases which feature improvements based on community feedback and translations in 4 additional languages.
Below we provide some highlights from OONI's activities in April 2017.
## Stable release of ooniprobe's distribution for Raspberry Pis ("Lepidopter")
We released the stable version of ooniprobe's distribution for Raspberry Pis (lepidopter). It includes the latest version of ooniprobe (2.2.0), as well as a series of improvements and bugfixes.
The full changelog for this release can be found here: https://github.com/TheTorProject/lepidopter/blob/master/ChangeLog.md#lepidop...
To install this version of the image follow the instructions detailed here: https://ooni.torproject.org/install/lepidopter/
## Proof of Concept (PoC) for orchestrator backend
We have implemented a proof of concept prototype for the OONI orchestrator backend specified here: https://github.com/TheTorProject/ooni-spec/blob/master/opos/OONI-Probe-Orche...
In this iteration of the probe orchestration mechanism (source code available here: https://github.com/TheTorProject/proteus). The following functionality is supported:
* Ability for probes to register with the orchestration registry (https://github.com/TheTorProject/proteus/tree/master/proteus-registry) * Ability for probes to update the metadata about their probe (https://github.com/TheTorProject/proteus/tree/master/proteus-registry) * Support for sending push notifications via Apple Push Notifcation service and Firebase Cloud Messagging (https://github.com/TheTorProject/proteus/tree/master/proteus-notify) * Support for administrators to schedule measurements via a web interface (https://github.com/TheTorProject/proteus/tree/master/proteus-frontend) * Ability for probes to receive the tasks they have been notified about and mark them as accepted, rejected or done (https://github.com/TheTorProject/proteus/tree/master/proteus-events) * Multiple architecture build system
The next steps include the deployment of the testing infrastructure for the orchestration backend and starting to test it out inside of the mobile apps.
## Proof of Concept (PoC) for orchestrator in probe
A proof of concept prototype was written to support orchestration inside of measurement-kit: https://github.com/measurement-kit/measurement-kit/pull/1210.
The client component supports the basic orchestration workflow that includes:
* Probes registering with the orchestration system; * Probes updating the probe metadata; * Probe listing the tasks they should be running; * Probes accepting, rejecting and concluding a task.
A sample command line interface that demonstrates the full orchestration workflow has also been implemented to do end-to-end testing.
## New ooniprobe releases
We worked towards new releases of ooniprobe in April. The latest version of ooniprobe includes:
* Integration of OONI's HTTP Header Field Manipulation test into ooniprobe mobile apps
* Improvements to ooniprobe based on community feedback
* Translations in 4 additional languages: Arabic, Farsi, Russian and Greek
* Support for Right-To-Left languages in ooniprobe mobile and ooniprobe desktop
ooniprobe now supports a total of 9 languages: English, Arabic, Farsi, Russian, Hindi, Spanish, French, Italian and Greek.
## Beta version of next-generation pipeline
The pipeline is ready to process data on a daily basis, it has already reduced our storage usage by factor of ~3x enabling faster data processing. The pipeline still has some bugs to be investigated and some features to be implemented to replace prev-generation pipeline, but all stability bugs are fixed at the moment.
The beta version of the pipeline can be found at https://github.com/TheTorProject/ooni-pipeline/pull/62
## Improvements to OONI infrastructure
We have done a series of improvements to the OONI infrastructure, including adding support for better monitoring based on prometheus: https://github.com/TheTorProject/ooni-sysadmin/pull/106 (and https://github.com/TheTorProject/ooni-sysadmin/pull/108). Some probes for sending us notifications when some of the core infrastructure components (bouncer, collectors, test-helpers, ooni-explorer and ooni-measurements) go down have been setup.
You can inspect the public stats by accessing the grafana frontend to prometheus at the following URL (username: guest, password: guest): http://prometheus.infra.ooni.io:3000/
## Outreach and community engagement
On 7th April we presented OONI at the International Journalism Festival (IJF) in Perugia, Italy. Information about our talk, titled "Uncovering Internet Censorship", can be found here: http://www.journalismfestival.com/programme/2017/uncovering-internet-censors...
We also hosted our monthly community meeting on https://slack.openobservatory.org/ on 25th April (meeting minutes: http://meetbot.debian.net/ooni/2017/ooni.2017-04-25-15.59.log.html). As part of the meeting, we discussed the translation of ooniprobe and approaches for upcoming data visualizations.
## Userbase
In April 2017 ooniprobe was run 57,471 times from 1,535 different vantage points across 158 countries around the world.
This information can also be found through our stats here: https://measurements.ooni.torproject.org/stats
~ The OONI team