[tor-project] OONI Monthly Report: July 2017

Wed Aug 16 08:27:49 UTC 2017

Hi Tor!

This is what OONI has been up to in July 2017.

# OONI Monthly Report: July 2017

The OONI team had an exciting month, having had the opportunity to host the first OONI Partner Gathering and to attend the Citizen Lab Summer Institute (CLSI).

We made a new release of our mobile apps, including push notifications to remind users to repeat tests regularly, and the integration of the DASH streaming test which is designed to measure the quality of tested networks by emulating a video streaming. We made significant progress on the re-engineering of our data processing pipeline and on probe orchestration. We also updated various test lists in collaboration with our partners, and we hosted our monthly community meeting.

## Organized and hosted the first OONI Partner Gathering

Thanks to support from the OTF Community Lab, the Ford Foundation, and the Media Democracy Fund (MDF), we were able to host our first OONI Partner Gathering!

The event was hosted at the University of Toronto on 10th & 11th July 2017.

The aim of this two-day event was to bring our international partners together to share skills, knowledge, and research findings. We reflected on our collaboration over the last year, and we brainstormed on future goals and priorities to improve our collaboration on the study of internet censorship.

We published a report which provides an overview of the event, its sessions, partner challenges and needs, future goals and priorities, and event outcomes. The report can be found here: https://ooni.torproject.org/post/ooni-partner-gathering-2017/ 

The event's agenda and session notes can be found here: https://github.com/OpenObservatory/gatherings/tree/master/partner-gatherings/2017-07-toronto

As part of the event, we interviewed some of our partners. Their interviews will be published on our website within the next month(s).

## New release: Push notifications and integration of the DASH streaming test into the ooniprobe mobile apps

During July we made a new release of our mobile apps, including the following two exciting new features:

* Push notifications

* DASH streaming test

Push notifications will allow us to send messages and alerts to OONI probes all around the world. This is a particularly important feature as it will allow us to communicate directly with probe users, provide instructions, and to encourage users to run specific tests when emergent censorship events are reported by local communities.

We implemented a first type of notification that sends a text message and an array of URLs, when this notification is opened in the app, a custom web browser opens the first URL and if there is some connectivity error, the user can cycle to the next (to avoid possible censorship).

This first push notification support for apps are only informative: it allows us to send messages to ooniprobe users and engage them to run the app more, in the next release there will be a support to run tests remotely.

OONI's new DASH streaming test is designed to measure video streaming performance. DASH, in particular, is designed to measure the quality of tested networks by emulating a video streaming. This test can be useful for examining cases of throttling and potential net neutrality violations.

We published a detailed description of the DASH test on our website, which can be found here: https://ooni.torproject.org/nettest/dash/

## Progress on the re-engineering of our data processing pipeline

This month we made a lot of progress on the data processing pipeline. 

* We migrated blockpages fingerprints to the new pipeline database format to make migration of OONI Explorer to the news DB format possible.
* We tackled some differences in old and new pipeline results while running two pipelines side-by-side.
* We started hardening distributed pipeline infrastructure with firewalls and better monitoring to prevent incidents.
* We started the process of freeing unused computing resources (virtual machines) allocated to OONI infrastructure in OTF cloud.

## Progress on probe orchestration

In the area of probe orchestration we made a series of very important improvements, namely:

* We deployed to production probe orchestration with message alert push notifications
* Released a new version of the ooniprobe mobile apps that supports receiving push notifications
* Made various improvements and bugfixes to the orchestration system

Learn more by checking out the Changelog: https://github.com/TheTorProject/proteus/blob/master/ChangeLog.md

## Updated test lists

During July we created a new test list for Mali, in collaboration with our local partners. This test list can be found here: https://github.com/citizenlab/test-lists/pull/192

We also updated the Egyptian test list to include URLs that were recently reported to be blocked by locals. The updates can be found here: https://github.com/citizenlab/test-lists/pull/194

Furthermore, we added BBC domains to multiple country-specific test lists: https://github.com/citizenlab/test-lists/pull/193

## Attended the Citizen Lab Summer Institute (CLSI)

The OONI team had the opportunity to attend the Citizen Lab Summer Institute (CLSI) in Toronto from 12th-14th July 2017 (https://citizenlab.ca/summerinstitute/2017.html).

As part of the event, we facilitated the following sessions:

* Presentation of OONI as part of the "Measurement Methods and Tool Showcase" session (Day 2)

* Introduction to Network Measurement Research (Day 2)

* OONI Partnership Program (Day 2)

* Internet Blackout Detection (Day 3)

* Contextualizing Censorship Measurement Research (Day 3)

As an outcome of our participation at the CLSI, we expect to form new partnerships, to better inform the development of our methodologies (particularly those related to measuring internet blackouts), and to create a community-driven database that contextualizes network measurement data.

### Community meeting

We hosted our monthly community meeting on https://slack.openobservatory.org/ on 25th July 2017. As part of the meeting, we discussed the next steps for the development of a methodology for the automatic detection and examination of internet blackouts in regions of India.

## Userbase

In July 2017 ooniprobe was run 65,331 times from 1,606 different vantage points across 158 countries around the world.

This information can also be found through our stats here: https://measurements.ooni.torproject.org/stats

~ The OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20170816/a27e40a2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 208 bytes
Desc: Message signed with OpenPGP using AMPGpg
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20170816/a27e40a2/attachment.sig>