Hello,
The OONI team made steady progress in November 2017.
We reported on recent censorship events in Pakistan, as well as on cases of DNS misconfiguration. We also updated various test lists in collaboration with community members, and we presented OONI at the Free Society Conference and Nordic Summit (FSCONS). We are currently working on improving the design and usability of our mobile apps, and we completed the first draft of the onboarding process for the OONI Probe Android app. We also made an alpha release of node.js bindings for measurement-kit, made progress on implementing dynamic serving of test lists via the orchestration backend, and we started writing an architecture and threat model document for probe orchestration.
# Published research report on recent censorship events in Pakistan
In collaboration with Bytes for All we published a research report examining recent censorship events in Pakistan.
This report is available here: https://ooni.torproject.org/post/how-pakistan-blocked-social-media/
Through the analysis of OONI network measurements collected from Pakistan, we confirmed the blocking of 14 news websites and various social media sites and instant messaging apps during Islamist protests.
Our report received the following press coverage: https://www.thenews.com.pk/print/250865-internet-body-issues-report-on-news-...
# Published research report on cases of DNS misconfiguration* *
As part of our analysis of OONI measurements and investigation into DNS failures, we uncovered various cases of DNS misconfiguration.
We reported these cases in the following post: https://ooni.torproject.org/post/not-quite-network-censorship/
# Progress on mobile app redesign
We are currently working on improving the design and usability of our mobile apps. During November we completed the first draft of the re-designed onboarding process for the OONI Probe Android app, and we worked on the Interaction Design elements.
# Updated test lists
We collaborated with community members on updating various test lists. During November, we updated the following test lists:
1. Sierra Leone: https://github.com/citizenlab/test-lists/pull/262
2. Pakistan: https://github.com/citizenlab/test-lists/pull/259
3. Nigeria: https://github.com/citizenlab/test-lists/pull/255%C2%A0&%C2%A0https://gi...
4. India: https://github.com/citizenlab/test-lists/pull/261
5. Algeria: https://github.com/citizenlab/test-lists/pull/249
# Architecture & Threat model for Probe Orchestration
We started writing an architecture and threat model document for the Probe Orchestration system and we also started collaborating with Cure53 to get it audited by them.
# OONI data publishing
Amazon accepted us into their Open Data program (https://aws.amazon.com/government-education/open-data/) https://aws.amazon.com/government-education/open-data/%29 and as a result all the public OONI data is available for free on s3.
You can access all our public JSON reports inside of this s3 bucket: s3://ooni-data/autoclaved/jsonl/
Example: https://s3.amazonaws.com/ooni-data/autoclaved/jsonl/2017-11-06/20171031T1815...
(If you were previously using s3 hosted data, the bucket name changed from ooni-public to ooni-data)
Moreover we also offer, for batch import, lz4 compressed daily dumps of all measurements.
These are available here: s3://ooni-data/autoclaved/jsonl.tar.lz4/
Example: https://s3.amazonaws.com/ooni-data/autoclaved/jsonl.tar.lz4/2017-12-01/dash....
To get a listing and metadata of all the files in a given bucket, you may look for the index.json.gz file
Example: https://s3.amazonaws.com/ooni-data/autoclaved/jsonl.tar.lz4/2017-12-01/index...
If you are using the AWS Command Line Interface (https://docs.aws.amazon.com/cli/latest/reference/) https://docs.aws.amazon.com/cli/latest/reference/%29 and have a AWS access key configured, be sure to pass in the --no-sign-request option.
Example:
$ aws s3 --no-sign-request ls s3://ooni-data/autoclaved/jsonl/2017-12-04/ We will be releasing a short blog post explaining how to use this data, once Amazon updates their Open Data page to list us as a project.
# Test lists orchestration
Progress was made on implementing dynamic serving of tests lists via the orchestration backend. See: https://github.com/TheTorProject/proteus/pull/29
# Node.js bindings for Measurement-kit
We made an alpha release of node.js bindings for measurement-kit:
https://github.com/measurement-kit/measurement-kit-node
It's not yet ready for usage by the general public, but we have been using this as a foundation for the development of our new OONI Probe desktop apps.
# Community activities
We presented OONI at the Free Society Conference and Nordic Summit (FSCONS) in Norway on 4th November 2017. Information about our presentation is available here: https://frab.fscons.org/en/fscons17/public/events/309.
We also attended the OTF Summit in Valencia, and hosted our monthly community meeting on 28th November 2017. As part of the community meeting, we discussed strategies for monitoring emergent censorship events and rapidly responding to them (in light of recent censorship events in Pakistan, for example).
# Research citing OONI data
During November OONI data was cited in the following research:
* Citizen Lab Submission to the United Nations Special Rapporteur on violence against women, its causes and consequences: https://citizenlab.ca/wp-content/uploads/2017/11/Final-UNSRVAG-CitizenLab.pd...
* Internet Censorship Capabilities in Cyprus: An investigation of online gambling blocklisting: https://www.researchgate.net/publication/319482512_Internet_Censorship_Capab...
# Userbase
In November 2017 OONI Probe was run 184,800 times from 3,544 different vantage points across 202 countries around the world.
This information can also be found through our stats here: https://api.ooni.io/stats
~ The OONI team.