- ooni-dev - lists.torproject.org

OONI weekly dev meeting today Monday 23rd of February 18:00 UTC
by Arturo Filastò 23 Feb '15

23 Feb '15

Hello Oonitarians, We have skipped the last two weekly dev meetings because of various necessities, but I am going to suggest we try and make it today. There are a lot of interesting things to discuss and think about together. These are the topics that I have in mind, feel free to append to this list if you think that there is something else: * Discuss the outcome of the "Future of OONI" thread: https://lists.torproject.org/pipermail/ooni-dev/2015-February/000246.html * Updates on the migration of the pipeline * Discuss how to get 1.2.3 released on debian * Updates on libight and iOS integration * Finishing the bridge reachability study The meeting will happen as usual on the #ooni channel on irc.oftc.net at 18:00 UTC (19:00 CET, 13:00 EST, 10:00 PST). See you soon! ~ Arturo

1 0

Let's come up with the roadmap for the future of OONI
by Arturo Filastò 22 Feb '15

22 Feb '15

Hello Oonitarians and Divisionists, I would love to have your feedback on what you believe to be the most important topics for the future of OONI. I have made a list of what I believe are all possible and interesting tasks to perform, but we can't do them all and for sure we can't do them all at once. For this reason it would be very useful if you could express a vote from 1-5: 1: Nah, this is boring and pointless 2: Not really super important, I would give priority to other stuff 3: Useful, I would do it 4: This would be awesome 5: Epic! Feel free to also expand these topics with questions and feedback (or new ones). At the end of the list I will give you a table to cast your vote. # Get daily OONI measurements from 50 countries This means focusing on getting a large and diverse reliable OONI userbase. It does not simply mean to get at least 1 measurement in 50 countries, but it means either establishing relationships with trusted parties in 50 countries, or expanding our userbase to a point where we have at least 1 measurement per day for every test per country (from the same network). Ways of achieving this are: * Rent a VPS in X number of countries * Running the adopt an ooniprobe program (see below for more details) * Establish an agreement with operators in 50 countries to host the probing infrastructure (as previously stated) # Develop OONI tests for censorship circumvention tools This involves devising a methodology for testing the reliability of censorship circumvention tools in various countries. It means testing censorship circumvention tool both open source and propertairy. A cursory list of the protocols/tools we could be interested includes, but is not limited to: * Tor * VPN * Web proxies (hidemyass, etc.) * SSH tunnel * Freegate * Psiphon * Ultrasurf * alkasir If you think we should be testing some other tools too, please add to this list. Useful resources: http://cyber.law.harvard.edu/publications/2011/2011_Circumvention_Tool_Eval… http://cyber.law.harvard.edu/publications/2010/Circumvention_Tool_Usage # Develop scheme for orchestrating ooni-probes This means coming up with a protocol that allows an OONI test developer to schedule a measurement to be run with a certain input they decide on a set of probes in country X. Obviously security considerations need to be taken into account and access will be in the initial stage only limited to a very restricted set of OONI developers that will be made public. # Implement data analytics and visualization for OONI tests We have a bunch of data and we would like to give meaning to it. This would involve writing tools for querying the data in the database and extract useful analytical information from it. Based on this data we can then start looking at historical OONI data and provide some sample visually supported reports. This is a list of tests we should develop analytics for: * HTTP requests * DNS Consistency * DNS Injection * TCP Connect * HTTP Invalid Request Line * HTTP Header Field Manipulation * Multi protocol port traceroute # Implement pub-sub system for ooni collectors Currently OONI collectors (the things you send your ooniprobe measurement results to) keep in sync thanks to a bunch of shell scripts and cronjobs. To have more real time data it would be useful to have a pub-sub mechanism that allows the pipeline to subscribe to all the collectors and the collectors will then publish the collected reports to it, as soon as they are submitted. This will allow the OONI data to go through the data pipeline much faster (instead of ~2 hours, perhaps just some minutes or even less potentially). # Reach production quality ooni rasperry-pi (beagle-board) images This involves implementing what is specified in the lepidopter specification: https://github.com/anadahz/lepidopter/blob/master/specification.md We should then provide scripts for building the image yourself or how to download and burn it to an SD card on Windows, OSX, Linux (with screenshots). As a bonus we could also offer shipping of pre-made raspberry pi images already burn to an SD card, similar to what is done with rasbpian images. # Promote and further develop OONI on mobile (Android, iOS) This involves improving the GUI of OONI on mobile and getting it into the Google Play store and the Apple App store. We should also work on making it easier for developers of existing iOS and Android apps to add internet measurement capabilities to their app by linking to libight. # Do research based on OONI This would involve doing some research on internet censorship based on OONI probe or on internet measurement in general and publishing them in peer reviewed venues. # Publish monthly reports about the status of internet censorship in a country This would be sort of like a monthly e-zine, where every month we analyse the status of internet censorship in a given country. It should be backed by OONI data, but the core of it should be an editorial type piece explaining the context of the country and how censorship is affecting them. I expect most of this work to be carried out by somebody that has a background in journalism and social sciences. # Run "adopt an ooni-probe" campaign This is basically a program where people make a donation to support getting somebody in an interesting country an ooniprobe or receiving an ooniprobe themselves. # Integration with other censorship measurement projects This means finding better ways of using data from other projects as part of OONI research or getting them to use our data. # Reaching out to communities inside of censored regions This means getting a better understanding of what "benefit" we can give to people that are running the tool. On this topic also see: https://trac.torproject.org/projects/tor/ticket/14760 # Redesign the website for ooni This means removing the horrible bootstrap basic template of the current website and restructuring it so people actually understand what OONI is and how it can be useful to them. # Hold an international internet censorship conference This means running an event where we invite a bunch of people that are active in the field of internet censorship and discuss ideas and possible collaborations. # Implement a GUI for ooniprobe This means picking up where we left off with the OONI web gui and finish implementing it. This could potentially also be useful for the orchestration task as we could implement access control on the HTTP API that the web interface consumes. Goal | Vote ---------------------------------- Get daily OONI measurements | from 50 countries | 0 ---------------------------------- Develop OONI tests for | censorship circumvention | tools | 0 ---------------------------------- Develop scheme for | orchestrating ooni-probes | 0 ---------------------------------- Implement data analytics | and visualization for OONI | tests | 0 ---------------------------------- Implement pub-sub system | for ooni collectors | 0 ---------------------------------- Reach production quality | ooni rasperry-pi | (beagle-board) images | 0 ---------------------------------- Promote and further | develop OONI on mobile | (Android, iOS) | 0 ---------------------------------- Publish monthly reports | about the status of | internet censorship in | a country | 0 ---------------------------------- Run "adopt an ooni-probe" | campaign | 0 ---------------------------------- Integration with other | censorship measurement | projects | 0 ---------------------------------- Reaching out to communities| inside of censored regions | 0 ---------------------------------- Redesign the website for | ooni | 0 ---------------------------------- Hold an international | internet censorship | conference | 0 ---------------------------------- Implement a GUI for | ooniprobes | 0 ---------------------------------- Do research based on | OONI | 0 ---------------------------------- Thanks for taking the time to go through this long email, you can as always find us on IRC #OONI irc.oftc.net. Have fun! ~ Arturo

8 8

Volunteering some developer time
by Kevin Murray 13 Feb '15

13 Feb '15

Hi, I'd love to get involved with the OONI project. I'm doing a PhD in high-performance/scientific computing, working with large experimental datasets. I have experience coding in C, python and R, and a modest understanding of statistics. I have also contributed code (mostly test cases) to little-t tor. Is there a particular part of the OONI infrastructure that would like a volunteer? If possible, it would be great to have a longer-term project, working with a mentor or similar, though I know everyone is very busy. I'm happy to work on any part of OONI. Thanks for your all work, and I hope I can be of assistance, Kevin --- Kevin Murray spam(a)kdmurray.id.au I'm an FSF member -- Help us support software freedom! <https://www.fsf.org/join> GPG pubkey: http://www.kdmurray.id.au/static/A4B4EE6A.asc FPR: 656C 0632 1EAB 2C3F 3837 9767 17C2 8EB1 A4B4 EE6A

2 1

Timeline of Tor blocking
by Sadia Afroz 04 Feb '15

04 Feb '15

Hi all, I am trying to make a list of events when different censorship regimes blocked Tor. The goal is to understand the censorship arms race, especially capabilities of current censors and how long it took Tor to resolve those issues. The attached pdf has all the events I could find. I collected these events from the Tor blog, the bug tracker and the censorship wiki. For some of the events I couldn't find any dates and also I'm not sure if this is a complete list of all the censorship events. Can any of you help me find dates of some of the events or direct me to other sources of information to find more events? Thanks! -- Sadia

3 4

Rescheduling ooni weekly dev meeting
by Arturo Filastò 02 Feb '15

02 Feb '15

Hi Oonitarians! Today at 19:00 CET we were supposed to have the weekly meeting, but I would like to ask if it were possible to reschedule it. Moreover some OONI people are at FOSDEM so I don't expect a lot of people to attend. I am going to suggest we simply postpone the meeting to next week and hold it on next monday at the same time. Are there problems with doing so? Would you prefer to do it sooner? ~ Arturo

1 0

Reminder for weekly OONI dev meeting Monday 2015-01-26 18:00 UTC (19:00 CET)
by Arturo Filastò 26 Jan '15

26 Jan '15

Hello Oonitarians, This is a reminder that today there will be the weekly OONI meeting. It will happen as usual on the #ooni channel on irc.oftc.net at 18:00 UTC (19:00 CET, 13:00 EST, 10:00 PST). Everybody is welcome to join us and bring their questions and feedback. See you later, ~ Arturo

1 0

Event at NEXA center and next OONI dev meeting
by Arturo Filastò 23 Jan '15

23 Jan '15

Hello Oonitarians! We skipped this weeks meeting due to a lot of us being busy with an event at the NEXA center. It was a very interesting opportunity to meet a lot of great network measurement and network neutrality researchers and discuss possible areas of collaboration. I will give a brief summary of some of the most relevant things WRT OONI. # Libight hackfest On the first day we did a hackfest on libight. At the end of it we were able to produce a build of libight for iOS [1] as well as a mockup of the GUI [2]. We also finished the integration of the SOCKS client [3] that will allow us to use tor from libight. Overall we are quite close to having a working prototype for iOS and soon also for Android. # NNTools meeting Enrico Gregori and Valerio Luconi from Italian National Research Council presented their work on Portolan, that is trying to map BGP interconnections with traceroutes. They have developed a desktop and mobile application that is very cool and works well! Later we discussed about possible collaborations, in particular deploying their tests as part of our raspberry pi deployment. For the full schedule of events see: http://nexa.polito.it/nntools2015 Regarding the next developer meeting I am going to suggest we do it as usual on Monday at 19:00 CET (18:00 UTC). That's it, until next week! Have fun! ~ Arturo [1] https://github.com/alemela/libight_iOS [2] https://people.torproject.org/~art/ooni/mockups/ooni-ight-assets.zip [3] https://github.com/TheTorProject/libight/pull/71

1 0

M-Lab Experiment Providers Survey
by Chris Ritzo 19 Jan '15

19 Jan '15

Dear M-Lab Experiment Developer: Measurement Lab is beginning an update to our documentation and web presence. In the process we are updating contacts, and information about each test on the platform to encourage broad, replicable research and analysis of data collected through the M-Lab platform. We're reaching out to you for help. You know your experiments best, and it's a chance for us to make contact to ensure you have the most up to date contacts for M-Lab, should you need anything from us. Information about your experiment’s methodologies will be made available on the M-Lab public website, to provide new researchers and others a quick introduction to using the data generated by your experiments. M-Lab staff will present this information to you for review prior to publication. Please take a moment to complete this researcher survey: http://goo.gl/jsLpBc M-Lab staff are also interested in collaborating with you on new tools we're building to provide API access to M-Lab experiments’ public data and provide visualizations of that data for audiences beyond the research community. Lastly, the current M-Lab operations team at New America's Open Technology Institute consists of Chris Ritzo, Jordan McCarthy, Nathan Kinkade, and Steph Alarcon. We maintain a support tracker which you or others can use by emailing support(a)measurementlab.net <mailto:support@measurementlab.net>. You are likely already subscribed to our lists but they are below if you are not: * ops(a)measurementlab.net <mailto:ops@measurementlab.net> Announcements and Discussion of M-Lab Operations https://groups.google.com/a/measurementlab.net/forum/#!forum/ops <https://groups.google.com/a/measurementlab.net/forum/#%21forum/ops> * discuss(a)measurementlab.net <mailto:discuss@measurementlab.net> Public Discussion Group for Measurement Lab https://groups.google.com/a/measurementlab.net/forum/#!forum/discuss <https://groups.google.com/a/measurementlab.net/forum/#%21forum/discuss> Lastly, if you have specific needs you'd like to discuss with us about your experiments or the M-Lab platform, or about this survey and documentation initiative, please reach out to me directly, and thank you in advance for helping us improve documentation and use of M-Lab experiments' data. Best regards, Chris Ritzo Project Manager for Measurement Lab Open Technology Institute <http://newamerica.org/oti>at New America <http://newamerica.org>

1 0

On the ethics of soliciting measurements and informed consent
by Arturo Filastò 15 Jan '15

15 Jan '15

Hello Oonitarians, During yesterdays we had a very interesting conversation about the ethics of measurements, informed consent and methodologies for achieving it. These are very important discussions to have and we agreed that we should continue it in this thread. For those that were not on IRC at that time I will explain briefly what it was that sparked the debate. If you are interested in reading the full transcript of the meeting private message me and I will send it to you. A problem that we have with OONI and I think is common to most network measurements projects is that of acquiring reliable vantage points in non western countries. By reliable I mean vantage points where the tool in question is not just run once and forgotten, but is periodically run, say once every day. One way of acquiring vantage points is to rent VPS' and setting up the tool on such VPS'. The problem with this approach, though, is that what you are measuring is not the network that a real user in that country would be using. To overcome this issue I have come up with a scheme where by I get in contact with people from countries that interest us and give them some money to buy a raspberry pi and setup ooniprobe on it. As an incentive to keep the probe running and gathering data with a daily resolution I then pay them a small monthly fee to cover bandwidth and power costs. It turns out that not only is this cheaper than renting a VPS in that country, but it also gives us more accurate results, since the measurements are done from the users DSL home connection. The problem with this approach is that we need to make it absolutely clear that there is some risk involved in running the software and the amount of risk varies greatly from country to country. So far I have limited this to a very small set of people (3 in total, 2 paid and 1 not paid) that I have personally vetted and made sure that they have read and understood what is written here: https://github.com/thetorproject/ooni-probe#read-this-before-running-oonipr…. Some people are of the opinion that still this is not enough and that by paying them the risk is increased. It is not yet fully clear to me why that would be the case, nor what can be done to make the situation better. Some have suggested we consult some lawyers that have background in international law to tell us how we can make this situation better. I believe this is probably a good idea. It was also mentioned that Stony Brook university may also have valuable feedback in this area and we should also reach out to them. I invite all the people present during yesterdays meeting to integrate their feedback into this thread and forward this email to people that can further advise. ~ Arturo

9 12

Reminder for ooni weekly meeting today at 19:00 CET
by Arturo Filastò 13 Jan '15

13 Jan '15

This is to remind you that today Tuesday 13th of January at 19:00 CET (18:00 UTC) there will be the weekly ooni dev meeting. The channel is #ooni, the network is irc.oftc.net. See you there. ~ Arturo

1 0