January 2015 - ooni-dev - lists.torproject.org

Reminder for weekly OONI dev meeting Monday 2015-01-26 18:00 UTC (19:00 CET)
by Arturo Filastò 26 Jan '15

26 Jan '15

Hello Oonitarians, This is a reminder that today there will be the weekly OONI meeting. It will happen as usual on the #ooni channel on irc.oftc.net at 18:00 UTC (19:00 CET, 13:00 EST, 10:00 PST). Everybody is welcome to join us and bring their questions and feedback. See you later, ~ Arturo

1 0

Event at NEXA center and next OONI dev meeting
by Arturo Filastò 23 Jan '15

23 Jan '15

Hello Oonitarians! We skipped this weeks meeting due to a lot of us being busy with an event at the NEXA center. It was a very interesting opportunity to meet a lot of great network measurement and network neutrality researchers and discuss possible areas of collaboration. I will give a brief summary of some of the most relevant things WRT OONI. # Libight hackfest On the first day we did a hackfest on libight. At the end of it we were able to produce a build of libight for iOS [1] as well as a mockup of the GUI [2]. We also finished the integration of the SOCKS client [3] that will allow us to use tor from libight. Overall we are quite close to having a working prototype for iOS and soon also for Android. # NNTools meeting Enrico Gregori and Valerio Luconi from Italian National Research Council presented their work on Portolan, that is trying to map BGP interconnections with traceroutes. They have developed a desktop and mobile application that is very cool and works well! Later we discussed about possible collaborations, in particular deploying their tests as part of our raspberry pi deployment. For the full schedule of events see: http://nexa.polito.it/nntools2015 Regarding the next developer meeting I am going to suggest we do it as usual on Monday at 19:00 CET (18:00 UTC). That's it, until next week! Have fun! ~ Arturo [1] https://github.com/alemela/libight_iOS [2] https://people.torproject.org/~art/ooni/mockups/ooni-ight-assets.zip [3] https://github.com/TheTorProject/libight/pull/71

1 0

M-Lab Experiment Providers Survey
by Chris Ritzo 19 Jan '15

19 Jan '15

Dear M-Lab Experiment Developer: Measurement Lab is beginning an update to our documentation and web presence. In the process we are updating contacts, and information about each test on the platform to encourage broad, replicable research and analysis of data collected through the M-Lab platform. We're reaching out to you for help. You know your experiments best, and it's a chance for us to make contact to ensure you have the most up to date contacts for M-Lab, should you need anything from us. Information about your experiment’s methodologies will be made available on the M-Lab public website, to provide new researchers and others a quick introduction to using the data generated by your experiments. M-Lab staff will present this information to you for review prior to publication. Please take a moment to complete this researcher survey: http://goo.gl/jsLpBc M-Lab staff are also interested in collaborating with you on new tools we're building to provide API access to M-Lab experiments’ public data and provide visualizations of that data for audiences beyond the research community. Lastly, the current M-Lab operations team at New America's Open Technology Institute consists of Chris Ritzo, Jordan McCarthy, Nathan Kinkade, and Steph Alarcon. We maintain a support tracker which you or others can use by emailing support(a)measurementlab.net <mailto:support@measurementlab.net>. You are likely already subscribed to our lists but they are below if you are not: * ops(a)measurementlab.net <mailto:ops@measurementlab.net> Announcements and Discussion of M-Lab Operations https://groups.google.com/a/measurementlab.net/forum/#!forum/ops <https://groups.google.com/a/measurementlab.net/forum/#%21forum/ops> * discuss(a)measurementlab.net <mailto:discuss@measurementlab.net> Public Discussion Group for Measurement Lab https://groups.google.com/a/measurementlab.net/forum/#!forum/discuss <https://groups.google.com/a/measurementlab.net/forum/#%21forum/discuss> Lastly, if you have specific needs you'd like to discuss with us about your experiments or the M-Lab platform, or about this survey and documentation initiative, please reach out to me directly, and thank you in advance for helping us improve documentation and use of M-Lab experiments' data. Best regards, Chris Ritzo Project Manager for Measurement Lab Open Technology Institute <http://newamerica.org/oti>at New America <http://newamerica.org>

1 0

On the ethics of soliciting measurements and informed consent
by Arturo Filastò 15 Jan '15

15 Jan '15

Hello Oonitarians, During yesterdays we had a very interesting conversation about the ethics of measurements, informed consent and methodologies for achieving it. These are very important discussions to have and we agreed that we should continue it in this thread. For those that were not on IRC at that time I will explain briefly what it was that sparked the debate. If you are interested in reading the full transcript of the meeting private message me and I will send it to you. A problem that we have with OONI and I think is common to most network measurements projects is that of acquiring reliable vantage points in non western countries. By reliable I mean vantage points where the tool in question is not just run once and forgotten, but is periodically run, say once every day. One way of acquiring vantage points is to rent VPS' and setting up the tool on such VPS'. The problem with this approach, though, is that what you are measuring is not the network that a real user in that country would be using. To overcome this issue I have come up with a scheme where by I get in contact with people from countries that interest us and give them some money to buy a raspberry pi and setup ooniprobe on it. As an incentive to keep the probe running and gathering data with a daily resolution I then pay them a small monthly fee to cover bandwidth and power costs. It turns out that not only is this cheaper than renting a VPS in that country, but it also gives us more accurate results, since the measurements are done from the users DSL home connection. The problem with this approach is that we need to make it absolutely clear that there is some risk involved in running the software and the amount of risk varies greatly from country to country. So far I have limited this to a very small set of people (3 in total, 2 paid and 1 not paid) that I have personally vetted and made sure that they have read and understood what is written here: https://github.com/thetorproject/ooni-probe#read-this-before-running-oonipr…. Some people are of the opinion that still this is not enough and that by paying them the risk is increased. It is not yet fully clear to me why that would be the case, nor what can be done to make the situation better. Some have suggested we consult some lawyers that have background in international law to tell us how we can make this situation better. I believe this is probably a good idea. It was also mentioned that Stony Brook university may also have valuable feedback in this area and we should also reach out to them. I invite all the people present during yesterdays meeting to integrate their feedback into this thread and forward this email to people that can further advise. ~ Arturo

9 12

Reminder for ooni weekly meeting today at 19:00 CET
by Arturo Filastò 13 Jan '15

13 Jan '15

This is to remind you that today Tuesday 13th of January at 19:00 CET (18:00 UTC) there will be the weekly ooni dev meeting. The channel is #ooni, the network is irc.oftc.net. See you there. ~ Arturo

1 0

Request to reschedule ooni dev meeting to Tuesday 13th
by Arturo Filastò 12 Jan '15

12 Jan '15

Hi, I have to study for an exam on Tuesday so it would be ideal for me if we could move the next ooni dev meeting to Tuesday. Does that work for those interested in attending? I would suggest we do it at the same time (19:00 CET). ~ Arturo

1 1

Re: [ooni-dev] [tor-dev] FTBFS for ooniprobe on Fedora 21
by Jacek Wielemborek 12 Jan '15

12 Jan '15

W dniu 08.01.2015 o 12:11, Arturo Filastò pisze: > > > On 1/8/15 1:21 AM, Jacek Wielemborek wrote: >> Hello, >> >> Below is the log. Am I missing some package or something? >> > > [ snip ] > >> >> pcap_ex.c:18:23: fatal error: pcap-int.h: No such file or directory >> >> # include <pcap-int.h> >> >> ^ >> >> compilation terminated. >> >> error: command 'gcc' failed with exit status 1 >> > > [ snip ] > >> [1:18:05][~]$ yum provides pcap-int.h >> Loaded plugins: auto-update-debuginfo, langpacks >> (cut) >> No matches found >> > > From the looks of it you are missing the libpcap-dev package. I believe > in fedora it is called libpcap-devel. > > If you still encounter other issues you should reach us on the ooni-dev > mailing list (ooni-dev(a)lists.torproject.org) or try asking around on IRC > #ooni irc.oftc.net. > > ~ Arturo I do have libpcap-devel installed, it's just this pcap-int.h header that is both missing and not to be found in any Fedora package: $ rpm -qa | grep pcap-dev libpcap-devel-1.6.2-1.fc21.x86_64

2 1

Ethics of Censorship Measurement
by Arturo Filastò 09 Jan '15

09 Jan '15

Hello, I was pointed to this mailing list by Ben Zevenbergen. It seems like there are a few familiar faces in here and I believe some of you are already quite familiar with the tool in question. We have recently had some discussions on our OONI mailing list about ethics of internet censorship related measurements and what should be the best procedure for getting informed consent from our users. You can find this thread here: https://lists.torproject.org/pipermail/ooni-dev/2014-December/000205.html A volunteer started writing up some improvements to our current warning message (that is found here: https://github.com/TheTorProject/ooni-probe#read-this-before-running-oonipr…) and you can find the improvements to it here: https://lists.torproject.org/pipermail/ooni-dev/2015-January/000208.html Some people have pointed out that the above message contains some wording that is a bit too vague and that can lead to excessively scaring users (or possibly even putting them in danger because they have acknoledged that what they are doing could be legal). This discussion mainly occurred on IRC so unfortunately it's not captured anywhere, but I would be happy to further elaborate on it if you are interested. What we currently would need most is somebody that takes a look at the tool and thinks about what could be the real risks that a user of it could possibly face (if any) and come up with a wording that makes these risks clear to them. I am happy to further discuss this either via Skype or on our mailing list. ~ Arturo

2 1

Re: [ooni-dev] Ooniprobe in Latvia
by Arturo Filastò 08 Jan '15

08 Jan '15

As agreed with Aleksejs we are going to move this discussion onto the list. On 1/4/15 8:40 PM, Aleksejs Popovs wrote: > Hi Arturo, > > First of all, sorry for contacting you directly. Ooni-talk seems to be > quite dead, and I am not sure that this is appropriate for ooni-dev. > Feel free to redirect me somewhere else. > > Secondly, great job on the 31C3 OONI presentation! > > Now, onwards to what I wanted to tell you about. Here in Latvia, > DPI-based filtering is used to block HTTP(S) connections to online > gambling websites, as mandated by the law on gambling. However, there is > also speculation originating from ISPs on the possibility of this being > implemented for unlicensed online mass media, which to me sounds scary > as hell. There don't appear to be any reports from Latvia in either > OONI's report repos or Open Net Initiative's lists. > Blocking of gambling sites is in fact something very common in greedy western countries. How are they implementing blocking for HTTPS sites? It is quite unusual to see that happening, but having information on that would be interesting. > I wanted to create an OONI report that would demonstrate this censorship > in my ISP's (Lattelecom, one of the biggest ones) network. Lattelecom > uses DPI on port 80 to find requests containing "Host: <blockedhost>" > and serve them a page like this: > https://b.popovs.lv/images/blocked_website.png (they also do something > similar for HTTPS with self-signed certs). I picked a random blocked > URL, unibet.net <http://unibet.net>, put both HTTP and HTTPS versions of > it into a text file, and then put a URL of a page on my personal > website, popovs.lv <http://popovs.lv> (which isn't blocked), to use as a > baseline. > > I ran the test, and it reported some errors and that "censorship is > probably not happening" (which applies to my homepage, I guess). Here's > the ooniprobe log and the > report: https://popovs.lv/crap/ooni/ooni_run.txt https://popovs.lv/crap/ooni/report-http_requests-2015-01-04T165420Z.yamloo > > Looking at the report, I saw that, while requests to my homepage went > through just fine (and, as expected, were not censored), requests to the > censored pages didn't show the censorship message, but instead showed > various errors. I got confused as to why I could receive a parsing > error, but it all cleared up when I tried looking at the plain headers > using netcat: https://popovs.lv/crap/ooni/netcat.txt . That's right, > there were no HTTP headers at all — their censorship setups just spits > HTML out right away. I'm genuinely surprised that browsers actually > render that. The same idiocy seems to be happening with HTTPS. > Oh my, that is some super ghetto censorship equipment at work. We are relying on twisted's HTTP parsing library so it appears that it does not support very well responses that are out of spec. There is in the making a new HTTP test template in this branch: https://github.com/thetorproject/ooni-probe/tree/feature/http-template and it may be a good idea to support in it also logging HTTP responses that are out of spec. In the meantime what you can do to overcome this limit of ooniprobe is that you could run the http_filtering_bypassing experimental test. If they are doing blocking based on HTTP Host header field that will trigger the blocking when running the "test_normal_request", but will also identify some possible ways to bypass the filter by doing some slightly modified requests (that is requests that a normal web server would accept, but may be erroneously matched by the filter). With this test we were able to detect some filtering bypassing techniques in Turkmenistan and Uzbekistan: https://ooni.torproject.org/tab-tab-come-in-bypassing-internet-blocking-to-… Since this test does not use the full HTTP library, but just uses plain TCP to form the HTTP request and simply logs the HTTP response as a string without parsing it. > So, I'm not even sure about what I want from you: I guess I just wanted > you to know about this situation. I don't know how exactly are the OONI > reports analysed — do you consider errors like this one to be cases of > censorship? I guess you wouldn't want to implement some hacks to support > my ISPs stupid quirks, but I just want to know if I can help in any > further way to report on the net censorship here in Latvia. > As I said above I think it's a good idea to support these sorts of weird behaviors ISP filtering equipment has. We may see this behavior in the future and it's useful to be able to link it to the filtering technology used by Latvia. > Huge thanks to you for all of your work on OONI and other net freedom > and privacy-related projects! > > Best regards, > Aleksejs Popovs Thanks for your email ~ Arturo

4 6

Re: [ooni-dev] [otf-projects] Revamping the Legal Lab
by Arturo Filastò 05 Jan '15

05 Jan '15

Hi Esther, I think I had heard of this initiative, but only now I realize how it could be of good use for my project. I work on OONI an internet censorship measurement platform. Recently on our mailing list and on IRC we have had some discussions about the ethics of measurement and how to better informs users of they risks they may face when using our software. You can read more about this discussion here: https://lists.torproject.org/pipermail/ooni-dev/2014-December/000205.html https://lists.torproject.org/pipermail/ooni-dev/2014-December/000206.html https://lists.torproject.org/pipermail/ooni-dev/2014-December/000207.html Recently a volunteer has suggested some ways to improve our warning text and you can see that here: https://lists.torproject.org/pipermail/ooni-dev/2015-January/000208.html In there there are some question marks that would require some legal feedback. Is this something you or somebody from your team would be available to take a look at? We would also like in general to be able a go to person for legal support in the country of one of our users. I think it would be great if we could add to the legal disclaimer that for some set of countries we know who these legal help people are and that users should promptly contact us and we shall direct them to them. Any feedback and further advice would be great. Thanks! ~ Arturo On 1/5/15 3:29 PM, Esther Lim wrote: > Dear OTF Project, > > Happiest of Happy New Year. > > I am Esther, the newest addition to the OTF team and the current lead > for the OTF Legal Lab. I very much look forward to working with all of > you one way or another. > > We are looking to restructure, revamp, and remake the Legal Lab to > better suit your needs. But first, we need to know what might be helpful. > > To remind you, the Legal Lab has mostly served as an aggregating point > and a mediator between various Legal Clinics and our projects. We aim to > continue that aspect, but plan to expand in other ways. > > Please share your thoughts! You can find the Legal Lab Survey Here > <https://docs.google.com/forms/d/1njfBDX23lXTu_alKcixD539YrTEQ79xkYgGVJ-VgBA…> > > Thanks in advance, >

2 2