[ooni-dev] Ooniprobe in Latvia

[Arturo Filastò]

As agreed with Aleksejs we are going to move this discussion onto the list.

On 1/4/15 8:40 PM, Aleksejs Popovs wrote:
> Hi Arturo,
> 
> First of all, sorry for contacting you directly. Ooni-talk seems to be
> quite dead, and I am not sure that this is appropriate for ooni-dev.
> Feel free to redirect me somewhere else.
> 
> Secondly, great job on the 31C3 OONI presentation!
> 
> Now, onwards to what I wanted to tell you about. Here in Latvia,
> DPI-based filtering is used to block HTTP(S) connections to online
> gambling websites, as mandated by the law on gambling. However, there is
> also speculation originating from ISPs on the possibility of this being
> implemented for unlicensed online mass media, which to me sounds scary
> as hell. There don't appear to be any reports from Latvia in either
> OONI's report repos or Open Net Initiative's lists.
> 

Blocking of gambling sites is in fact something very common in greedy
western countries.

How are they implementing blocking for HTTPS sites? It is quite unusual
to see that happening, but having information on that would be interesting.


> I wanted to create an OONI report that would demonstrate this censorship
> in my ISP's (Lattelecom, one of the biggest ones) network. Lattelecom
> uses DPI on port 80 to find requests containing "Host: <blockedhost>"
> and serve them a page like this:
> https://b.popovs.lv/images/blocked_website.png (they also do something
> similar for HTTPS with self-signed certs). I picked a random blocked
> URL, unibet.net <http://unibet.net>, put both HTTP and HTTPS versions of
> it into a text file, and then put a URL of a page on my personal
> website, popovs.lv <http://popovs.lv> (which isn't blocked), to use as a
> baseline.
> 
> I ran the test, and it reported some errors and that "censorship is
> probably not happening" (which applies to my homepage, I guess). Here's
> the ooniprobe log and the
> report: https://popovs.lv/crap/ooni/ooni_run.txt https://popovs.lv/crap/ooni/report-http_requests-2015-01-04T165420Z.yamloo
> 
> Looking at the report, I saw that, while requests to my homepage went
> through just fine (and, as expected, were not censored), requests to the
> censored pages didn't show the censorship message, but instead showed
> various errors. I got confused as to why I could receive a parsing
> error, but it all cleared up when I tried looking at the plain headers
> using netcat: https://popovs.lv/crap/ooni/netcat.txt . That's right,
> there were no HTTP headers at all — their censorship setups just spits
> HTML out right away. I'm genuinely surprised that browsers actually
> render that. The same idiocy seems to be happening with HTTPS.
> 

Oh my, that is some super ghetto censorship equipment at work.

We are relying on twisted's HTTP parsing library so it appears that it
does not support very well responses that are out of spec.

There is in the making a new HTTP test template in this branch:
https://github.com/thetorproject/ooni-probe/tree/feature/http-template

and it may be a good idea to support in it also logging HTTP responses
that are out of spec.

In the meantime what you can do to overcome this limit of ooniprobe is
that you could run the http_filtering_bypassing experimental test.
If they are doing blocking based on HTTP Host header field that will
trigger the blocking when running the "test_normal_request", but will
also identify some possible ways to bypass the filter by doing some
slightly modified requests (that is requests that a normal web server
would accept, but may be erroneously matched by the filter).

With this test we were able to detect some filtering bypassing
techniques in Turkmenistan and Uzbekistan:
https://ooni.torproject.org/tab-tab-come-in-bypassing-internet-blocking-to-categorize-dpi-devices.html

Since this test does not use the full HTTP library, but just uses plain
TCP to form the HTTP request and simply logs the HTTP response as a
string without parsing it.

> So, I'm not even sure about what I want from you: I guess I just wanted
> you to know about this situation. I don't know how exactly are the OONI
> reports analysed — do you consider errors like this one to be cases of
> censorship? I guess you wouldn't want to implement some hacks to support
> my ISPs stupid quirks, but I just want to know if I can help in any
> further way to report on the net censorship here in Latvia.
> 

As I said above I think it's a good idea to support these sorts of weird
behaviors ISP filtering equipment has. We may see this behavior in the
future and it's useful to be able to link it to the filtering technology
used by Latvia.

> Huge thanks to you for all of your work on OONI and other net freedom
> and privacy-related projects!
> 
> Best regards,
> Aleksejs Popovs

Thanks for your email

~ Arturo