[ooni-talk] Congrats on the iPhone and Android app!

[Arturo Filastò]

Hi Andri,

Thanks for your interest in OONI and your kind words!


On 10/02/2017 09:10 +0000, Andri Effendi wrote:
> Why are some websites appearing in the results as being censored, yet
> when I try to go to the URL it works?

Yes, there are some false positives in the results and although we do
take some measures to reduce them, they sometimes occur.

For further reduction of false positives we handle this in the data
processing pipeline.

False positives can occur because:

1) Sometimes when you do a DNS resolution for a domain (converting
something like google.com into 8.8.8.8) from two different locations you
get back different IPs. We compensate for this by also doing reverse
lookups on the IPs and checking if the reverse matches, but sometimes
this is not enough.
In the data pipeline we have more advanced heuristics to take this into
account.

2) Sometimes the content of a site changes very dramatically between two
different locations (for example because when you access a site from
Turkey the content is localized in turkish). The state of the art in
this field is to use the body length (see:
https://www.cs.princeton.edu/~bj6/papers/imc2014-blockpage-detection.pdf),
but we also use other things such as HTTP Headers and the HTML Title
tag, but it is sometimes not enough.

3) Sometimes the site is not particularly reliable and it will fail to
be up when the probe connects to it, but it is up when our control
connects to it. We are working on addressing this issue by measuring the
global availability of all sites we test and using this as a factor to
reduce false positives in this area.
> I know there are risks of running OONI, but what level is the risk?

You can read more information about what are the risks associated to
running ooniprobe at this page:
https://ooni.torproject.org/about/risks/
> Is it just going on sites like thepiratebay.org?
>
> Or OONI going to be probing (Internationally) illegal sites like Drugs
> and Abuse material going to be probed in tests as well?

The list of sites we use for testing is a project we work on together
with the CitizenLab. You can find the full list of sites we test here:
https://github.com/citizenlab/test-lists/blob/master/lists/global.csv

Some of the rational behind how they are chosen is described a bit here:
https://github.com/citizenlab/test-lists#what-is-it

Let me know if you have any further questions,

~ Arturo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/ooni-talk/attachments/20170210/b39d4072/attachment.html>