<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi Andri,</p>
<p>Thanks for your interest in OONI and your kind words!<br>
</p>
<br>
<div class="moz-cite-prefix">On 10/02/2017 09:10 +0000, Andri
Effendi wrote:
</div>
<blockquote cite="mid:589D837B.9060900@gmx.de" type="cite">
<pre wrap="">
Why are some websites appearing in the results as being censored, yet
when I try to go to the URL it works?
</pre>
</blockquote>
<br>
Yes, there are some false positives in the results and although we
do take some measures to reduce them, they sometimes occur.<br>
<br>
For further reduction of false positives we handle this in the data
processing pipeline.<br>
<br>
False positives can occur because:<br>
<br>
1) Sometimes when you do a DNS resolution for a domain (converting
something like google.com into 8.8.8.8) from two different locations
you get back different IPs. We compensate for this by also doing
reverse lookups on the IPs and checking if the reverse matches, but
sometimes this is not enough.<br>
In the data pipeline we have more advanced heuristics to take this
into account.<br>
<br>
2) Sometimes the content of a site changes very dramatically between
two different locations (for example because when you access a site
from Turkey the content is localized in turkish). The state of the
art in this field is to use the body length (see:
<a class="moz-txt-link-freetext"
href="https://www.cs.princeton.edu/%7Ebj6/papers/imc2014-blockpage-detection.pdf">https://www.cs.princeton.edu/~bj6/papers/imc2014-blockpage-detection.pdf</a>),
but we also use other things such as HTTP Headers and the HTML Title
tag, but it is sometimes not enough.<br>
<br>
3) Sometimes the site is not particularly reliable and it will fail
to be up when the probe connects to it, but it is up when our
control connects to it. We are working on addressing this issue by
measuring the global availability of all sites we test and using
this as a factor to reduce false positives in this area.<br>
<blockquote cite="mid:589D837B.9060900@gmx.de" type="cite">
<pre wrap="">
I know there are risks of running OONI, but what level is the risk?
</pre>
</blockquote>
<br>
You can read more information about what are the risks associated to
running ooniprobe at this page: <br>
<a class="moz-txt-link-freetext" href="https://ooni.torproject.org/about/risks/">https://ooni.torproject.org/about/risks/</a><br>
<blockquote cite="mid:589D837B.9060900@gmx.de" type="cite">
<pre wrap="">
Is it just going on sites like thepiratebay.org?
Or OONI going to be probing (Internationally) illegal sites like Drugs
and Abuse material going to be probed in tests as well?
</pre>
</blockquote>
<br>
The list of sites we use for testing is a project we work on
together with the CitizenLab. You can find the full list of sites we
test here: <br>
<a class="moz-txt-link-freetext" href="https://github.com/citizenlab/test-lists/blob/master/lists/global.csv">https://github.com/citizenlab/test-lists/blob/master/lists/global.csv</a><br>
<br>
Some of the rational behind how they are chosen is described a bit
here:<br>
<a class="moz-txt-link-freetext" href="https://github.com/citizenlab/test-lists#what-is-it">https://github.com/citizenlab/test-lists#what-is-it</a><br>
<br>
Let me know if you have any further questions,<br>
<br>
~ Arturo<br>
</body>
</html>