[ooni-dev] Exposing too many or too few options from ooni tests

[poly]

My opinion is that it would be best if we exposed such functionality,
but disabled sending over the results of any such custom tests. I think
we implicitly offer some guarantee that the metrics that we display
publicly are accurate to a reasonable extent.

This way the measurement records are kept accurate and users are free
to run the tests as they wish.

On Tue, 2016-03-15 at 18:18 +0100, Arturo Filastò wrote:
> Hello Oonitarians,
> 
> I was having a discussion with vasilis about some changes that I
> recently did to the lantern
> and psiphon test around exposing some extra configuration options and
> would like to know
> what your opinion is on the matter.
> 
> Basically these tests will try to run the lantern and psiphon tool,
> then attempt to connect to
> a certain website with it and verify if the response it gets from the
> website is the expected one.
> 
> In the past it was possible to configure the URL to be fetched and
> the response body that is
> expected with a sane default.
> 
> I have changed this to no longer be the case and instead use a
> hardcoded value of a website
> that we expect to not change in the future and an expected result for
> it.
> In the specific case it’s http://www.google.com/humans.txt
> 
> The reason for doing this is that I want to avoid the possibility of
> a user misconfiguring the
> URL and expected body to something that is not true (I say foo.com
> results “bar” while it
> actually returns “foo”) and leading to inconsistent results.
> 
> The argument against this is that the website we use for testing may
> change in the future
> and if we don’t notice then we can still have inconsistent results.
> 
> To this I believe that even if that were to become the case it’s more
> likely that us developers
> of the tool will notice and hence ship an update than expect the user
> to tweak their ooniprobe
> to provide valid measurements.
> 
> I believe that exposing some settings that can lead to measurements
> that are not true is
> sub-optimal, but I would like to hear contrasting opinions.
> 
> ~ Arturo
> 
> _______________________________________________
> ooni-dev mailing list
> ooni-dev at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/ooni-dev
-- 
poly
@0xPoly
https://darkdepths.net/pages/contact-keys.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.torproject.org/pipermail/ooni-dev/attachments/20160316/3ba1f5ac/attachment.sig>