[ooni-dev] On the ethics of soliciting measurements and informed consent

[Collin Anderson]

Hi Arturo and Oonitarians,

Thank you for starting this thread, I think this might be a more constructive of a medium than the frenetic exchange that occurred over IRC. My response could probably drag on, but the more that I have written, the more that I recall that I have narrow concerns and that my objection comes from the plan dovetailing into another thread we have had about the current state of data sanitization and risk disclosure for OONI on M-Lab.

There were a number of directions in that IRC conversation, so for time I do not want to reenact it or attempt a line for line critique of those logs. Moreover, there were some parallels that were made to current and historical programs that I do not think are constructive, and previous efforts at network measurement collection often differ in risks than those under OONI.

My argument was not that by paying individuals the risk faced by them increases. Instead, between the deployment strategy and the collection of data from new sources, I want to point out that:

The risk disclosures and documentation of data policies for OONI are currently inadequate for actual informed consent, in both the description of risks and lack of localization. In fact, I could not find a privacy or data retention policy anywhere on ooni.nu that would inform me what others do with my data (including banal matters like what logging is done by bouncers and helpers).
Paid participation in OONI data collection has the possibility of distorting personal assessments of risk in a manner that may encourage improper choices. Renumeration is not in itself bad, but instead represents an area that OONI has not previously engaged in, and it might not be appropriate to launch into this immediately.

I suspect there is actually enough standing research on the second point, since similar issues are present in pharmacological studies, etc. There is the concurrent work on ethnics in censorship measurement that has developed over the past year or so, and I was surprised that OONI does not seem connected to it, hence the Stony Brook comment. However, my point here was that this is an issue that should be acknowledged, that OONI cannot fully account for these influences, that others in our community could probably inform us how to do this correctly, and that, until then, OONI should not engage in such a practice. We can continue this argument if others are interested in it, but I would be surprised if you could not avoid this issue for now by finding volunteers that do not require money to participate.

The first matter is more straightforward and solvable within a shorter window of time. Thus far, installation of OONI has required a meaningful amount of work that has created high barriers to entry, which has been a proxy for informed consent. The more that OONI offers a pre-packaged product (so to speak), the more that it needs to reassess the state of these aspects of the project; so it might not even be about those three people (but you do also want a process for selecting candidates that is vetted by others by the way).

The documentation on risks is currently inadequate and in some cases nonexistent.  I noted three mentions of potential risk within the course of downloading and running OONI to conduct the HTTP Field Manipulation test.

Upon runtime:

WARNING: running ooniprobe involves some risk that varies greatly from country to country. You should be aware of this when running the tool. Read more about this in the manpage or README.

From the README:

Running ooniprobe is a potentially risky activity. This greatly depends on the jurisdiction in which you are in and which test you are running. It is technically possible for a person observing your internet connection to be aware of the fact that you are running ooniprobe. This means that if running network measurement tests is something considered to be illegal in your country then you could be spotted.

Futhermore, ooniprobe takes no precautions to protect the install target machine from forensics analysis. If the fact that you have installed or used ooni probe is a liability for you, please be aware of this risk.

From ts-006-header-field-manipulation.md:

If the user is behind a transparent HTTP proxy that sets the X-Forwarded-For header their IP address will end up being part of the final report.

In summary, I am told that a network intermediary could determine that I am running a test and that a device search could reveal that I have installed the tool.

Between these three notices, I am not told how OONI uses my data or who has access to it, and what assurances are offered about the confidentiality of my reports. After all, OONI does attempt to conceal my identity through reporting AS by default and submitting over Tor, but is there any further data sanitization – will the maintainers seek to clean obvious PII from reports? Moreover, the header-field-manipulation.md notice is only on the bottom of the spec (not displayed at runtime) and is inadequate in addressing the potential risks of de-anonymization – we all know that this is a more expansive problem (e.g. the tracking codes inserted into headers).

Physical devices likely necessitate even further documentation, off the cuff, 1.) will OONI push down new tests to the probes that could change the risks to the users, and if so, what process will be provided for opting out of those tests? 2.) What is the default configuration for reporting and what tests are run against what targets?

—

I want to push aggressively back on the notion that lawyers will help in anything other than advising OONI on its own liabilities. Within the countries where collaboration with international (or “illegal”) non-governmental organizations is deemed a crime, the retention of lawyers to support participants is likely to be of little help and one should not assume that as a fallback strategy.

My counterproposal was simply that OONI should instead focus within this pilot to countries that qualify under any arbitrary metric as less prone to risk, this might use Freedom House or Transparency International style rankings. Under that condition, I would be less concerned about payments. The pilot should attempt to operate as a larger initiative would, and have a survey that would measure the understanding of the participant on their risks, as well as factors like usability and needs. During that time, OONI should reach out to colleagues who are also deploying rPi-based tests in order to understand how they have approached informed consent, to shore up its literature on risk as I have noted above, and translate disclosure material if necessary.

I think this has been a decent start and given people enough to push back on, so I’ll leave it there. Thanks again for starting this Arturo and I hope the 31c3 presentation went well.

Cordially,
Collin

> On Dec 23, 2014, at 2:15 AM, Arturo Filastò <art at torproject.org> wrote:
> 
> Hello Oonitarians,
> 
> During yesterdays we had a very interesting conversation about the
> ethics of measurements, informed consent and methodologies for achieving it.
> These are very important discussions to have and we agreed that we
> should continue it in this thread.
> 
> For those that were not on IRC at that time I will explain briefly what
> it was that sparked the debate. If you are interested in reading the
> full transcript of the meeting private message me and I will send it to you.
> 
> A problem that we have with OONI and I think is common to most network
> measurements projects is that of acquiring reliable vantage points in
> non western countries.
> By reliable I mean vantage points where the tool in question is not just
> run once and forgotten, but is periodically run, say once every day.
> One way of acquiring vantage points is to rent VPS' and setting up the
> tool on such VPS'. The problem with this approach, though, is that what
> you are measuring is not the network that a real user in that country
> would be using.
> 
> To overcome this issue I have come up with a scheme where by I get in
> contact with people from countries that interest us and give them some
> money to buy a raspberry pi and setup ooniprobe on it.
> As an incentive to keep the probe running and gathering data with a
> daily resolution I then pay them a small monthly fee to cover bandwidth
> and power costs.
> It turns out that not only is this cheaper than renting a VPS in that
> country, but it also gives us more accurate results, since the
> measurements are done from the users DSL home connection.
> 
> The problem with this approach is that we need to make it absolutely
> clear that there is some risk involved in running the software and the
> amount of risk varies greatly from country to country.
> So far I have limited this to a very small set of people (3 in total, 2
> paid and 1 not paid) that I have personally vetted and made sure that
> they have read and understood what is written here:
> https://github.com/thetorproject/ooni-probe#read-this-before-running-ooniprobe.
> 
> Some people are of the opinion that still this is not enough and that by
> paying them the risk is increased.
> It is not yet fully clear to me why that would be the case, nor what can
> be done to make the situation better.
> 
> Some have suggested we consult some lawyers that have background in
> international law to tell us how we can make this situation better.
> I believe this is probably a good idea.
> 
> It was also mentioned that Stony Brook university may also have valuable
> feedback in this area and we should also reach out to them.
> 
> I invite all the people present during yesterdays meeting to integrate
> their feedback into this thread and forward this email to people that
> can further advise.
> 
> ~ Arturo

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/ooni-dev/attachments/20141227/740a8b7a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.torproject.org/pipermail/ooni-dev/attachments/20141227/740a8b7a/attachment-0001.sig>