<div dir="ltr">Okay. Can do. Tabling this for a month.<div><br></div><div>-V<br><div><div><br></div><div><br></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 8, 2016 at 3:55 PM, Griffin Boyce <span dir="ltr"><<a href="mailto:griffin@cryptolab.net" target="_blank">griffin@cryptolab.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hey Virgil,<br>
<br>
While I know you and I have talked about this in private recently, it seems like a good time to table this discussion for a couple of weeks. Considering everything else that's going on, this might not be the ideal time for everyone to contribute to the discussion.<br>
<br>
<3<br>
Griffin<br>
<br>
<br>
<br>
<br>
Virgil Griffith wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
Here's yet another data point indicating the policy on crawling .onion<br>
needs to be clarified. The new and popular OnionStats tool doesn't<br>
even respect /robots.txt, see:<br>
<a href="https://onionscan.org/reports/may2016.html" rel="noreferrer" target="_blank">https://onionscan.org/reports/may2016.html</a><br>
<br>
So now we have *three* different positions among respected members of<br>
the Tor community.<br>
<br>
(1) isis et al: robots.txt is insufficient<br>
--- "Consent is not the absence of saying 'no' — it is explicitly<br>
saying 'yes'."<br>
<br>
(2) onionlink/ahmia/notevil/grams: we respect robots.txt<br>
--- "Default is yes, but you can always opt-out."<br>
<br>
(3) onionstats/memex: we ignore robots.txt<br>
--- "Don't care even if you opt-out."<br>
<br>
-V<br>
<br>
On Wed, Jun 8, 2016 at 1:34 AM, Virgil Griffith <<a href="mailto:i@virgil.gr" target="_blank">i@virgil.gr</a>> wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello all.<br>
<br>
I wrote on this topic earlier at:<br>
<br>
<br>
</blockquote>
<a href="https://lists.torproject.org/pipermail/tor-project/2016-May/000411.html" rel="noreferrer" target="_blank">https://lists.torproject.org/pipermail/tor-project/2016-May/000411.html</a><br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<br>
This is me again asking for clarification. I choose this issue<br>
because it is the most self-contained of the various ones raised by<br>
isis et al, and it seemed wise to clarify this becoming opening up a<br>
new one. If someone from Tor management writes me that social<br>
reasons prohibit search engines from being addressed at this time, I<br>
will drop it.<br>
<br></div></div>
Given the lack of prior reaction as well as <a href="http://ahmia.fi" rel="noreferrer" target="_blank">ahmia.fi</a> [1] getting<span class=""><br>
funded for GSoC (ahmia has followed /robots.txt from day zero), I<br>
tentatively conclude this crawling .onion is non-controversial,<br>
i.e., "Per Tor community standards, search engines obeying<br>
robots.txt are a-okay. Equivalently, indexing .onion content is<br>
treated equivalently as any other part of the web."<br>
<br>
But, to motivate as well as give any concerned parties an<br>
opportunity to be hard, I have republished the onion2bitcoin as well<br>
as the bitcoin2onion anonymizing only the final 4 characters of the<br>
.onion address instead of final 8.<br>
<br>
-- <a href="http://virgil.gr/wp-content/uploads/2016/06/onion2btc.html" rel="noreferrer" target="_blank">http://virgil.gr/wp-content/uploads/2016/06/onion2btc.html</a><br>
-- <a href="http://virgil.gr/wp-content/uploads/2016/06/btc2onion.html" rel="noreferrer" target="_blank">http://virgil.gr/wp-content/uploads/2016/06/btc2onion.html</a><br>
<br>
-V<br>
<br>
On Tue, May 31, 2016 at 10:05 PM, Virgil Griffith <<a href="mailto:i@virgil.gr" target="_blank">i@virgil.gr</a>><br>
wrote:<br>
This seems like something people would have opinions on. Anyone?<br>
<br>
-V<br>
<br>
On Monday, 30 May 2016, Virgil Griffith <<a href="mailto:i@virgil.gr" target="_blank">i@virgil.gr</a>> wrote:<br>
<br>
Hello all.<br>
<br>
I am preparing a longer response to the issues Isis et al mentioned.<br>
Most are interrelated, but this one is not. And I wanted to get<br>
clarification on it.<br>
<br>
Isis expressed a concern about making a list of bitcoin addresses<br>
from .onion, citing, "Consent is not the absence of saying 'no' —<br>
it is explicitly saying 'yes'."<br>
<br></span>
For what it's worth, <a href="http://ahmia.fi" rel="noreferrer" target="_blank">ahmia.fi</a> [1] actually supports regex searching<span class=""><br>
right out of the box. In fact, a single line of JSON spits out all<br>
known bitcoin addresses ahmia knows about.<br>
<br>
For example, here's an anonymized list going .onion -> BTC which I<br>
mined from Ahmia,<br>
* <a href="http://virgil.gr/wp-content/uploads/2016/05/btc-on-dot-onion.html" rel="noreferrer" target="_blank">http://virgil.gr/wp-content/uploads/2016/05/btc-on-dot-onion.html</a><br>
[6MB]<br>
<br>
And here's the same information going BTC -> .onion<br>
* <a href="http://virgil.gr/wp-content/uploads/2016/05/btc2domains.v2.txt" rel="noreferrer" target="_blank">http://virgil.gr/wp-content/uploads/2016/05/btc2domains.v2.txt</a><br>
[2mb]<br>
<br>
If you want to check the results you can ask Juha for the JSON query<br>
to do this.<br>
<br>
Lets go out on a limb and assume that regexs are okay. Is the issue<br>
then .onion search-engines? I understand Isis's preference for<br>
there to always be affirmative consent but does that mean that until<br>
such a standard exists all search engines from onion.link, <a href="http://ahmia.fi" rel="noreferrer" target="_blank">ahmia.fi</a><br></span>
[1], MEMEX, NotEvil, and Grams are violating official Tor community<div><div class="h5"><br>
policy?<br>
<br>
----<br>
Here's how I currently see this. I put on my amateur legal hat and<br>
say, "Well, the Internet/world-wide-web is considered a public<br>
space. Onion-sites are like the web, but with masked speakers."<br>
<br>
*<br>
<br>
</div></div></blockquote><div><div class="h5">
<a href="https://www.hks.harvard.edu/m-rcbg/research/j.camp_acm.computer_internet.as.public.space.pdf" rel="noreferrer" target="_blank">https://www.hks.harvard.edu/m-rcbg/research/j.camp_acm.computer_internet.as.public.space.pdf</a><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
* <a href="http://aims.muohio.edu/2011/02/01/is-the-internet-a-public-space/" rel="noreferrer" target="_blank">http://aims.muohio.edu/2011/02/01/is-the-internet-a-public-space/</a><br>
<br>
Ergo, I would argue that, by default, content on .onion is public<br>
the same way everything else on the web is. If you don't want to be<br>
"indexed", for physical spaces you go in-doors, or for the web you<br>
put up a login. As an aside, the web-standard is actually *kinder*<br>
than physical public spaces because on the web one can have an<br>
unobstrusive /robots.txt saying, "please don't index me". Which is<br>
a great thing.<br>
<br>
Whereas some would say Tor users are "anonymous", others would<br>
instead say any and everything Tor is "private". I believe this<br>
needs to be clarified. I once proposed to Roger that he delineate<br>
the sub-types of privacy in the same way Stallman delineated his<br>
"Four Freedoms". Roger replied that he preferred using the broad<br>
catch-all term "Privacy". These confusions may be a caveat of using<br>
a broad catch-all term. Interpreting broadly, Isis is correct.<br>
However, this conclusion has a lot of unpleasant ramifications.<br>
<br>
Comments appreciated,<br>
-V<br>
<br>
P.S. Mildly related, I saw this today involving DARPA, and Tor.<br>
<a href="http://thehackernews.com/2016/05/darpa-trace-hacker.html" rel="noreferrer" target="_blank">http://thehackernews.com/2016/05/darpa-trace-hacker.html</a><br>
<br>
"""<br>
The aim of Enhanced Attribution program is to track personas<br>
continuously and create “algorithms for developing predictive<br>
behavioral profiles.”<br>
"""<br>
<br>
I hope you all are aware this flows directly from MEMEX. Right?<br>
This, and MEMEX, seems a much more appropriate target for outrage.<br>
A lot of this work that numerous community members have worked on<br>
gives even me pause.<br>
</blockquote>
<br>
<br>
<br></div></div>
Links:<br>
------<br>
[1] <a href="http://ahmia.fi" rel="noreferrer" target="_blank">http://ahmia.fi</a><br>
<br>
_______________________________________________<br>
tor-project mailing list<br>
<a href="mailto:tor-project@lists.torproject.org" target="_blank">tor-project@lists.torproject.org</a><br>
<a href="https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project" rel="noreferrer" target="_blank">https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project</a><span class="HOEnZb"><font color="#888888"><br>
</font></span></blockquote><span class="HOEnZb"><font color="#888888">
<br>
-- <br>
There are 10 kinds of people in the world: those who understand binary, those who don't, and people who didn't expect a base 3 joke.<br>
_______________________________________________<br>
tor-project mailing list<br>
<a href="mailto:tor-project@lists.torproject.org" target="_blank">tor-project@lists.torproject.org</a><br>
<a href="https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project" rel="noreferrer" target="_blank">https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project</a><br>
</font></span></blockquote></div><br></div>