[tor-project] Griffin's ICFP February
griffin at cryptolab.net
Thu Mar 3 13:05:11 UTC 2016
-------- Original Message --------
Subject: [icfp-active] Griffin's ICFP February
Date: 2016-03-03 13:50
From: Griffin Boyce <griffin at cryptolab.net>
To: ICFP Active <icfp-active at opentechfund.org>
In February, I fought logistics and small delays, but got some
interesting initial results from Russia. Ultimately, I was not able to
gather enough data to complete a paper before the PETS deadline.
However, I have proposed a talk for the HOPE conference in NYC this
summer . My co-author, Jeff Landale, and I will continue working on
a paper with the goal of submitting for USENIX:FOCI.
Initial results from Russia have been surprising. Of the Alexa top 1M
sites, 153,000 seem to be blocked. I've run the test twice, and will
re-run the test from another location, but if the block percentage
really is ~15.3%, then that is quite extreme! I was guessing that
around 2000 sites would be blocked.
Towards the end of the month I prepared for and attended the Internet
Freedom Festival in Valencia, Spain. As part of the paper-writing
process, I wrote up a first draft of the project methodology, which
appears below. Please let me know if you see any gaps or have
additional tests to suggest.
# Website Diagnostics
The sites to be tested are divided into segments of 10,000 websites,
plus custom tests for sites focused on circumvention. This way, if
there’s an error during the test, it’s easier to perform a re-test
without having to test over a million sites (again).
The first test is a simple check to see if the site is available (code
200) or if it’s down. This test is performed from both the target
region and a presumed-uncensored area (typically Germany or the US) to
ensure that the site is not simply down for maintenance. (In the
process, I determine whether it’s a DNS block or an IP block). Once I
have a list of not-(code 200) sites, I dig in deeper. I take a
screenshot of all sites using EyeWitness. Then I see if the site is
being blocked or is giving a proper Block Page. If the block pages
follow a common format *and* give the block reason, I scrape the reasons
and map them to the domains. There can be interesting differences found
here. For example, in Indonesia SMBC Comics is blocked for ‘pornography
and promoting bigotry and sectarian violence,’ when the site is not
pornographic and contains minimal fantasy violence.
If the site is inaccessible for whatever reason, I check for TCP reset
packets, check if there is an SSL/TLS error, check for forced downloads,
and for some sites check for MitM. MitM test is only performed for
sites where a user would typically be trying to download something, such
as Lantern or Tor Browser. In cases of possible MitM, I collect PCAP
data (packet captures) and attempt to download the relevant parts of the
website to compare with a non-suspicious version of the website. While
some regional differences may exist -- news in different languages, for
example -- differences in executables or malware injections are
indicators of active man-in-the-middle attacks.
Once all website tests are performed, I categorize the sites by
cross-referencing Alexa data. Categorizing website content is one of
the harder problems at scale, so relying on Alexa for categorization
appears to be the best solution. For example, it would be impossible
for me to categorize 153,000 blocked websites from Russia given the
sheer number of websites involved.
# Tor & Lantern tests
Once all websites are categorized, I check whether or not Tor Browser
and Lantern are usable within the country. Lantern has a built-in test
for this purpose. For Tor Browser, I use a modified version of Philipp
Winter’s ExitMap, and feed it an up-to-date list of Tor nodes. It then
cycles through thousands of individual tor circuits to check whether
every entry node is available from the test location. From there, I
determine whether all of Tor is blocked, or only some nodes. If only
some guard nodes are blocked, this may mean (depending on the age of the
unblocked nodes relative to the blocked nodes) that the unblocked guard
nodes were set up to track users or indicate that the system for
blocking Tor nodes hasn’t updated since those nodes were created. I
then test whether different kinds of bridges are blocked within the
country by making circuits using bridges (including
obfs2/obfs3/obfs4/fte/scramblesuit and standard bridges). For some
locations, we will also test flashproxy- and snowflake-backed Tor
connectivity. All Tor-related tests are run twice, with pre-selected
nodes where possible, to minimize errors and ensure test accuracy.
# Tests Performed
OONI: website accessibility, HTTP error codes, TCP reset packets
ExitMap (hacked/customized): checks Tor network node connectivity
Tor daemon: test bridge connectivity
Custom nmap script: to get data on SSL/TLS certificates
Custom scripts: to compare SSL/TLS certificates between test locations,
compare websites for MitM determination, collecting blocked website
HTML, collecting blocked website reasons (as needed), and formatting
# Websites Tested
The top one million sites, as determined by Alexa’s traffic analyses.
This includes a very diverse group of religious and LGBT websites. In
addition, I am testing some circumvention websites. As mentioned above,
these are grouped into segments so that re-tests can be performed
without having to re-test everything.
TN =( -- coordinating local tests instead
KZ =( -- ISP set up Crunchbang OS instead of Debian
 Due to this, I will not be attending PETS.
“We have to create; it is the only thing louder than destruction.”
~ Andrea Gibson
More information about the tor-project