commit 1858d4df39c6fd02b12839119d185c58e44c9cc9 Author: Karsten Loesing <karsten.loesing@gmx.net> Date: Tue May 7 12:22:57 2013 +0200 Rewrite tools page to make it actually useful. --- web/WEB-INF/tools.jsp | 163 ++++++++++++++-------------------- web/images/tor-metrics-overview.png | Bin 0 -> 246279 bytes 2 files changed, 67 insertions(+), 96 deletions(-) diff --git a/web/WEB-INF/tools.jsp b/web/WEB-INF/tools.jsp index d19b8af..a6ee0ff 100644 --- a/web/WEB-INF/tools.jsp +++ b/web/WEB-INF/tools.jsp @@ -12,117 +12,88 @@ <div class="main-column"> <h2>Tor Metrics Portal: Tools</h2> <br> - <p>This page contains a collection of tools that can be used to - gather statistics as provided on the <a href="data.html">Data</a> - page and to process the resulting files to generate - <a href="graphs.html">graphs</a>.</p> - <ul> - <li><a href="#metrics-db">Metrics data processor</a></li> - <li><a href="#metrics-web">Metrics website</a></li> - <li><a href="#dirarch">Directory-archive script</a></li> - <li><a href="#torperf">Torperf</a></li> - <li><a href="#exonerator">ExoneraTor</a></li> - <li><a href="#visitor">VisiTor</a></li> - </ul> - <br> - <a name="metrics-db"></a> - <h3><a href="#metrics-db" class="anchor">Metrics data - processor</a></h3> - <br> - <p>The metrics data processor is a Java application that parses - Tor's directory data and the data from various other Tor services, - possibly removes sensitive parts from them, and then outputs the - <a href="data.html">data</a> in <a href="formats.html">formats</a> - that are feasible for later analysis.</p> - <ul> - <li>Browse the - <a href="https://gitweb.torproject.org/metrics-db.git/tree">Git - repository</a></li> - <li><tt>git clone git://git.torproject.org/metrics-db</tt></li> - </ul> - <br> - <a name="metrics-web"></a> - <h3><a href="#metrics-web" class="anchor">Metrics website</a></h3> - <br> - <p>The metrics website software consists of a Java database - importer and Tomcat application that makes Tor's directory data - easily accessible. This website is run by the metrics website - software.</p> - <ul> - <li>Download - <a href="dist/metrics-web-0.0.1.tar">metrics-web 0.0.1</a> - (<a href="dist/metrics-web-0.0.1.tar.asc">sig</a>)</li> - <li>Browse the - <a href="https://gitweb.torproject.org/metrics-web.git/tree">Git - repository</a></li> - <li><tt>git clone git://git.torproject.org/metrics-web</tt></li> - </ul> + <p>The metrics website and related websites depend on a collection + of services that measure, archive, process, and finally present Tor + network data. + This page gives an overview of these services.</p> + <img src="/images/tor-metrics-overview.png"/> <br> - <a name="dirarch"></a> - <h3><a href="#dirarch" class="anchor">Directory-archive - script</a></h3> + <a name="measure"></a> + <h3><a href="#measure" class="anchor">Measuring Tor network + data</a></h3> <br> - <p>The directory-archive script consists of a bunch of shell - scripts that periodically download relay descriptors, sort them - into a directory structure, and compile monthly tarballs. The - tarballs are quite similar to the ones provided on the - <a href="data.html#relaydesc">Data</a> page, with a few - exceptions: the provided tarballs are the result of combining two - directory-archive script outputs, splitting v3 votes and v3 - consensuses into separate tarballs and replacing all colons in - filenames with dashes. The <a href="#metrics-db">metrics database - software</a> contains similar functionality to the - directory-archive script.</p> + <p>Tor network data is measured at various places: <ul> - <li>Browse the <a href="https://gitweb.torproject.org/tor.git/tree/HEAD:/contrib/directory-archive">contrib/directory-archive/</a> - subdirectory in the Tor sources</li> - <li><tt>git clone git://git.torproject.org/tor</tt></li> + <li><a href="https://gitweb.torproject.org/tor.git">tor</a> + relays and bridges gather aggregate usage statistics and publish + descriptors containing data about Tor network structure and + usage.</li> + <li><a href="https://gitweb.torproject.org/torperf.git">Torperf</a> + is a set of utilities for testing Tor performance from a client + perspective.</li> + <li><a href="https://gitweb.torproject.org/tordnsel.git">TorDNSEL</a> + is a Tor DNS-based exit list that runs periodic checks whether + relays use different IP addresses for exiting to the Internet + than they use to register in the Tor network.</li> + <li><a href="https://gitweb.torproject.org/bridgedb.git">BridgeDB</a> + assigns bridges to distributors and gives them out via HTTPS or + email.</li> </ul> <br> - <a name="torperf"></a> - <h3><a href="#torperf" class="anchor">Torperf</a></h3> + <a name="archive"></a> + <h3><a href="#archive" class="anchor">Archiving Tor network + data</a></h3> <br> - <p>Torperf is a little tool that measures Tor's performance as - users experience it. Torperf uses a trivial SOCKS client to - download files of various sizes over the Tor network and notes how - long substeps take.</p> + <p>All Tor network data is downloaded, possibly sanitized, and + then archived by a single tool:</p> <ul> - <li>Download - <a href="dist/torperf-0.0.1.tar">Torperf 0.0.1</a> - (<a href="dist/torperf-0.0.1.tar.asc">sig</a>)</li> - <li>Browse the <a href="https://gitweb.torproject.org/torperf.git">Git repository</a></li> - <li><tt>git clone git://git.torproject.org/torperf</tt></li> + <li><a href="https://gitweb.torproject.org/metrics-db.git">metrics-db</a> + contains five components for archiving relay descriptors, bridge + descriptors, Torperf results, TorDNSEL exit lists, and BridgeDB + pool assignments.</li> </ul> <br> - <a name="exonerator"></a> - <h3><a href="#exonerator" class="anchor">ExoneraTor</a></h3> + <a name="process"></a> + <h3><a href="#process" class="anchor">Processing Tor network + data</a></h3> <br> - <p>ExoneraTor parses the relay descriptor archives to answer the - question whether some IP address was a Tor relay. This script is - available as a Python and a Java version with equivalent - functionality. There is also a web version of - <a href="/exonerator.html">ExoneraTor</a> available.</p> + <p>In some cases, processing and presenting Tor network data is + separated for maximum flexibility. + In particular, there is currently one tool that processes but does + not present Tor network data:</p> <ul> - <li>Download - <a href="dist/exonerator-0.0.2.tar">ExoneraTor 0.0.2</a> - (<a href="dist/exonerator-0.0.2.tar.asc">sig</a>)</li> - <li>Browse the <a href="https://gitweb.torproject.org/metrics-utils.git/tree/HEAD:/exonerator">Git repository</a> - <li><tt>git clone git://git.torproject.org/metrics-utils</tt></li> + <li><a href="https://gitweb.torproject.org/onionoo.git">Onionoo</a> + provides Tor network status information in JSON format via a + RESTful web service.</li> </ul> <br> - <a name="visitor"></a> - <h3><a href="#visitor" class="anchor">VisiTor</a></h3> + <a name="present"></a> + <h3><a href="#present" class="anchor">Presenting Tor network + data</a></h3> <br> - <p>VisiTor is a script that parses a web server log and the exit - list archives to tell how many of the requests come from Tor - users. VisiTor expects exit lists in the format described in - <a href="https://www.torproject.org/tordnsel/exitlist-spec.txt">exitlist-spec.txt</a>.</p> + <p>There are currently three main websites and a few additional + tools presenting Tor network data: <ul> - <li>Download - <a href="dist/visitor-0.0.4.tar">VisiTor 0.0.4</a> - (<a href="dist/visitor-0.0.4.tar.asc">sig</a>)</li> - <li>Browse the <a href="https://gitweb.torproject.org/metrics-utils.git/tree/HEAD:/visitor">Git repository</a></li> - <li><tt>git clone git://git.torproject.org/metrics-utils</tt></li> + <li><a href="https://gitweb.torproject.org/metrics-web.git">metrics-web</a> + is the software behind this website, including a database to + aggregate most of the presented statistics.</li> + <li><a href="https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-6498">task-6498</a> + aggregates data to visualize fast exits in the Tor network.</li> + <li><a href="https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-2718">task-2718</a> + takes the number of daily Tor users as input and attempts to + find anomalies that might be indicative of censorship.</li> + <li><a href="https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-8462">task-8462</a> + estimates daily users from reported directory request + statistics.</li> + <li><a href="https://gitweb.torproject.org/doctor.git">DocTor</a> + is a service that periodically checks the Tor network for + consensus conflicts and other hiccups.</li> + <li><a href="https://gitweb.torproject.org/atlas.git">Atlas</a> + is a web application to discover relays and bridges that uses + Onionoo as its data back-end.</li> + <li><a href="https://gitweb.torproject.org/compass.git">Compass</a> + is a web application that uses Onionoo's data to display + information about fast exits in the Tor network.</li> </ul> </div> </div> diff --git a/web/images/tor-metrics-overview.png b/web/images/tor-metrics-overview.png new file mode 100644 index 0000000..8ffaca9 Binary files /dev/null and b/web/images/tor-metrics-overview.png differ