[tor-commits] [metrics-web/master] Rewrite tools page to make it actually useful.

karsten at torproject.org karsten at torproject.org
Tue May 7 10:23:42 UTC 2013


commit 1858d4df39c6fd02b12839119d185c58e44c9cc9
Author: Karsten Loesing <karsten.loesing at gmx.net>
Date:   Tue May 7 12:22:57 2013 +0200

    Rewrite tools page to make it actually useful.
---
 web/WEB-INF/tools.jsp               |  163 ++++++++++++++--------------------
 web/images/tor-metrics-overview.png |  Bin 0 -> 246279 bytes
 2 files changed, 67 insertions(+), 96 deletions(-)

diff --git a/web/WEB-INF/tools.jsp b/web/WEB-INF/tools.jsp
index d19b8af..a6ee0ff 100644
--- a/web/WEB-INF/tools.jsp
+++ b/web/WEB-INF/tools.jsp
@@ -12,117 +12,88 @@
     <div class="main-column">
         <h2>Tor Metrics Portal: Tools</h2>
         <br>
-        <p>This page contains a collection of tools that can be used to
-        gather statistics as provided on the <a href="data.html">Data</a>
-        page and to process the resulting files to generate
-        <a href="graphs.html">graphs</a>.</p>
-        <ul>
-          <li><a href="#metrics-db">Metrics data processor</a></li>
-          <li><a href="#metrics-web">Metrics website</a></li>
-          <li><a href="#dirarch">Directory-archive script</a></li>
-          <li><a href="#torperf">Torperf</a></li>
-          <li><a href="#exonerator">ExoneraTor</a></li>
-          <li><a href="#visitor">VisiTor</a></li>
-        </ul>
-        <br>
-        <a name="metrics-db"></a>
-        <h3><a href="#metrics-db" class="anchor">Metrics data
-        processor</a></h3>
-        <br>
-        <p>The metrics data processor is a Java application that parses
-        Tor's directory data and the data from various other Tor services,
-        possibly removes sensitive parts from them, and then outputs the
-        <a href="data.html">data</a> in <a href="formats.html">formats</a>
-        that are feasible for later analysis.</p>
-        <ul>
-          <li>Browse the
-          <a href="https://gitweb.torproject.org/metrics-db.git/tree">Git
-          repository</a></li>
-          <li><tt>git clone git://git.torproject.org/metrics-db</tt></li>
-        </ul>
-        <br>
-        <a name="metrics-web"></a>
-        <h3><a href="#metrics-web" class="anchor">Metrics website</a></h3>
-        <br>
-        <p>The metrics website software consists of a Java database
-        importer and Tomcat application that makes Tor's directory data
-        easily accessible.  This website is run by the metrics website
-        software.</p>
-        <ul>
-          <li>Download
-          <a href="dist/metrics-web-0.0.1.tar">metrics-web 0.0.1</a>
-          (<a href="dist/metrics-web-0.0.1.tar.asc">sig</a>)</li>
-          <li>Browse the
-          <a href="https://gitweb.torproject.org/metrics-web.git/tree">Git
-          repository</a></li>
-          <li><tt>git clone git://git.torproject.org/metrics-web</tt></li>
-        </ul>
+        <p>The metrics website and related websites depend on a collection
+        of services that measure, archive, process, and finally present Tor
+        network data.
+        This page gives an overview of these services.</p>
+        <img src="/images/tor-metrics-overview.png"/>
         <br>
-        <a name="dirarch"></a>
-        <h3><a href="#dirarch" class="anchor">Directory-archive
-        script</a></h3>
+        <a name="measure"></a>
+        <h3><a href="#measure" class="anchor">Measuring Tor network
+        data</a></h3>
         <br>
-        <p>The directory-archive script consists of a bunch of shell
-        scripts that periodically download relay descriptors, sort them
-        into a directory structure, and compile monthly tarballs. The
-        tarballs are quite similar to the ones provided on the
-        <a href="data.html#relaydesc">Data</a> page, with a few
-        exceptions: the provided tarballs are the result of combining two
-        directory-archive script outputs, splitting v3 votes and v3
-        consensuses into separate tarballs and replacing all colons in
-        filenames with dashes. The <a href="#metrics-db">metrics database
-        software</a> contains similar functionality to the
-        directory-archive script.</p>
+        <p>Tor network data is measured at various places:
         <ul>
-          <li>Browse the <a href="https://gitweb.torproject.org/tor.git/tree/HEAD:/contrib/directory-archive">contrib/directory-archive/</a>
-          subdirectory in the Tor sources</li>
-          <li><tt>git clone git://git.torproject.org/tor</tt></li>
+          <li><a href="https://gitweb.torproject.org/tor.git">tor</a>
+          relays and bridges gather aggregate usage statistics and publish
+          descriptors containing data about Tor network structure and
+          usage.</li>
+          <li><a href="https://gitweb.torproject.org/torperf.git">Torperf</a>
+          is a set of utilities for testing Tor performance from a client
+          perspective.</li>
+          <li><a href="https://gitweb.torproject.org/tordnsel.git">TorDNSEL</a>
+          is a Tor DNS-based exit list that runs periodic checks whether
+          relays use different IP addresses for exiting to the Internet
+          than they use to register in the Tor network.</li>
+          <li><a href="https://gitweb.torproject.org/bridgedb.git">BridgeDB</a>
+          assigns bridges to distributors and gives them out via HTTPS or
+          email.</li>
         </ul>
         <br>
-        <a name="torperf"></a>
-        <h3><a href="#torperf" class="anchor">Torperf</a></h3>
+        <a name="archive"></a>
+        <h3><a href="#archive" class="anchor">Archiving Tor network
+        data</a></h3>
         <br>
-        <p>Torperf is a little tool that measures Tor's performance as
-        users experience it. Torperf uses a trivial SOCKS client to
-        download files of various sizes over the Tor network and notes how
-        long substeps take.</p>
+        <p>All Tor network data is downloaded, possibly sanitized, and
+        then archived by a single tool:</p>
         <ul>
-          <li>Download
-          <a href="dist/torperf-0.0.1.tar">Torperf 0.0.1</a>
-          (<a href="dist/torperf-0.0.1.tar.asc">sig</a>)</li>
-          <li>Browse the <a href="https://gitweb.torproject.org/torperf.git">Git repository</a></li>
-          <li><tt>git clone git://git.torproject.org/torperf</tt></li>
+          <li><a href="https://gitweb.torproject.org/metrics-db.git">metrics-db</a>
+          contains five components for archiving relay descriptors, bridge
+          descriptors, Torperf results, TorDNSEL exit lists, and BridgeDB
+          pool assignments.</li>
         </ul>
         <br>
-        <a name="exonerator"></a>
-        <h3><a href="#exonerator" class="anchor">ExoneraTor</a></h3>
+        <a name="process"></a>
+        <h3><a href="#process" class="anchor">Processing Tor network
+        data</a></h3>
         <br>
-        <p>ExoneraTor parses the relay descriptor archives to answer the
-        question whether some IP address was a Tor relay. This script is
-        available as a Python and a Java version with equivalent
-        functionality. There is also a web version of
-        <a href="/exonerator.html">ExoneraTor</a> available.</p>
+        <p>In some cases, processing and presenting Tor network data is
+        separated for maximum flexibility.
+        In particular, there is currently one tool that processes but does
+        not present Tor network data:</p>
         <ul>
-          <li>Download
-          <a href="dist/exonerator-0.0.2.tar">ExoneraTor 0.0.2</a>
-          (<a href="dist/exonerator-0.0.2.tar.asc">sig</a>)</li>
-          <li>Browse the <a href="https://gitweb.torproject.org/metrics-utils.git/tree/HEAD:/exonerator">Git repository</a>
-          <li><tt>git clone git://git.torproject.org/metrics-utils</tt></li>
+          <li><a href="https://gitweb.torproject.org/onionoo.git">Onionoo</a>
+          provides Tor network status information in JSON format via a
+          RESTful web service.</li>
         </ul>
         <br>
-        <a name="visitor"></a>
-        <h3><a href="#visitor" class="anchor">VisiTor</a></h3>
+        <a name="present"></a>
+        <h3><a href="#present" class="anchor">Presenting Tor network
+        data</a></h3>
         <br>
-        <p>VisiTor is a script that parses a web server log and the exit
-        list archives to tell how many of the requests come from Tor
-        users. VisiTor expects exit lists in the format described in
-        <a href="https://www.torproject.org/tordnsel/exitlist-spec.txt">exitlist-spec.txt</a>.</p>
+        <p>There are currently three main websites and a few additional
+        tools presenting Tor network data:
         <ul>
-          <li>Download
-          <a href="dist/visitor-0.0.4.tar">VisiTor 0.0.4</a>
-          (<a href="dist/visitor-0.0.4.tar.asc">sig</a>)</li>
-          <li>Browse the <a href="https://gitweb.torproject.org/metrics-utils.git/tree/HEAD:/visitor">Git repository</a></li>
-          <li><tt>git clone git://git.torproject.org/metrics-utils</tt></li>
+          <li><a href="https://gitweb.torproject.org/metrics-web.git">metrics-web</a>
+          is the software behind this website, including a database to
+          aggregate most of the presented statistics.</li>
+          <li><a href="https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-6498">task-6498</a>
+          aggregates data to visualize fast exits in the Tor network.</li>
+          <li><a href="https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-2718">task-2718</a>
+          takes the number of daily Tor users as input and attempts to
+          find anomalies that might be indicative of censorship.</li>
+          <li><a href="https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-8462">task-8462</a>
+          estimates daily users from reported directory request
+          statistics.</li>
+          <li><a href="https://gitweb.torproject.org/doctor.git">DocTor</a>
+          is a service that periodically checks the Tor network for
+          consensus conflicts and other hiccups.</li>
+          <li><a href="https://gitweb.torproject.org/atlas.git">Atlas</a>
+          is a web application to discover relays and bridges that uses
+          Onionoo as its data back-end.</li>
+          <li><a href="https://gitweb.torproject.org/compass.git">Compass</a>
+          is a web application that uses Onionoo's data to display
+          information about fast exits in the Tor network.</li>
         </ul>
     </div>
   </div>
diff --git a/web/images/tor-metrics-overview.png b/web/images/tor-metrics-overview.png
new file mode 100644
index 0000000..8ffaca9
Binary files /dev/null and b/web/images/tor-metrics-overview.png differ



More information about the tor-commits mailing list