[or-cvs] [ernie/master] Add tools page listing the currently available and upcoming metrics tools.
karsten at torproject.org
karsten at torproject.org
Tue Feb 16 22:20:52 UTC 2010
Author: Karsten Loesing <karsten.loesing at gmx.net>
Date: Tue, 16 Feb 2010 23:19:46 +0100
Subject: Add tools page listing the currently available and upcoming metrics tools.
Commit: 24f09a78117c5c04bfb199cf0c14927d25102618
---
website/bridge-users-graphs.html | 2 +-
website/consensus-graphs.html | 2 +-
website/data.html | 4 +-
website/docs.html | 43 -----------
website/gettor-graphs.html | 2 +-
website/graphs.html | 4 +-
website/index.html | 6 +-
website/new-users-graphs.html | 2 +-
website/papers.html | 2 +-
website/recurring-users-graphs.html | 2 +-
website/reports.html | 5 +-
website/tools.html | 143 +++++++++++++++++++++++++++++++++++
website/torperf-graphs.html | 2 +-
13 files changed, 159 insertions(+), 60 deletions(-)
delete mode 100644 website/docs.html
create mode 100644 website/tools.html
diff --git a/website/bridge-users-graphs.html b/website/bridge-users-graphs.html
index 07e02ba..9a834d1 100644
--- a/website/bridge-users-graphs.html
+++ b/website/bridge-users-graphs.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
diff --git a/website/consensus-graphs.html b/website/consensus-graphs.html
index 721fb29..ea2f3df 100644
--- a/website/consensus-graphs.html
+++ b/website/consensus-graphs.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
diff --git a/website/data.html b/website/data.html
index 540ccf1..e1edd1f 100644
--- a/website/data.html
+++ b/website/data.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a class="current">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
@@ -32,7 +32,7 @@
that are too sensitive to publish. All signatures can be
<a href="https://www.torproject.org/verifying-signatures">verified</a>
using Karsten's PGP key (0xF7C11265). The following data are
- available (see the <a href="docs.html">Docs</a> section for
+ available (see the <a href="tools.html">Tools</a> section for
details on processing the files):</p>
<ul>
<li><a href="#relaydesc">Relay descriptor archives</a></li>
diff --git a/website/docs.html b/website/docs.html
deleted file mode 100644
index dae4e73..0000000
--- a/website/docs.html
+++ /dev/null
@@ -1,43 +0,0 @@
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
-<html>
- <head>
- <title>Tor Metrics Portal: Docs</title>
- <meta http-equiv=Content-Type content="text/html; charset=iso-8859-1">
- <link href="http://www.torproject.org/stylesheet-ltr.css" type=text/css rel=stylesheet>
- <link href="http://www.torproject.org/favicon.ico" type=image/x-icon rel="shortcut icon">
- </head>
- <body>
- <div class="center">
- <table class="banner" border="0" cellpadding="0" cellspacing="0" summary="">
- <tr>
- <td class="banner-left"><a href="https://www.torproject.org/"><img src="http://www.torproject.org/images/top-left.png" alt="Click to go to home page" width="193" heigth="79"></a></td>
- <td class="banner-middle">
- <a href="/">Home</a>
- <a href="graphs.html">Graphs</a>
- <a href="reports.html">Reports</a>
- <a href="papers.html">Papers</a>
- <a href="data.html">Data</a>
- <a class="current">Docs</a>
- </td>
- <td class="banner-right"></td>
- </tr>
- </table>
- <div class="main-column">
- <h2>Tor Metrics Portal: Docs</h2>
- <br/>
- <p>This page contains all the documentation that is necessary to
- process the files in the <a href="data.html">Data</a> section and
- generate the <a href="graphs.html">graphs</a> and
- <a href="reports.html">reports</a> that can be found on this site.
- This documentation is subdivided by topic:</p>
- <ul>
- <li>... (work in progress)</li>
- </ul>
- </div>
- </div>
- <div class="bottom" id="bottom">
- <p>"Tor" and the "Onion Logo" are <a href="https://www.torproject.org/trademark-faq.html.en">registered trademarks</a> of The Tor Project, Inc.</p>
- </div>
- </body>
-</html>
-
diff --git a/website/gettor-graphs.html b/website/gettor-graphs.html
index c69a794..e25951e 100644
--- a/website/gettor-graphs.html
+++ b/website/gettor-graphs.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
diff --git a/website/graphs.html b/website/graphs.html
index 219dc2b..7106fc6 100644
--- a/website/graphs.html
+++ b/website/graphs.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
@@ -28,7 +28,7 @@
<p>The graphs on this page visualize a small portion of the data
gathered in the Tor Metrics Project. They are generated by a
combination of shell scripts, Java applications, and R code (for
- details see the <a href="docs.html">Docs</a> section). The
+ details see the <a href="tools.html">Tools</a> section). The
following graphs are available:</p>
<ul>
<li><a href="#relays">Relays in the Tor network</a></li>
diff --git a/website/index.html b/website/index.html
index b5caa9d..5c4d722 100644
--- a/website/index.html
+++ b/website/index.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
@@ -41,8 +41,8 @@
analysis of metrics data</li>
<li>Download the <a href="data.html">data</a> that is behind the
graphs and reports to make your own evaluations</li>
- <li>Read the <a href="docs.html">docs</a> explaining how to parse
- and evaluate the metrics data</li>
+ <li>Try out the <a href="tools.html">tools</a> to parse and
+ evaluate the metrics data</li>
</ul>
</div>
</div>
diff --git a/website/new-users-graphs.html b/website/new-users-graphs.html
index e2f1428..0ba52a3 100644
--- a/website/new-users-graphs.html
+++ b/website/new-users-graphs.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
diff --git a/website/papers.html b/website/papers.html
index 8f1f4c1..19d148a 100644
--- a/website/papers.html
+++ b/website/papers.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a class="current">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
diff --git a/website/recurring-users-graphs.html b/website/recurring-users-graphs.html
index dc4410e..70e3fdf 100644
--- a/website/recurring-users-graphs.html
+++ b/website/recurring-users-graphs.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
diff --git a/website/reports.html b/website/reports.html
index cb01363..dfdd80e 100644
--- a/website/reports.html
+++ b/website/reports.html
@@ -17,7 +17,7 @@
<a class="current">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
@@ -28,8 +28,7 @@
<p>The reports linked on this page are designed to interactively
analyze the data gathered in the Tor Metrics Project. These
reports are generated on demand, partially with user-specified
- parameters, using Eclipse BIRT, PostgreSQL, and Java (for details
- see the <a href="docs.html">Docs</a> section). The following
+ parameters, using Eclipse BIRT, PostgreSQL, and Java. The following
reports are available (only pre-generated PDFs and PNGs, because
the BIRT Report Viewer is currently not installed):</p>
<ul>
diff --git a/website/tools.html b/website/tools.html
new file mode 100644
index 0000000..2775fff
--- /dev/null
+++ b/website/tools.html
@@ -0,0 +1,143 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
+<html>
+ <head>
+ <title>Tor Metrics Portal: Tools</title>
+ <meta http-equiv=Content-Type content="text/html; charset=iso-8859-1">
+ <link href="http://www.torproject.org/stylesheet-ltr.css" type=text/css rel=stylesheet>
+ <link href="http://www.torproject.org/favicon.ico" type=image/x-icon rel="shortcut icon">
+ </head>
+ <body>
+ <div class="center">
+ <table class="banner" border="0" cellpadding="0" cellspacing="0" summary="">
+ <tr>
+ <td class="banner-left"><a href="https://www.torproject.org/"><img src="http://www.torproject.org/images/top-left.png" alt="Click to go to home page" width="193" heigth="79"></a></td>
+ <td class="banner-middle">
+ <a href="/">Home</a>
+ <a href="graphs.html">Graphs</a>
+ <a href="reports.html">Reports</a>
+ <a href="papers.html">Papers</a>
+ <a href="data.html">Data</a>
+ <a class="current">Tools</a>
+ </td>
+ <td class="banner-right"></td>
+ </tr>
+ </table>
+ <div class="main-column">
+ <h2>Tor Metrics Portal: Tools</h2>
+ <br/>
+ <p>This page contains a collection of tools that can be used to
+ gather statistics as provided on the <a href="data.html">data</a>
+ page and to process the resulting files to generate
+ <a href="graphs.html">graphs</a>.</p>
+ <ul>
+ <li><a href="#ernie">Metrics portal software</a></li>
+ <li><a href="#descdb">Descriptor database</a></li>
+ <li><a href="#dirarch">Directory-archive script</a></li>
+ <li><a href="#bridgesan">Bridge descriptor sanitizer</a></li>
+ <li><a href="#aggregator">Descriptor aggregator</a></li>
+ <li><a href="#torperf">Torperf</a></li>
+ </ul>
+ <br/>
+ <a id="ernie"/>
+ <h3>Metrics portal software</h3>
+ <br/>
+ <p>The <a href="graphs.html">graphs</a> on this website are
+ generated using ERNIE, the Enhanced R-based tor Network
+ Intelligence Engine (why ERNIE? because nobody liked BIRT; sorry
+ for misspelling Tor). ERNIE consists of Java code that parses
+ the various input data formats and writes CSV files and R code
+ that processes the CSV files to generate graphs. Of course, ERNIE
+ can be used to generate customized graphs without the need to put
+ them on a website.</p>
+ <ul>
+ <li>Browse the
+ <a href="http://gitweb.torproject.org//ernie.git">Git repository</a></li>
+ <li><tt>git clone git://git.torproject.org/git/ernie</tt></li>
+ </ul>
+ <br/>
+ <a id="descdb"/>
+ <h3>Descriptor database</h3>
+ <br/>
+ <p>An alternative approach for analyzing the statistical
+ <a href="data.html">data</a> is to import them into a local
+ database and query it. The descriptor database consist of a
+ database schema for PostgreSQL and/or MySQL, Java code to import
+ data, and example SQL queries.</p>
+ <ul>
+ <li><i>(Work in progress)</i></li>
+ </ul>
+ <br/>
+ <a id="dirarch"/>
+ <h3>Directory-archive script</h3>
+ <br/>
+ <p>The directory-archive script consists of a bunch of shell
+ scripts that periodically download relay descriptors, sort them
+ into a directory structure, and compile monthly tarballs. The
+ tarballs are quite similar to the ones provided on the
+ <a href="data.html#relaydesc">data</a> page, with a few
+ exceptions: the provided tarballs are the result of combining two
+ directory-archive script outputs, splitting v3 votes and v3
+ consensuses into separate tarballs and replacing all colons in
+ filenames with dashes.</p>
+ <ul>
+ <li>Browse the <a href="http://gitweb.torproject.org//tor.git?a=tree;f=contrib/directory-archive;h=095a4216c06b5afc6274eddd39f0238067377fe4;hb=HEAD">contrib/directory-archive/</a> subdirectory in the Tor sources</a></li>
+ <li><tt>git clone git://git.torproject.org/git/tor</tt></li>
+ </ul>
+ <br/>
+ <a id="bridgesan"/>
+ <h3>Bridge descriptor sanitizer</h3>
+ <br/>
+ <p>The bridge authority Tonga keeps a list of bridges in order to
+ serve bridge addresses and descriptors to its clients. Every half
+ hour, Tonga takes a snapshot of the known bridge descriptors for
+ later statistical analysis. As a guiding principle, the Tor
+ project makes all data that it uses for statistical analysis
+ available to the interested public, in order to maximize
+ transparency towards the community. However, the bridge
+ descriptors contain the IP addresses and other contact information
+ of bridges that must not be made public, or the purpose of bridges
+ as non-public entry points into the Tor network would be obsolete.
+ This script takes the half-hourly snapshots as input, removes all
+ possibly sensitive information from the descriptors, and puts out
+ the sanitized bridge descriptors that are safe to be
+ published.</p>
+ <ul>
+ <li>Browse the
+ <a href="https://svn.torproject.org/svn/projects/archives/trunk/bridge-desc-sanitizer/">SVN repository</a></li>
+ <li><tt>svn co https://svn.torproject.org/svn/projects/archives/trunk/bridge-desc-sanitizer</tt></li>
+ </ul>
+ <br/>
+ <a id="aggregator"/>
+ <h3>Descriptor aggregator</h3>
+ <br/>
+ <p>The descriptor aggregator is a Java application that combines
+ the functionality of the directory-archive script (including the
+ postprocessing tasks of merging, splitting, and renaming) and the
+ bridge descriptor sanitizer. The descriptor aggregator runs on a
+ node that has access to non-sanitized bridge descriptors and
+ produces monthly tarballs plus daily updated partial tarballs for
+ the current month.</p>
+ <ul>
+ <li><i>(Work in progress)</i></li>
+ </ul>
+ <br/>
+ <a id="torperf"/>
+ <h3>Torperf</h3>
+ <br/>
+ Torperf is a little tool that measures Tor's performance as users
+ experience it. Torperf uses a trivial SOCKS client to download
+ files of various sizes over the Tor network and notes how long
+ substeps take.</p>
+ <ul>
+ <li>Browse the
+ <a href="https://svn.torproject.org/svn/torperf/trunk/">SVN repository</a></li>
+ <li><tt>svn co https://svn.torproject.org/svn/torperf/trunk torperf</tt></li>
+ </ul>
+ </div>
+ </div>
+ <div class="bottom" id="bottom">
+ <p>"Tor" and the "Onion Logo" are <a href="https://www.torproject.org/trademark-faq.html.en">registered trademarks</a> of The Tor Project, Inc.</p>
+ </div>
+ </body>
+</html>
+
diff --git a/website/torperf-graphs.html b/website/torperf-graphs.html
index 76e89fc..0cbbeaf 100644
--- a/website/torperf-graphs.html
+++ b/website/torperf-graphs.html
@@ -17,7 +17,7 @@
<a href="reports.html">Reports</a>
<a href="papers.html">Papers</a>
<a href="data.html">Data</a>
- <a href="docs.html">Docs</a>
+ <a href="tools.html">Tools</a>
</td>
<td class="banner-right"></td>
</tr>
--
1.6.5
More information about the tor-commits
mailing list